You are providing peer review in the disruptive age of the internet (ie you do so without anyone's permission) - personally, I've looked at the "evidence" and I agree with your interpretation of the evidence. This only makes me a skeptic relative to the accuracy of statements of Fitzpatrick et al.
I've been wondering about the implications of this story for the journal Science since I've been hearing rumors that that there is a rebuttal article forthcoming in Science. Unlike a journal like the Auk - that can subject a submission to a review process that can go on for a long time - I've been told that Science makes a "go no go" decision quickly.
I had not noticed this interesting essay by the editor of Science until now and I certainly think it is noteworthy.
I am curious if Science has disclosed what kind of "peer review" the Fitzpatrick story underwent? I have not heard such information. Who were the peer reviewers? The tone of the editor's writing makes it seem like he was "tapped" to "participate" in an event which had already been decided, as much as he was there to pass judgment on a controversial and extraordinary claim. In a way all the "peers" arrived at the journal's door and in unison said, "we concur". This is not peer review.
I also feel a bit uneasy about the quotes in the Arkansas Times article concerning "peer review".
Let's be clear--if anyone is a "peer" who has standing to "review" the findings in the Fitzpatrick et al. paper that Science published, it is Jerome Jackson. He is the definition of "peer" in this case. He isn't some guy in MN with a blog ... for Luneau to dismiss Jackson's comments as "not peer reviewed" ... and for Simon to tell Jackson to "write a science article" rises to the level of contempt for the idea of peer review in the first place. When Jackson says "I think the bird in the video is a pileated for the following reasons", that is peer review.
Peer review is not the "circle" of people who are "brought in early" to "work" this amazing discovery.
In any objective use of the word peer, the man who wrote the IBWP entry in BNA gets to wear the title "peer" ...
Monday, March 06, 2006
Peer review
An anonymous person emailed me the following:
Subscribe to:
Post Comments (Atom)
Posts
5 comments:
As I said before here, the paper would probably have been refereed by 2-3 academic ornithologists (clearly JJ wasn't one of them), which remain anonymous.
Some excellent points here. And, of course, peer reviewers need not "remain anonymous" -- usually they're named in the acknowledgments. It's only in potentially contentious cases (like this one, to be sure) that peer reviewers normally request anonymity. Often, the point of contention is some pre-existing disagreement between the reviewer and the author. By reviewing anonymously, a known critic of the author can provide peer-review without their existing feud getting in the way of everyone's objectivity.
Actually, in ornithology and ecology, peer reviewers selected by journal editors are most commonly anonymous and are rarely named in the acknowledgements.
You are to be applauded for bringing a healthy dose of skepticism to the excitement around what I, and many others, hope is the rediscovery of the ivory-bill. I fear, however, in your zeal to present a contrary point of view you are giving short shrift to the seemingly exhaustive efforts on the part of the Cornell researchers to dissect their own "evidence" and provide supporting conclusions at so many turns. You can disagree with their conclusions, but by all reports, they are actively making their case in the best spirit of scientific discourse I can imagine. The Cornell folks, as far as I can tell, are presenting the evidence they have and making it available for all to analyze and draw their own conclusions. That's the very nature of science. The Jackson piece in The Auk was not a scientific analysis of the evidence presented. It was an opinion piece which jumped from innuendo to thoughtful analysis to person observations. Jackson's piece was, indeed, not peer reviewed. True, Jackson is a "peer," but his Auk article was not a peer-reviewed article in the spirit of the Science article published by Fitzpatrick et al. Does that make Jackson's opinions irrelevant or have less relevant? Certainly not. But, to be fair to the scientific process you appear to believe in, it is important to recognize that Jackson's piece was only his opinion. One presumes Fitzpatrick et al. will have something to say in the next quarterly issue of The Auk. I suspect it will be an interesting bit of discourse. Until then, let's all take a deep breath, remind ourselves we are watching good science, with all its messiness, re-analysis, debate and respectful disagreement. Anything less in this instance would be its own travesty. Onwards!
Jackson could well have been one of the reviewers, as well as Sibley or any of the other present-day critics, and only later formed stronger opinions after more consideration and more time to examine the evidence fully. That is just a conjecture, of course. However, the parts of Jackson's Auk piece that deal directly with the evidence read very much like a critical manuscript review in style and tone: "I am not convinced the bird in the video is anything other than a pileated woodpecker;" "the sightings leave much to be desired;" "you have not ruled out all reasonable alternative explanations for the audio evidence;" etc. (paraphrasing, of course, but I hope you will agree, accurately).
If the rumors are correct, the key players will be having their time in the Arena next week, with a critical paper and the original authors' response. It should be borne in mind that these sots of discussions almost never end with one camp or the other capitulating until and unless conclusive new results come forward. I would not be expecting any retreat on anyone's part in the near future.
Post a Comment