Published on October 15, 2007
Editorial Peer Review: Editorial Peer Review Part of the Solution, or Part of the Problem? Irving E. Rockwood, Editor & Publisher: Irving E. Rockwood, Editor & Publisher 100 Riverview Center, Middletown, CT 06457 A publishing unit of the Association of College & Research Libraries, a division of the American Library Association Founded 1964 Peer Review: What Is It?: Peer Review: What Is It? One of the “checks and balances” whereby scientific communities achieve quality. (Jaron Lanier, Digital Maoism) …a critical, judicious, evaluation performed by one’s equals. (Manuscript Peer Review, Pharmacotherapy 21(4): 395-404) …a process used by journal editors to help select (and improve) manuscripts for publication…aka “Editorial Peer Review” “Traditional” Peer Review: “Traditional” Peer Review One or more reviews by subject matter experts Reviewers selected by editor Reviewer identity not disclosed to author (=“blind” review) Author identity may (or may not) be disclosed to reviewer If not =“double-blind” review Editor has final decision Variations on a Theme: Variations on a Theme “Open peer review”—reviewer identity is disclosed to author Allow author to select, or at least nominate, his or her reviewers Numerous other variations… Common Ground—the Process: Common Ground—the Process Step 1. Editor examines manuscript to determine if it warrants further consideration. Step 2. If yes, editor commissions one or more reviews—simultaneously or seriatim Step 3. Editor determines whether to: Accept the article for publication Return it with invitation to revise and resubmit Reject the article Note: This process applies to books too. The Origins of Peer Review: The Origins of Peer Review More than three centuries old Attributed to Henry Oldenburg, Secretary of the Royal Society of London, and founder of Philosophical Transactions (1665), the “world’s oldest scientific journal in continuous existence,” who introduced the practice of soliciting opinions on manuscripts from colleagues who were more knowledgeable in the area in question. First, Find the Culprits!: First, Find the Culprits! Henry Oldenburg and Philosophical Abstractions, (1665) Journal content now available through JSTOR Oldenburg’s availability unknown A Notable Exception to Peer Review: A Notable Exception to Peer Review Einstein’s “Annus Mirabilis” papers Published in 1905 issue of Annalen der Physik, Max Planck (father of quantum theory and Nobel prize winner) and Wilhelm Wien (also a Nobel prize recipient) Publication process: Planck reads papers Planck publishes them That Was Then, and This Is Now: That Was Then, and This Is Now Editorial peer review essentially synonymous with scholarly journal Peer review norm adopted at different times in different fields, and in different locations In medicine dates from the post WWII era Has itself become an object of study, e.g. five International Congresses on Peer Review and Biomedical Publication—1989-2005 Is Everybody Happy?: Is Everybody Happy? Perhaps not… What’s Wrong? The Critique(s): What’s Wrong? The Critique(s) Peer review is: Unreliable Unfair Fails to validate or authenticate Peer review is: Unstandarized Idiosyncratic Open to every sort of bias But Wait, There’s More…: But Wait, There’s More… Peer review Stifles innovation Perpetuates the status quo Rewards the prominent Peer review Causes unnecessary delay in publication Is very expensive Is insufficiently tested We Could Go On, But We Won’t: We Could Go On, But We Won’t Instead, see the now extensive literature on peer review, e.g. the proceedings of the five International Congresses on Peer Review in Biomedical Publication (JAMA) Drummond Rennie, “Editorial Peer Review: its development and rationale” http://resources.bmj.com/pdfs/rennie.pdf is a particularly useful introductory source The Response: Yes, But…: The Response: Yes, But… Winston Churchill on democracy, “it has been said that democracy is the worst form of government except all those other forms that have been tried from time to time.” Matt Hodgkinson, BioMedCentral editor, “It’s easy to criticize peer review, but it’s harder to come up with a better system.” And Then Came…the Web!: And Then Came…the Web! New Proposals, New Possibilities…: New Proposals, New Possibilities… Open peer review Reviewers forego anonymity Author selection of reviewers Author picks for his or her own list Or from a list provided by publisher Post-publication review Skip the entrance exam Let the marketplace of ideas decide A Web 2.0 idea…? New Models: Case 1: New Models: Case 1 All submitted articles within scope are immediately posted on the Web for a 90 day discussion period At end of “review” period, authors given option to revise; revised article sent out for “pass-fail” review” If “pass,” article is published New Models: Case 2: New Models: Case 2 Authors select reviewers from among BD editorial board members Reviews published alongside author’s responses as part of article Three reviews required New Models: Case 3: New Models: Case 3 Pre-publication review focuses on technical rather than subjective issues All published papers made available for community-based open peer review including online annotation, discussion, and rating Managing Editor, Chris Surridge The Question of the Day: The Question of the Day Is editorial peer review broken, and if so, what should we do about it?