Peer review is a general process whereby an article, before being accepted for publication in a scientific journal, is reviewed by several scientists (peers) knowledgeable in the field to which the article pertains. It is considered a cornerstone of modern scientific research.
When a scientist submits a paper to a scientific journal for publication, the editor of the journal selects several reviewers and sends them the paper to review. The reviewers send the editor their comments and questions about the paper. The editor then decides whether to reject the paper, publish it as-is, or send it back to the author with a list of problems to correct before the paper can be accepted.
In many cases, and especially with reputable journals, the editor removes the name of the author before sending the article out to the reviewers. The purpose is to help ensure that the reviewers judge the paper on its own merits, and not on the reputation of its author.
A number of issues are addressed when a paper is reviewed:
- The claim being made, and its merit
- The proposed experiments and methodology for data gathering
- Whether stages of the paper contain bias or assumptions that skew the results
- Whether the quality and quantity of evidence meets the burden of proof
- Whether the analysis and conclusions logically follow from the data
A violation of any one of these areas can invalidate the whole paper, and the author will have to fix the problems.
Strengths of Peer Review
- Peer review helps ensure that grossly inaccurate or poorly-reasoned papers are not published in research journals. Thus, the reader of a peer-reviewed journal is assured that the articles are at least good enough to be considered worthy of publication by several experts in the field.
- Contrariwise, if an article has not been peer-reviewed, this can arouse suspicion: if the paper is any good, why hasn't it been peer-reviewed?
Weaknesses of Peer Review
- Since an article must be read, understood, and reviewed by the reviewers (who are usually working scientists, and thus have full-time jobs already), peer review can delay publication of a paper by months or even years, especially if the editor sends it back for correction. This can delay advances by others in the field.
- Peer review is only as good as the reviewers. These may be unfamiliar with the subject treated in the paper (e.g., if a paper on the social impact of economic policies is submitted to a social science journal, the reviewers may not recognize errors in the parts of the paper that talk about economics).
- Scientists are only human, and a paper that strays too far from the conventional wisdom in a field may be dismissed by experts, especially if the evidence supporting the article's ideas is weak.
- Some creationists and intelligent design proponents have decided to start their own peer-review journals, such as the Progress in Complexity, Information, and Design (PCID), ran by William A. Dembski. Such journals are meant to overcome the common complaint that creationist or intelligent design claims aren't peer reviewed. Such journals are not impartial and are biased, so despite that papers "pass" the peer review within these journals, the process has effectively failed. If the point is to ensure the paper has accurately applied the scientific method, and all the reviewers at the journal habitually bypass the scientific method when it's inconvenient, the journal cannot achieve the goal. The journal itself must have a history of accuracy and impartiality.