MACIEJ HENNEBERG ^{Note 1}

Summary: A brief historical overview of the origins of peer review reveals that it is neither the best means of evaluating contributions to science nor the one most commonly used during the period in which the modern scientific method developed. Throughout history, most scientists published their views without formal review and peers published their criticisms openly. It is argued here that peer review as now undertaken by most scientific journals stifles scientific communication, slows the advancement of knowledge and encourages dishonest behavior among referees. Alternatives to peer review that have already been used by some journals and funding bodies are described. Since these alternatives have proved themselves in practice, the now commonly practised form of peer review can be abandoned or modified. Electronic communication can facilitate this process.

Science, communication, society
Science is a specific collective human activity (Hull 1988). It is about acquiring an understanding of the world that is of practical value. It is different from both commerce and ideology and hence its practice in society has to be controlled by a separate set of rules and behavioral norms. Exchange of information and opinions among practitioners of science is crucial to its progress. Results of scientific research are useless unless they are communicated to other scientists and the public at large.

Presentation by scientists of the results of their research to others, be the work academic or industrial, is thus essential to the scientific enterprise. Because the nature of scientific work, at least according to Karl Popper (1972), is to approach the truth through a series of increasingly accurate or useful approximations, no piece of research can be held to be definitive. All results are transitional and approximative. Although scientists strive for perfection, by the very nature of their endeavor they cannot achieve perfection. It follows that neither a paper presenting new results of observation or theory nor a proposal for a new investigation can be perfect. They can only be ?good enough.? One of the paradoxes of science, which purports to measure various phenomena with great precision, is that it cannot define a precise measurement of ?good enough.? Assessment of what ?enough? means is left to the judgement of individual scientists. Hence nonspecialists who cannot reach an informed opinion about some theory or discovery often tend to be guided by the consensus opinion of specialists.

Leaving assessment of research results and theories to the judgement of individuals, however well informed in a particular field of expertise, introduces a factor of subjective opinion into seemingly objective science. The ancient Greeks, who originated science as a specific endeavor based on logical argument and empirical tests, realized its susceptibility to subjective opinions. The Greek way of dealing with possibly biased opinions was to conduct open debates between scientists, to produce mathematically exact descriptions and to run empirical tests of statements made by learned persons. Broad discussion and empirical testing were the foundations of learning. Sometimes these were taken too far, as for example in Hellenistic Alexandria, where curious scientists conducted vivisections on condemned criminals (Persaud 1984). Roman ethics put an end to such practices and even forbade dissections of deceased humans. In Roman times, learning became more dependent on written texts than on observation of nature and theoretical arguments. Sanctity of the written word was entrenched by medieval European scholastics.

Science as a human endeavor is also open to social pressures expressed as moral and ethical norms. All human activities need to be organized and regulated by norms of behavior that constrain actions of individuals and attach value to decisions. In other words, individual opinions and actions must be censored and decisions justified.

There exists a basic contradiction between the free search for ideas and their empirical testing on the one hand, and a priori norms of behavior on the other. In the ideal world, free flow of ideas and unrestricted empirical testing should result in the greatest accumulation of knowledge. Empirical tests require time and resources that must somehow be provided by society. No economy is unlimited in size and, hence, no economy can support all possible experiments scientists might think of. Limited resources require that somebody decides which experiments are to be done and which not. Such decisions necessarily involve an element of subjectivity and are constrained by social norms. Scientists in their quest for a practical understanding of the world must navigate narrow straits between social conventions, economic reality and their own human limitations. Not least among these latter are egotism, greed and plain fear.

Historically, it seems that gentlemen-scientists with an elevated social position and independent means, fared the best in science. In the 16th century, at the dawn of the modern era, one such individual, the son of a middle-class family, holder of the respectable office of Canon of the Frombork Cathedral, published a book on the revolution of celestial bodies that revolutionized understanding of our place in the universe. The man was Nicholas Copernicus. His accomplishment was simple: he moved the Earth from the center of the world onto an orbit around the Sun. Being supported by the endowments attached to his post and having received a thorough education at Polish and Italian universities, he had the time and knowledge to conduct astronomical observations and to write a lengthy manuscript in relative peace. He realized the revolutionary nature of his work and dreaded the opinion of many learned colleagues as he clearly spoke against the then accepted views. And yet the book was published. It would not be possible today to publish a book full of minor errors and simplifications and arguing something so ludicrous as a complete reversal of a consensus of well-established authorities. Would it also be possible for a modern scientist to obtain a grant for a proposal based on the supposition that the major publications on the subject are wrong and that he will prove it by conducting observations from the roof of his residence? Absolutely not. Peer review protects us against such lunacies.

Debate of scientific theories and reinterpretation of results is a core of science. This debate benefits from the widest possible participation. This was well recognized by founders of scientific journals in the early centuries of the modern era. Scientific journals and published transactions of learned societies provided means of relatively fast and broad communication of ideas among scientists. Published works were there for everyone to read and it was open to everyone to publish counter arguments. Transactions of many societies included the text of both presented papers and the ensuing debate. All points of view were there in print for everyone to see and comment on. Reading such printed accounts could be bewildering for inexperienced readers, but sharpened their powers of reasoning as each argument had to be absorbed, evaluated and compared with other arguments.

With time it became obvious that not all written matter submitted to journals for publication or to societies for presentation could be published. A certain standard of quality had to be applied. Editors of journals and officers of societies made judgements as to what is acceptable and what not acceptable. Junior or less well-known scientists were not allowed to present their papers at the meetings of various societies, but their papers were introduced by recognized fellows who themselves were elected to fellowships by older and well-established colleagues. Fellowships of the Royal Society and many national academies are awarded in this manner even today.

Editors of journals and fellows of learned societies in the 19th century were usually broadly educated academics. They felt themselves to be well equipped to judge the quality of practically all research reports and theoretical papers submitted to them for publication or presentation. In making such judgements, they also kept in mind the reputation of the journals and societies, and so they also acted as guardians of publicly accepted ethical norms.

It is only natural for humans to evaluate possible consequences of a public statement before making it. Such evaluation, besides obvious reliance on an individual?s knowledge and experience, often tends to be based on the advice of friends, family members and colleagues. Hence many manuscripts of scientific works were read and commented on by the author?s friends and colleagues before being submitted for publication. Such was the regular practice of Charles Darwin, who tested his ideas on his friends before committing them to print (Desmond and Moore 1991).

Peer review today
As the 20th century dawned, the breadth of scientific endeavor became such that there were hardly any persons who could feel competent to evaluate contributions in more than a limited range of subjects. Thus, editors started asking specialists to evaluate papers submitted to journals. The specialists acted in an advisory capacity, and editors still took full responsibility for their decisions. In order to encourage frankness by referees, editors undertook to ensure that they would remain anonymous to the author. With time, overworked editors transferred all work of evaluating manuscripts and suggesting corrections to referees. Referees took make-or-break decisions, but still remained anonymous. There is a commonly held view that refereeing of papers ensures that they become definitive publications, and that what is published must be true and need not be questioned. This is obviously wrong as referees are as prone to errors as are authors. Although it is less likely that a paper will contain major mistakes after it has been scrutinized by several reviewers, errors in published papers are nevertheless common. The proof lies in a number of corrections published in Errata sections of major journals.

The most common form of peer review today is for an editor to send a manuscript to several specialists asking them to answer standard questions about the paper and to provide written comments. The questions nearly always include the one regarding recommendation to publish ?as is,? ?with revisions? or ?reject.? The process used to referee grant proposals is virtually the same, although instead of ?accept? or ?reject? referees are asked to assign a score or a grade to the proposal.

Being asked to referee a paper or a grant proposal anonymously is the dream of anybody seeking power?power without responsibility. Referees acting under such circumstances may be tempted to reject, or delay, publication of papers they disagree with, as their pronouncements regarding the quality of work under review are unlikely to be questioned. The same goes for the refereeing of grant proposals (Goodstein 1996).

Since all the thinking about delaying publication of ?unwelcome? results is done in the heads of anonymous referees, it cannot be examined directly. It can, though, be teased out statistically. Increased reliance of journals on decisions made by reviewers increases time from submission of a paper to its final acceptance, because referees recommend alterations to the content and form of the manuscript. The number and difficulty of proposed alterations may be proportional to the intention to delay publication. Many journals have adopted the practise of printing the ?date submitted? and the ?date accepted? as a footnote to a paper. I have examined the delay between submission and acceptance in two journals: the American Journal of Physical Anthropology (AJPA) and the Medical Journal of Australia (MJA). In 1980, when AJPA first published the information, the average waiting time from submission to acceptance was 6.9 months (Median = 4.8, SD = 4.5) whereas in 1996 it increased significantly ( P < 0.01) to 15.7 months (Me = 11.3, SD = 9.8). Papers published in MJA in 1984 had an average waiting time of 3.5 months (Me = 2.0, SD = 3.3), which increased significantly ( P < 0.05) to 5.6 months (Me = 4.4, SD = 2.5) in 1995. It is strange that these increases occurred during a period in which the quality of research work should have improved and the efficiency of mails and other communication systems increased. Slowdown in communication of scientific results is obviously to the detriment of scientific progress and appears linked to reliance on the process of peer review.

The peer review of grants has its origin in industrial practice. Many people propose to do many things but resources are limited. How can a small number of people who hold the purse strings know everything? The answer is that they cannot, and so they must ask specialists. The best solution is to ask people working on problems similar to the one a grant application proposes to solve. They will be able to highlight merits and point out errors of the proposal. Of course they will be more frank if ensured anonymity. Since people working in similar areas usually know each other and either agree and collaborate or dislike each other and compete among themselves, the result of a review of a particular application will depend on considerations external to the application. These considerations, besides likes and dislikes, also include political and financial elements. What being anonymous and having power means to a referee has been discussed above. A good grantsman will write an application in such a way as to avoid being criticized. It will either be bland, proposing an innocuous piece of research, or it will propose to continue what the author has been successfully (in terms of grants) doing before. Any testing of somebody else?s theory or encroachment into someone else?s area of research creates a high risk of failure.

Paradoxically, some good scientists who do not feel inclined to use anonymous powers to manipulate the system, are not keen to act as referees. This happens to the extent that one of the major science funding agencies now coerces people into refereeing grants by saying that it will automatically refuse grants to academics who do not agree to referee proposals of others. For an honest but busy academic there is very little attraction in refereeing a paper or a grant proposal. It takes time and effort to write a well thought-out assessment and, since it is anonymous, no credit accrues to the reviewer even if, in the process, he comes up with a bright new idea. Authors of papers who receive substantial help from referees certainly recognize their indebtedness, as they often thank referees for their input in the ?Acknowledgments? section of their papers. Referees of grant proposals find themselves in an even worse predicament. Since they referee grants in their own area of interest, they may be strengthening their own competitors if they provide honest criticism that can be used to improve the proposal.

In general, peer review stifles scientific enquiry both by subjecting authors to anonymous critics whose comments cannot be directly challenged, and by self-censorship by authors who hesitate to state the exact rationale or goals of their proposed research for fear that it would not pass the scrutiny of their colleagues. The word colleagues describes referees more truthfully than the word peers, because, although all scientists have a right to do research and express opinions in any area, scientists are not all equal in experience, talent and diligence. Those who are experienced, talented and meticulous, rarely have peers in the sense of equals. How many physicists of the 20th century would consider themselves equals of Albert Einstein? Nevertheless, an enthusiastic and talented, but inexperienced postgraduate student is going to have her work judged by older, much more experienced professors as if her work had been done by a professor. A bit of understanding and encouragement, rather than strict criticism would perhaps help to develop a valuable scientist. As it is, she may be discouraged by rejection of her work following peer review.

Alternatives to current peer review practices
These days, everybody complains about deficiencies of peer review, but few believe that anything can be done about it. How such a view could pass peer review is beyond me. Over the years and in various countries systems of evaluation according to principles other than ?peer review? have been applied with good results. These systems still rely on opinions of academics other than the author, but these opinions are not anonymous or dependent on reviewers ticking boxes or assigning a score.

Until about 20 years ago, many journals relied on editors to make decisions without formal refereeing of papers. Authors were expected to have their manuscripts read and discussed by their colleagues before submission, and it was customary for well-established academics to recommend to the Editor publication of papers written by younger colleagues or students, although this did not guarantee publication. At the same time, students were free to submit papers without professor?s recommendation. Editors, as they became conscious of good work by particular academics, approached them with invitations to publish in their journals. This tradition continues in a limited way even today. For example, only two years ago a respected French colleague approached one of my students to publish his freshly completed PhD thesis in a series of which he was an editor. Such an invitation does not mean that the manuscript will not need to be edited based on direction of the Editor and comments of whomever he has had review the manuscript. The decision, however, is very much out of the hands of peers. Although an editor is, like an anonymous peer reviewer, merely human, his name is known to authors and he has some recognized general policy for his journal. Furthermore, he has a clear interest in the reputation of the journal he edits. These goals are not in conflict with the specific research goals pursued by authors who may wish to publish in his journal.

The editorial system used by leading journals such as Nature run on somewhat similar lines. In the first instance, papers submitted to Nature are evaluated by its editors and only a small number that survive editorial scrutiny are sent for peer review, which therefore takes the form of technical advice to the editor. The extent of editorial control over the process of publishing papers in Nature is evident from the fact that the time from submission to acceptance has not changed over the last 12 years. My calculations based on 283 research papers indicate that the mean delay between submission and acceptance was the same in 1984 (3.1 months, Me = 2.9, SD = 1.7) as it was in 1996 (3.1 months, Me = 2.8, SD = 1.8). The system used by naturalSCIENCE is constructed along similar lines.

The need to allocate research monies unavoidably limits the freedom of scientific exploration, with the result that not all scientists get to pursue the research they want. However, the less the process of allocation of funds stifles scientific freedom the better. Giving money for three-year bits of circumscribed research, a practice that seems now to be quite common, is the worst means of fund allocation. It seems to have arisen from the need to give peace of mind to administrators who cannot make too big a mistake by giving away funds in dribs and drabs to projects having tight schedules and detailed budgets. How can one budget for a discovery? Until the end of 1995, an alternative system was used by the major South African government agency the Foundation for Research Development. In a modified form, combined with project-oriented funding, the same method is still used under the new democratic South African government. The system is based on the principle that good scientists produce even better science when given resources. It relies on a scientist?s track record. A funding submission consists of a Curriculum Vitae with a description of research achievements, concentrating on the most recent past. The candidate is invited to name internationally renowned referees, although the agency may use other persons as well. The submission is evaluated by several international referees in order to establish whether the applicant is an international leader in his field (A), a leader in a specialized area (B), or a scientist who makes regular internationally recognized contributions to his field (C). If none of those descriptions applies, the person is classified D (inactive, no funding will be allocated). There is a special provision for young scientists who have only just completed their doctorate but have proven their worth in other ways (Y). Formerly, an academic was reevaluated every 4 years. Funding was allocated for 4 years based on a category and a one-paragraph general description of the planned research. Now, a more detailed research plan is usually required. The idea is simple?trust a researcher and she will produce results. Similar methods of funding include giving a renowned researcher an endowed chair with research funds attached, or establishing a university research unit for an eminent scholar, which will receive university-funded equipment and staff. In case of younger people, funds such as scholarships or postdoctoral fellowships are allocated by a process based on recommendations by supervisors and evaluation of previous work and experience by variously constituted panels.

It seems ludicrous today to mention past systems in which academic appointments carried a salary designed to cover research expenses, or departmental budgets including lines for research equipment, consumables and travel. And yet, under such arrangements, good research was done because academics were genuinely interested in what they were doing and did not have to rack their brains for ideas that looked acceptable on a grant proposal. Most of the Nobel Prizes won earlier this century were for work done under such circumstances.

There are ways, short of abolishing it, to make peer review less obstructive. The simplest is to take away from reviewers the power to make decisions on a paper or a grant. Instead, they should be asked to provide substantive written comments but not to tick a box recommending a concrete decision or to assign a score. Then the editor or a panel of a grant agency will have to make a decision based on their own assessment of the work and specific points raised by the referees. More work for decision makers, but a fairer deal for authors. The other way is to make referees' names known to the author. This introduces subjective bias, but of a different kind?if a referee wishes to criticize the work he can still do it and it will carry more weight as his personal reputation is at stake. Every time I referee a paper from which names of authors have not been removed (for the purpose of ?blind? refereeing), I insist that the editor reveals my name to the author, even if my comments are very critical. This ensures equality?I know her name, she knows mine.

Generally, it seems that the authors should be free to publish their results and conclusions provided that they are presented in a technically correct manner (which editors can check for themselves), and colleagues should be free to publish their criticisms and derive any credit that may be due for innovative comments, and for exposing themselves to debate. Introduction of electronic means of communication goes a long way toward making possible this ideal of free exchange of scientific information. Cost of electronic publishing is lower and hence the volume of exchanged information can increase. Traceability of electronically published pieces is good, and hence appropriate credit can be given to authors and their work can be formally quoted.

References
Desmond A. and J. Moore. 1991. Darwin. Penguin Books, London, 807 p.

Goodstein, D. 1996. Conduct and misconduct in science. In The Flight from Science and Reason, Annals of the New York Academy of Sciences, Volume 775. Eds. P.R. Gross, N. Levitt, and M.W. Lewis. New York Academy of Science, New York, pp 31?38.

Hull, D.L. 1988. Science as a process: an evolutionary account of the social and conceptual development of science. University of Chicago Press, Chicago, 586 p.

Persaud, T.V.N. 1984. Early history of human anatomy: from antiquity to the beginning of the modern era. C.C. Thomas, Springfield, IL, 200 p.

Popper, K.R. 1972. Objective knowledge: an evolutionary approach. Clarendon Press, Oxford, 395 p.

About the Author
Since 1973, Maciej Henneberg has been engaged in research in biological anthropology and anatomy with special emphasis on human evolution. A native of Poland, he was detained for 100 days by authorities as a Solidarity Union activist and harassed by state police until he left Poland as a refugee in 1984. He has held academic appointments at the A. Mickiewicz University, and the Universities of Texas, Cape Town and Witwatersrand and in 1994 was a Visiting Scholar in the Institute of Biological Anthropology at the University of Oxford. Recently he was appointed the inaugural Wood Jones Professor of Anthropological and Comparative Anatomy, in the Department of Anatomy and Histology, which he heads, at the University of Adelaide. Besides Polish and English, Professor Henneberg has studied both Russian and Italian and understands more than half a dozen other languages. He has conducted excavations in Poland, USA, Italy, South Africa and Namibia and undertaken human population surveys in Poland, USA, South Africa and Australia. He has published over 120 papers in learned journals and his work has been described in many newspaper and magazine articles including articles in the National Geographic, New York Times and New Scientist. His wife Renata is a dental anthropologist and co-author of many of his published works.

Related Articles

naturalSCIENCE Editorial. 1997. Does peer review serves bureaucracy better than science? http://naturalscience.com/ns/articles/edit/ns_ed02.html .

naturalSCIENCE Editorial. 1998. Nature versus NASA: A question of scientific correctness? http://naturalscience.com/ns/articles/edit/ns_ed03.html .

Your comment on this item is invited and should be addressed to: publisher@naturalscience.com . For further information on submitting a contribution to naturalSCIENCE , please see the Author Guide