I have participated in the academic publishing process as an author, referee and associate editor (AE). It is apparent to me that the system is broken and should be -- not just modified -- but completely replaced.
How the current system "works". For those not familiar, when one wishes to publish a paper (at least in mathematics), one submits that paper to a journal. At that point, the editor of the journal assigns an AE to handle the paper. The AE finds (typically) two referees to review the paper. The referees are supposed to read the paper, and write a report to the AE in which they point out potential problems with the paper, make suggestions on how the paper could be improved, and make a recommendation as to whether or not the paper should be accepted as is, revised, or rejected. The AE reads the reports and the paper, and makes his own decision. If the AE decides that a revision is appropriate, he/she provides the authors of the paper with the two referee reports and an AE report. The authors then have a chance to revise and resubmit the paper, at which point the AE and referees review the paper again. The process repeats until the paper is either accepted or rejected by the AE. Throughout the process, the author is kept in the dark as to whom the AE and referees are.
Problems with the current system. Ugh! Where to begin? First, the system is slow. It can two a month or longer for an AE to find two appropriate referees. Referees usually take between 2 - 4 months to write their reports. And the AE may take another month to read the referee reports and make a decision. In all, authors typically wait between 3-6 months before they receive a decision on their paper (though, it is not unheard of to wait over a year). If a paper goes through multiple rounds of revision, it can take up to 3 years to publish a paper.
Second, the quality of the referee (and AE) reports is often very poor. Referees are not paid for their work. So, other than some good will from the AE, there is very little incentive to write a high-quality report. In fact, writing a high-quality report is likely to result in further requests to referee from the same AE. So there is a strong dis-incentive to write a high-quality report. Also, when referees and AEs are anonymous, they do not think carefully enough about what they are writing in their reports because there is no consequence for writing factually incorrect statements. For an analogy, think of your favorite anonymous on-line forum. How well-thought-out are the comments on that forum?
Third, the system is fraught with bias and conflicts of interest. Most academic communities are so small that friends and colleagues often review each other's work. I will not go into specifics, but speaking from anecdotal experience, I can assure you that friends do favors for their friends.
Fourth, articles in academic journals are not free to access. Many academics receive public funding in the form of government grants. Why should the results from these academics' research be limited to those who pay for access to journals?
Fifth, the review process is a huge time-suck on academics' time. There are thousands of articles that are published each year and a huge fraction of them are iterations on a well-known theme. It is not a good use of the Editors, AE's, and referees' time to review these papers.
Is there a better alternative? Rather than try to "fix" the current system, I am in favor of a complete overhaul. There are at least two websites -- arXiv.org and ssrn.com -- where academics (and non-academics) can publish original research without having to go through the refereeing process mentioned above. These websites are a good starting point for an alternative publication system. But, they do not provide a means for authors to receive feedback from others on their work. As such, I would propose augmenting arXiv.org with a reddit-like forum. Below each article, people could point out errors and offer suggestions on how a paper could be improved using their real verified names. Such a system would address many of the problems outlined above.
First, the proposed system would be much faster than current system. Academics could comment on articles as soon as they are posted. And authors could respond to comments as soon as they read them.
Second, if academics are forced to use their real names when they comment on an article, they will be much more likely to think carefully about what they are saying.
Third, papers would be reviewed by a community of people rather than just a few referees, AEs and Editors. This would reduce (though, not eliminate) much of the bias in the current system.
Fourth, articles would be free to access.
Fifth, the system would save time. Articles that offer only incremental contributions would not be reviewed.
Of course, the proposed alternative publication system will certainly have its own problems. But, given the state of the current academic publication process, some experimentation with an alternative is warranted.
No comments:
Post a Comment