Scholarly Communication and Peer Review: The Current Landscape and Future Trends

wellcomeA report commissioned by the Wellcome Trust focusing on the present state and the future of peer review was published last month. The goal of the report was “to examine the current landscape of peer review for research publications, including recent innovations and how they have worked in practice; and to gather and appraise the views of publishers and others as to how systems and processes may change over the next four to five years.” The report focused on six key areas of likely development in the near future:

  • Acknowledgement of the need to have greater transparency and openness in the review process;
  • Agreement on the desirability of greater interaction between editors, reviewers and authors;
  • Utilization by publishers of more innovative article-level metrics, such as Altmetric, Plum Analytics, and Impact Story;
  • Increasing interest in the provision of rewards in the form of scholarly credit and recognition for reviewers;
  • Desirability to improve guidance, training and feedback for reviewers;
  • Growing interest in distinguishing between the different purposes of peer review.

The full report.

The “Wild West” of Academic Publishing

In the present issue of Harvard Magazine Craig Lambert has a thought-provoking piece on the state of scholarly publishing: The “Wild West” of Academic Publishing: The troubled present and promising future of scholarly communication. He discusses such topics as the precarious economics of most university presses; the growing uncertain relationship between scholarly monograph publishing and the tenure process; the frequently exorbitant cost of many academic journals and the resultant deleterious effect on monograph publishing and acquisition; the swiftly growing Open Access movement and the complementary business models associated with OA publishing; diverse, experimental new models, both print and digital, in scholarly communication.

Bogus Journals Complicate the Open Access Movement

60x60The title of this post is the explanatory subtitle of a very good article that sums up the current state of scholarly publishing and its relationship with open access, in all its complexity.

The article, Some Research Wants to Be Free, Some Follows the Money, is available free in the Annals of Emergency Medicine, a journal published by giant Elsevier.

A few interesting observations:

Some commentators see the effect of the junk journals amplified beyond their actual scale. Biophysicist Cameron Neylon, PhD, advocacy director for PLoS, calls them “relatively small operations” but finds that the scammers’ conduct means that for OA publishing in general, “everyone gets tarred a bit with the same brush … . There’s an awful lot of dodgy behavior happening in subscription space, and people don’t seem to pay any attention to that, but that’s the nature of being the new people on the block.”

The junk publishers, in the PLoS perspective, are “the equivalent of a Nigerian e-mail scam. It’s at that level of sophistication in most cases, and it’s at that level of obviousness. It’s not so much that it’s a real problem; it’s that it is damaging for the industry as a whole.”

Regarding Jeffrey Beall’s list of predatory OA journals:

The binary simplicity of the blacklist, however, calls for caveats. “I would distinguish between low-quality journals and dishonest journals,” cautioned Dr. Suber. “They’re both problems, but they’re separate problems. And the same distinction exists on the subscription side: there are low-quality subscription journals and there are dishonest subscription journals. Some of them are in both categories at the same time. I think Beall’s list doesn’t distinguish low quality from dishonesty.”

From the conclusion:

Suboptimal practices have appeared in OA space, Dr. Neylon suggested, for a host of reasons, many of which are hard to see from inside the existing scientific publication system, eg, the hidden costs built into that system, “the perceived hierarchy of journals and the conservatism that drives the choices of how and where people publish,” the wide variance in the distribution of citations as a measure of how articles are received, the inability of any single publishing format or prestige metric to account for the ways different researchers might find articles useful.

“The biggest barrier to achieving the potential of open access,” he summarized, “is researchers ourselves.” The rise of opportunistic publishers may be a source of inadvertent heuristics, a visible symptom of a communication system awkwardly adjusting to needs that its insiders are only slowly beginning to recognize.

 

The OA “sting”

60x60A recent article in Science by John Bohannon  draws conclusions about the weakness of peer review in open access journals. Predictably, it is controversial, but the controversy has taken an unexpected turn, as commentators point out the flaws in the conclusions.

Briefly — the author submitted a bogus scientific paper with obvious flaws to many OA journals and many accepted it. What disturbs some readers is the apparent effort to suggest that this is just a problem associated with peer-review at OA journals. Several of the journals were published by traditional publishers Elsevier, Wolters Kluwer and Sage. This raises the unexplored question of what would have happened to the submission at those publishers’ traditional journals.

It is also worth noting that the article was rejected by many open access journal such as PLoS One and several Hindawi journals.

An article in The Chronicle of Higher Education contributes analysis and points to many other blog posts and articles that discuss. It is a good starting point for exploring the original article and the reaction.

Peter Suber, in his dispassionate style, points out some facts about the sting:

* Putting some of these together, the article shows that some OA journals are weak and some are strong.

* It doesn’t show that the problem it exposes is limited to OA journals, or intrinsic to OA journals. It exposes a problem with low-quality or dishonest journals, not with OA journals as such.

* It doesn’t show that low quality is non-existent or rare on the subscription side of the line. It merely singles out low quality on the OA side of the line.

 

eLife Peer Review Model

The new open access publication eLife has a unique group of funders (The Howard Hughes Medical Institute, Wellcome Trust, and the Max Planck Society) and employs an interesting new peer review model.
When a submitted article is invited for full peer review, it is assigned to a review editor. The peer reviewers, all active scientists, share and discuss their comments with each other. The review editor uses the comments to write one letter back to the author with instructions.
eLife describes the process:

Reviewers get together online to discuss their recommendations – communicating openly with one another before a decision is reached, refining their feedback, and working to provide clear and concise guidance to authors. If the work needs essential revision before it can be published, the reviewing editor incorporates those requirements into a single set of instructions for the author to move ahead to the next step. We aim to deliver decisions after peer review within four weeks.

Once the final article is published — this Decision Letter is part of the material openly accessible with the article. The Author’s response to the letter is also published, along with reader comments.
This gives the review process unprecedented transparency.

Why Decouple the Journal?

 “Today’s journals are still the best scholarly communication system possible using 17th century technology.”

Jason Priem, altmetrics innovator and creator of such tools as ImpactStory, gives persuasive reasons to “decouple” the journal.

He notes that we have not allowed the web to revolutionize scholarly communication and that “online journals are essentially paper journals delivered by faster horses.”

In addition to using altmetrics as a broader and more meaningful measure of impact, journals could be decoupled. Instead of all journals providing all services separately and redundantly, authors could pick and choose providers of the four major journal functions: dissemination, certification, archiving and registration from various (decoupled) providers. For instance — scholarly societies might provide peer review services and institutional or subject based repositories might provide archiving and registration, while the author might choose to do his own marketing through tweets, blogs and scholarly contacts.

Jason’s own description of this new publishing model is more eloquent. His in depth article has been published in Frontiers in Computational Neuroscience.
Kevin Smith has a recent blog post about his ideas that is helpful as well.