Preprints and Cardiovascular Science
Prescient or Premature?
Medical journals currently find themselves in the throes of 2 powerful trends: (1) a growing concern for the irreproducibility of scientific reports, and (2) a rising imperative to accelerate dissemination of new knowledge in the digital era. Without question, the individual merit of tackling each issue is readily apparent. Irreproducibility of scientific reports is surprisingly common and contributes to an overall loss of confidence in research by the public and other stakeholders. And who could argue with efforts to leverage new online technologies to distribute information as rapidly and widely as possible?
However, it also is evident that, at times, these 2 challenges of research reliability and timeliness may be at cross purposes. To tackle the former, editors have traditionally insisted on careful, detailed, and time-consuming vetting of a manuscript’s data, methods, and conclusions before publication. This largely occurs through the use of external peer review, and the American Heart Association portfolio of journals has prided itself on its ability to adhere to this high standard for its manuscripts. Efforts to address timeliness involve authors finding new ways to share raw data and preliminary conclusions, often before peer review, in the hopes of accelerating dissemination. Although natural tensions between these 2 challenges have always existed for journals and their authors, from a practical standpoint there really was no conflict. Until recently, it remained virtually impossible for authors to distribute their work without journals.
The emergence of preprint servers in the digital era changes this equation while also raising new questions. For those unfamiliar with them, preprints are online postings of scientific reports in a public venue before their official publication in a journal.1 The types of manuscripts represented by preprints include research articles, editorials, perspectives, and reviews. They are typically formatted as if ready for submission to a journal (and are frequently simultaneously under review at a journal). Some even have been peer reviewed and rejected, but the authors still wish to make the content publicly available. After posting, the manuscripts are usually available online within 24 hours, are given a digital object identifier, and are searchable by services like Google Scholar. For disciplines such as mathematics and physics, the use of preprint servers has been in place for more than 2 decades and is widely regarded as having accelerated information dissemination.
Why would an author post a manuscript on a preprint server? Some benefits are easily evident. In the typical process for scientific reports, preliminary manuscripts are submitted, reviewed, and then resubmitted before publication. This can sometimes go through multiple cycles and to multiple journals, stretching out for months and even years. Ultimately this limits the ability of authors to report findings in a timely manner, and evidence suggests that the process has not improved significantly over time. In a recent analysis reported by Nature of all articles in PubMed up to 2015 with listed submission and acceptance dates, the median time from submission to acceptance has remained ≈100 days, although time from acceptance to publication has decreased.2 The posting of preprints allows findings to be distributed immediately for discussion by the research community, while simultaneously enabling investigators to establish ownership and priority of new ideas.
The use of preprint servers is not without cost, however. In particular, significant concerns have been raised in the life sciences when reporting potentially impacts the evaluation, management, and treatment of humans. This view argues that studies in the life sciences are inherently different from those in disciplines like mathematics and physics. It is possible that most casual visitors to a preprint server are unlikely to take the time, or have the requisite expertise, to critically evaluate a report that could nonetheless affect clinical practice. The extent to which postings are prepared by authors who misinterpret (or overinterpret) their findings heightens this risk. And despite the hope that posting a preprint will spur discussion and debate about a topic, most studies are viewed rarely (if at all).
Some may rightfully point out that these issues are also true for abstracts or preliminary communications at scientific meetings (and even many published articles, for that matter, given that peer review is far from perfect). Yet over time researchers have developed an understanding that abstracts require further validation before being incorporated into clinical practice. The same culture does not exist around preprints at this time. This concern may explain why even in the life sciences, preprints have been adopted much more by basic scientists than by clinical researchers. For example, on bioRxiv right now, there are thousands of postings under topics such as genomics and bioinformatics but only a handful related to clinical trials.
Practical concerns about posting a preprint also exist. Some journals consider preprints as violating the widely adopted Ingelfinger rule, first established by the editors of The New England Journal of Medicine decades ago.3 This rule requires that articles under consideration by a journal are not previously published or under review at another journal. Its overall purpose is to ensure an article’s novel contribution to the literature. The Ingelfinger rule has maintained remarkable durability over the years, prohibiting submission to many journals of manuscripts already posted online as preprints.
However, some journals have begun to move in a new direction and no longer consider preprints as violating the Ingelfinger rule. Even for these journals, some authors have run into tough situations. A prominent example from 2012 suggested that a physics report in Nature was rejected after an initially favorable revise-and-resubmit decision because of the attention drawn to the manuscript as a preprint.4 This implies that journals may be less interested in an author’s work if it has already been made available to the public, especially if it is widely discussed. Pages in medical journals are a limited resource, and editors like to use this resource on science they believe is novel. This is an especially important concern for junior investigators who rely on publications in peer-reviewed journals for promotion and academic credit. A related concern by investigators is the possibility of being scooped by others without appropriate attribution when their work is available only as a preprint.
Notwithstanding these issues, a few preprint servers have been established in the life sciences in recent years and have generated substantial interest. The most prominent of these is hosted by the website bioRxiv, which was established in 2013. Influential journals such as Nature, Science, The Lancet, and The BMJ now explicitly state that they will no longer consider manuscripts posted on preprint servers as prior publications, bypassing the Ingelfinger rule. Simultaneously, funders like the National Institutes of Health and the Wellcome Trust have begun to allow preprints to be referenced in grant applications. The American Heart Association has also updated its policy to allow applicants “to cite interim research products, such as preprints, as part of their grant applications.” The American Heart Association defines preprints in this context as “a complete and public draft of a scientific document” that is “written in the style of a peer-reviewed journal article.” Finally, a growing number of prominent scientists have come out in favor of preprints as a means of accelerating scientific reporting.5
The recent Zika virus outbreak is a telling example of how the research community rallied around the need to push forward scientific reporting during a public health emergency, and the potential future of preprints. Given urgent concerns around that crisis, many journals, including those historically reluctant to consider reports previously disseminated as preprints, agreed to suspend their prior policies. The overall value of getting information quickly into the open for debate and immediate public vetting was believed to outweigh concerns of posting unreviewed clinical data. But if altering this balance between research reliability and timeliness was important for the Zika virus crisis, why should it be less so for other topics or at other times? On the flip side, it is true that much of scientific reporting lacks a degree of urgency comparable to that occurring with the Zika virus outbreak that would require journals to prioritize speed over research reliability.
At Circulation, Circulation: Cardiovascular Quality and Outcomes, and the other journals in the American Heart Association portfolio, these trends have encouraged us to revisit our policies regarding preprints. We believe there undoubtedly will be an increase in authors’ interest in posting manuscripts to preprint servers before submitting their work to our journals for formal publication. Our policy to date has not explicitly addressed this issue. Thus, we engaged in recent discussions regarding our policy and this burgeoning trend. In these conversations, many editors expressed both enthusiasm and concerns related to the preprint movement, largely reflecting the reasons previously mentioned in this article.
Based on these deliberations, organizational leadership established a policy that American Heart Association journals will consider manuscripts that have been previously posted online with authors specifically asked at the time of submission if their work was deposited in a preprint server. The full policy is available online at http://www.ahajournals.org/content/prior-publication-policy. Authors should be aware, however, that there could be implications of posting to preprint servers. For example, we reserve the right to consider how the posting of a preprint might affect decisions around priority, because a key goal of all the American Heart Association journals is to publish articles that are meaningful advances in science. This means that we view the policy as a tool for editors and a guidepost for authors, not an endorsement or repudiation of the preprint server process. We also will allow preprints to be cited in works (like abstracts) during an initial submission, with authors expected to update the reference once a preprint is accepted for publication at a journal. We believe this approach achieves the optimal balance between ensuring the competing demands of research reliability and timeliness while allowing us to learn how preprints will ultimately impact these trends.
As editors, we are highly committed to the joint goals of publishing science of the highest quality and accelerating its dissemination. Our evolving policy regarding scientific reports disseminated as preprints is one important step in achieving these vital goals. As always, we will benefit from your feedback.
J. Schultz provided research material and feedback for this paper. J. de Lemos, H. Krumholz, and J. Burke provided feedback on an earlier draft.
The opinions expressed in this article are not necessarily those of the editors or of the American Heart Association.
Circulation is available at http://circ.ahajournals.org.
- © 2017 American Heart Association, Inc.
- Bourne PE,
- Polka JK,
- Vale RD,
- Kiley R
- Jacobsmeyer B
- Harmon A