Preprints in Biomedical Research: Opportunity or Risk?
Preprints accelerate scientific communication - but COVID-19 showed what happens when unreviewed findings become policy. The case for preprints is real, and so are the risks.
On January 10, 2020, Chinese scientists posted the first SARS-CoV-2 genome sequence to an open database, less than two weeks after the first cluster of pneumonia cases was reported. That act of rapid, open sharing - bypassing the standard months-long journal review process - gave the world the blueprint for diagnostic tests and vaccine development. It was one of the most consequential scientific communications of the century, and it happened before peer review.
What a Preprint Is
A preprint is a complete scientific manuscript posted to a public server before - or sometimes instead of - formal peer review. In biomedicine, the two dominant platforms are medRxiv (for clinical and public health research, launched in 2019) and bioRxiv (for biological sciences, launched in 2013). Preprints are screened for basic scientific content and ethical compliance before posting, but they are not peer-reviewed in the way journals require. They carry a clear watermark stating that they have not been certified by peer review.
The model was well-established in physics and mathematics - arXiv has been running since 1991 - but biomedicine resisted preprints for decades, in part because of legitimate concerns about media uptake of unreviewed medical claims.
How COVID-19 Changed Preprint Culture
The pandemic compressed the usual timeline pressures of research communication into a matter of weeks. Governments and clinicians needed data faster than the journal system could provide. medRxiv postings on COVID-19 reached thousands within months. Many of those preprints were cited in policy documents, news coverage, and even preliminary regulatory guidance before peer review was complete. The result was a natural experiment in what happens when scientific findings are consumed at speed, without the friction that peer review is supposed to provide.
Some preprints were revised substantially after peer review. Some were retracted. The hydroxychloroquine evidence base - which influenced prescribing behaviour and stockpiling in multiple countries - included a substantial number of preprints with methodological problems that peer review later identified. This was not a failure of preprints as a concept; it was a failure of how preprints were consumed and communicated.
The Evidence Synthesis Challenge
For systematic reviewers and evidence synthesisers, preprints present a genuine methodological question: should they be included, excluded, or handled separately? The Cochrane Collaboration has updated its guidance to allow preprint inclusion, provided they are clearly identified and their unreviewed status is noted in the risk of bias assessment. The reasoning is sound - excluding preprints risks publication bias, particularly in fast-moving fields where the most recent evidence has not yet completed peer review.
The risk is also real. Preprints are more likely to contain errors than peer-reviewed publications. Effect sizes in preprints tend to be larger on average than those in the corresponding published versions. Including preprints in a meta-analysis without accounting for their different quality profile can inflate the pooled estimate in precisely the same way that publication bias does.
What Peer Review Does and Does Not Fix
It is worth being honest about what peer review actually delivers. The evidence that peer review reliably identifies errors, fraud, or methodological flaws is weaker than the scientific community tends to assume. Peer reviewers miss things. The process is slow, inconsistent across journals, and subject to its own biases. Journals with higher impact factors are not systematically more rigorous - they are more selective in ways that can introduce their own distortions.
The value of peer review is real but limited. It provides a structured check by domain experts, a mechanism for revision, and a public signal of quality that, imperfect as it is, carries information. A preprint is not worthless before peer review, and a published paper is not validated after it - the honest position is that both require critical appraisal.
Using Preprints Responsibly in Evidence Synthesis
The practical guidance for evidence synthesis teams working with preprints is as follows. Search medRxiv and bioRxiv as part of the grey literature strategy. Include preprints where they are relevant to the question, but flag them explicitly in the PRISMA flow diagram and in the evidence tables. Apply the same risk of bias tools you would apply to any other study, and note separately where the lack of peer review creates additional uncertainty. When a preprint and its corresponding published version are both available, use the published version and note any differences. Conduct sensitivity analyses that include and exclude preprints to test whether their inclusion changes the conclusions.
At NousLab, the acceleration of preprint culture is something we think about carefully in the design of evidence synthesis workflows. The answer is not to pretend that preprints do not exist - in some fields, they represent a substantial portion of the most current evidence. The answer is to handle them with appropriate rigour and to be transparent about the uncertainty they introduce.
Explore how we approach preprint handling as part of our evidence synthesis framework at NousLab features.
