The Pressure To Publish Rapidly: The Risks Vs. Rewards Of Fast-Tracked COVID-19 Studies
By Joseph Menzin, BHE
The speed at which the novel coronavirus has circled the globe has been astonishing. Equally unprecedented is the pace at which COVID-19 research has been released, with over 35,000 papers published between January 2020 and July 4, 2020 alone. With the fear surrounding the pandemic and the pressure to both understand the disease and find a cure, it’s not surprising the publication of scientific papers is being rushed.
Given this unprecedented volume of research and its rapid pace, it can be difficult for even the most seasoned scientists to rapidly determine research quality, study limitations, and the implications of the findings. Add to that the additional scrutiny papers are receiving from the media and politicians, and it’s a recipe for research to be misconstrued, over-hyped or, conversely, completely overlooked.
While time is of the essence as hospitals and health systems, drug regulatory agencies, and the general public prepare for the virus’s second wave, concerns around the dangers of accelerated research studies must also be considered.
Risks Associated With Dissemination Of Rushed Research
It is difficult to implement randomized controlled trials (RCTs) during a pandemic, so the early work on the effectiveness and safety of drug therapies, especially chloroquine and hydroxychloroquine, has focused on observational studies. There have been many single cohort studies of COVID-19 patients from one or a limited number of centers. Common methodologic problems permeate this research and there are dangers in understanding and interpreting findings. For example, without a control group, it’s difficult to know how patients would have fared without a specific treatment, such as hydroxychloroquine. Other problems include the selection of patients for treatment and completeness of follow-up, especially since patients may still be in the hospital when the data are analyzed, so outcomes are unknown.
Due to the condensed time frame, most studies have not been peer-reviewed at the time of release. Thousands of papers have been posted on preprint servers like medRxiv, arXiv, BioRciv, and ChemRxiv, which, while not new, have become far more popular during the pandemic. While these preprint papers are generally considered preliminary and are used by researchers for information sharing and to further scientific discussion, that hasn’t stopped them from being picked up in the media. In fact, the now-infamous paper touting the successful use of hydroxychloroquine in COVID-19 patients was itself a prepublication.
Expressions of concern around COVID-19 studies have made headlines in recent months, including a convoluted sequence of publications and subsequent retractions both espousing and cautioning against the use of chloroquine and hydroxychloroquine as potential COVID-19 treatments. These studies were published in pre-eminent medical journals, New England Journal of Medicine and The Lancet. The consequences of the publication of flawed data are not benign. Following the publication of these studies, clinical trials with hydroxychloroquine were suspended. In other instances, clinical trials in China were suspended due to logistical considerations, but this was deemed to be due to the failure of hydroxychloroquine treatment.
Striking A Balance: Risks Vs. Rewards
While the dangers of rushed research are clear, that’s not to say there aren’t benefits to the rapid reporting of results. With the novel coronavirus, the world faced an unprecedented conundrum: How do we prepare for and react to a virus we know nothing about? Initial findings, though preliminary, were – and still are – critical to fighting disease spread, including information about incubation time, common symptoms, and mechanism of infection.
For example, the drug remdesivir was rapidly put to use before peer-review. While we’re still learning whether remdesivir reduces the mortality rate of patients with severe manifestations of COVID-19, it has been shown to reduce ICU stays, potentially saving lives by freeing up hospital beds. Nonetheless, there still were skeptics questioning the FDA’s emergency authorization of the drug. Recently, British researchers announced that dexamethasone, an inexpensive steroid, has been shown to reduce COVID-19 fatalities by up to 1/3 in ventilated patients. While the news is certainly hopeful, the findings were released via press release and the details of the scientific paper itself have not been published. The anecdotal results are promising, but they raise many questions that have yet to be answered.
Fortunately, mechanisms are being put in place to mitigate flawed research from reaching the public domain and to help researchers rapidly parse through thousands of papers to find relevant, well-designed studies. According to Nature.com, “Without peer review, it’s hard to check the quality of the work, and sharing poor science could be harmful, especially when research can have immediate effects on medical practice. That has led platforms including bioRxiv and medRxiv to enhance their usual screening procedures.”
Additionally, databases like Microsoft’s COVID-19 Open Research Dataset can help researchers organize and identify relevant studies, allowing them to more easily parse through the torrent of information coming at them. Other responses include the 2019 Novel Coronavirus Research Compendium (NCRC), which contains studies curated and vetted by John Hopkins University researchers. Still, other solutions leverage AI, including the CORD-19 dataset, which contains “findings [that] have been extracted… by machine learning algorithms with a human curation overlay.” Access to COVID-19 data combined with rapid analytics platforms is enabling researchers to quickly generate real-world results and drive insights into the virus.
It’s also important for media outlets and policymakers to recognize poorly executed studies, of which there have been many. Guidance on the cautions associated with observational studies versus RCTs have also been published, which could prove instructive while we await results from RCTs of new treatments and vaccines.
A Careful Balance
COVID-19 has challenged us in ways we would never have thought possible. Ultimately, when this pandemic (or at least this first phase) is behind us, there will undoubtedly be lessons learned and key takeaways that will inform how we approach the collection and curation of scientific research.
About The Author
Joseph Menzin is the founder and CEO of BHE, an independent health analytics company with over two decades of experience in designing and conducting research that assesses the value of medical technologies.