The COVID-19 pandemic mobilized scientists around the world to start a flurry of clinical trials so they could better understand the new disease.
However, many studies initiated before mid-May failed to meet quality standards -- particularly in size and reach -- causing concerns that the evidence could be of little meaningful use in guiding the public response, scientists from Stanford Medicine and Yale wrote in a new research letter.
The trials conducted early on gave the medical community more knowledge about COVID-19, Mintu Turakhia, MD, an associate professor of cardiovascular medicine at Stanford, said in an interview. But larger patient studies with more controls will better inform protocols for treating the disease.
"These smaller trials aren't going to give us the best evidence we need," Turakhia, the senior author of the study published this week in JAMA Internal Medicine, said.
Unlikely to produce meaningful evidence
Turakhia and his colleagues looked at 1,551 COVID-19 studies listed in ClinicalTrials.gov, a registry of U.S. and international clinical studies, as of May 19.
In research, best practices call for enrolling large numbers of patients at multiple centers who are randomly assigned to receive either a treatment or a placebo, with neither patients nor caregivers knowing which course the patient is taking.
But among the trials evaluated, fewer than half were randomized, fewer than a quarter took place at more than one center and only about a third included more than 100 patients.
The researchers estimated that, based on trial design, only about 29% of the studies would produce data strong enough to potentially inform changes in clinical care.
"Even before results are known, most studies likely will not yield meaningful scientific evidence at a time when rapid generation of high-quality knowledge is critical," they wrote in the letter.
Maintaining standards
Including multiple sites is critical because patient populations vary at hospitals, as do protocols, Turakhia said in an interview.
He added that many of the studies measured a reduction in symptoms instead of whether the treatments reduced severity of the disease or likelihood of death -- more meaningful measures in a pandemic.
"There's a lot of information published that doesn't move the needle," said Turakhia, who is also the director of Stanford's Center for Digital Health. "We need to maintain our standards and not compromise just because this is a pandemic and the need is so pressing."
In the letter, the authors wrote that "rapid dissemination of studies with low-quality evidence can influence public opinion, government actions and clinical practice in potentially harmful ways."
Research network on standby
Such a problem could be avoided if a broad-scale, rapidly deployable system is set up for coordinating multicenter trials during global health crises, they wrote. They also encouraged institutional review boards to work more closely with investigators to ensure study designs are of high quality.
Consolidating small, single-center trials into a few large, simple, multicenter, randomized trials would provide physicians and public officials with better information, Turakhia asserted.
"These findings really make the case that a scientific response -- a research network that is on standby and ready to be deployed -- should be part of a pandemic preparation plan," he said. "We need to have the large, simple trial machinery ready to go."
An editor's note about the research also sounded a note of caution for scientists studying COVID-19:
We should not sacrifice well-designed trials in the haste to generate new knowledge. Given the high stakes, all of us -- clinicians, researchers, research funders, research regulators, and research publishers -- are responsible for ensuring that the scientific value derived from conducting the trial justifies the risk our patients bear when they participate and their expectations of contributing to societal good.
Photo by Jukka Niittymaa
