Nearly two decades ago, British anesthesiologist John Carlisle published an article on preventing postoperative nausea in the Cochrane Database of Systematic Reviews.
Afterwards, however, Carlisle wasn’t celebrating his publication. Instead, he decided to look more closely at some of the papers included in his literature review. A staggering 68 of the 737 papers reviewed were written by a researcher named Yoshitaka Fujii.
“Peculiar patterns were evident in his papers,” Carlisle tells Analyst News. Rates of headaches or dizziness, for instance, were often precisely the same across groups of patients in Fujii’s clinical trials. “That makes one suspicious. If you see it in not just one paper but lots of papers, then something can’t be right.”
Statistically, Carlisle’s investigation showed, such results were near-impossible — and the data in Fujii’s trials was much more likely fabricated. Indeed, of the 68 papers that Fujii had authored and Carlisle had included, 63 were ultimately retracted from the journals in which they were published.
“We need to be looking out for poor science, whether it’s fabricated or whether it’s unintentionally false,” says Carlisle, a longtime editor for the journal Anaesthesia who has developed statistical techniques to help identify problematic medical research. His methods have been adopted by at least two top medical journals — and have exposed scientific misconduct and errors in hundreds of papers that have been corrected or retracted.
Experts say a rampant culture of “publish or perish,” plus the rise of AI-based writing tools, predatory science journals and paper mills, has tainted scientific publication. Per some estimates, fewer than half of research studies published each year are credible. Such misconduct wastes time and money, damages trust in science, and can even endanger patients.
Last year, more than 10,000 research papers were retracted, marking an all-time high. These retractions could have occurred for a number of reasons: The results in the paper may be considered inaccurate, the paper may have contained plagiarized work, or the authors may have conflicts of interest or used unethical research practices.
The retractions themselves are a good sign, experts say, demonstrating that inaccurate or unethical research is being caught. But scientists worry the retracted articles are only a fraction of all the fraudulent work out there. And a small but growing number of researchers have devoted themselves to investigating and exposing this bad science.
Meet the science sleuths
Microbiologist Elisabeth Bik was a researcher at Stanford University when she stumbled across a few cases of unethical science. First her own work had been plagiarized by another scientist, and then she came across a paper reusing the same image to represent two separate experiments.
A decade later, Bik is now a full-time “science integrity consultant,” sleuthing out fraudulent work in research papers. Her speciality: identifying falsified images.
Bik says one of the biggest sources of fraudulent research is paper mills, which she describes as “networks or cartels of people who make a profit selling fake or very low-quality papers to authors who need an authorship.” Per some estimates, about 2% of papers are produced by paper mills.
“You just can’t work in a field like that. It’s like swimming in garbage. What you have to do is get out of the pool and start trying to clean it up.”
These paper mills may plagiarize PhD theses found online or offer general templates for research articles, making modifications for customers. Some researchers even pay paper mills to help boost their citation index, which indicates how often their work has been cited by other scientists (generally, more established or well-reputed researchers have high citation indices). Paper mills can generate articles that contain multiple citations to a specific scientist’s work and thus serve as “vessels for citations,” Bik says.
Cancer biology professor Jennifer Byrne, the director of biobanking at New South Wales Health Pathology, began researching publication integrity and research fraud after becoming overwhelmed by the amount of bogus research in her field.
“I feel like I didn’t have any choice, because now in the field that I used to work, I would estimate that there are far more paper mill papers than general papers,” says Byrne, who now works with the Association for Interdisciplinary Meta-Research and Open Science (AIMOS), an international organization promoting trustworthy research practices.
The sheer volume of fraud was appalling, she says, with many repetitive papers and fatal errors.
“You just can’t work in a field like that. It’s like swimming in garbage. What you have to do is get out of the pool and start trying to clean it up.”
Why fraud flourishes
In a community that values scientific reasoning and evidence, it may be hard to believe that paper mills and fraudulent research can thrive. But there are financial incentives at play.
For journal editors and publishers, it’s economically advantageous to publish papers, so research is sometimes published without thorough screening. The false information in fraudulent papers can be nuanced or well-disguised, making it difficult for journals to quickly distinguish fraudulent papers from authentic ones.
Experts point to a clear incentivization of quantity, rather than quality, of research output — both with the publishing industry and academic institutions. Publishing articles is critical for success and distinction in scientists’ fields. Universities often require PhD candidates or faculty members to publish a certain number of papers.
Many researchers, however, end up finding negative results — those that do not support their initial hypotheses. While such studies are crucial to the scientific literature, the “whole industry” of journals encourages scientists to publish positive results, Bik explains. Facing pressure to get published but with no compelling results of their own, researchers may turn to paper mills instead.
“The second problem is increasing commercialization of the publication enterprise, where more journals are owned by very, very large companies who are profit-driven,” says Byrne. “Paper mills … are poised to step in when all journals and publishers really care about is that bottom line. Because they will produce for money.”
Part of the problem also lies in the increasing popularity of open-access journals, which do not require readers to pay subscriptions. These open-access journals have helped democratize access to scientific knowledge, experts say, but some publishers are misusing this model and publishing nearly everything they are sent.
“Many open-access publishers publish on a ‘pay-per-paper’ model that drives a much more commercial mindset within publishing,” Bik tells Analyst News. The end result is that journals have compromised the quality of articles for the quantity that they can publish.
‘A work in progress’
Experts say systemic changes are needed to reduce the number of fraudulent papers.
Bik says that journals should move towards publishing more negative results — in fact, understanding negative results in one’s field is critical for researchers to design future experiments of their own.
Governments and regulators, too, can work to remove ads for paper mills across social media sites, she says. And open communication about better science research and research practices — such as through conferences like the ones AIMOS organizes — can enable scientists to together uphold ethical standards in their fields.
Up until recently, it was largely individuals like Carlisle, Bik and Byrne detecting fraudulent research on their own. More recently, larger organizations are playing key roles.
The analytics company Clarivate, for example, maintains a list of reputable journals in its Web of Science platform. Periodically, the list is reviewed, and journals that fail to adhere to Clarivate’s standards are removed. Back in March 2023, about 50 journals were pulled from the Web of Science, including well-known journals like the International Journal of Environmental Research and Public Health, which had published over 17,000 articles the year prior.
“The reality is science is constructed by humans who are doing the best that they can. And it’s a work in progress.”
Public trust in science has eroded in recent years with several high-profile missteps during the COVID-19 pandemic. Still, experts emphasize the need for continuing to trust the scientific process, acknowledging the limitations of scientific publishing while understanding that the majority of scientists are honest and well-intentioned.
In her own work debunking scientific misconduct, Bik treads carefully to ensure her work doesn’t promote dangerous anti-science narratives.
“It’s a double-edged sword because on one hand, I am worried about fraud in science,” she says. “On the other hand, I also think it’s a relatively small fraction, and I do not want to give the impression that all science is fraudulent. This is, I think, the danger of what I do.”
These scientists’ message to the public is to examine scientific information with a critical eye. Consumers of information in any field should strive to look for credible sources, cross-reference studies and consult experts.
The scientific method is a self-correcting system of inquiry that aims to uncover truths about the universe, experts remind. Overcoming the errors that are inevitably introduced to the scientific body of knowledge — whether from human error, bias or misconduct — demands skepticism and scrutiny from both researchers and the public.
“The world that we live in at the moment doesn’t have a lot of certainty — there are a lot of scary things happening,” Byrne says.
“That can lead people to look at science and think, ‘Oh, here’s some certainty at last.’ But the reality is science is constructed by humans who are doing the best that they can. And it’s a work in progress.”
