Given current artificial intelligence (AI) capabilities, does it matter who conducts scientific research? The intuitive response is that a fact is a fact, regardless of who finds it, but knowledge is never disseminated that way.
We receive most scientific claims without any means of checking them ourselves, so we tend to judge not the claim, but the credibility of whoever stands behind it. As the Harvard historian of science Steven Shapin shows, the production of scientific truth has long been inseparable from the social work of persuading peers.
My own research on scientific mobility, conducted a quarter-century ago, pointed in the same direction from a different angle. Reputation in science does not reside in ideas alone; it concentrates in a small number of institutional hubs: departments and laboratories whose standing lends weight to whatever emerges from them.
A scientist leaving a leading center carries part of its credibility along and can, under the right conditions, turn the receiving institution into a center of gravity in its own right.
Credibility has always needed a carrier, and the identity of that carrier has changed over time, often in the face of fierce resistance. When the authorship of the natural order passed from God to a realm open to human inquiry, the resistance was violent. When the individual academic gave way to credentialed institutions, peer review and research assessments, some critics objected that science had become overly bureaucratic and impersonal.
Now, with authorship poised to pass from humans to machines, the objection is framed in terms of attribution, originality and merit.
In all these cases, what we are being asked to trust has changed. Now the system that governs research, from mission-oriented funding to impact assessments, has gradually shifted its attention from whether a result is understood to whether it works, and machine-generated findings suit an enterprise that already values output over understanding.
The irony is that two decades of effort to make data open, searchable and machine-readable was supposed to ease collaboration among human researchers; but it also created the substrate on which autonomous research runs.
AlphaFold, which earned Demis Hassabis and John Jumper a share of the 2024 Nobel Prize in Chemistry, was trained on the Protein Data Bank, the accumulated work of an entire community of structural biologists.
Systems that generate and test their own hypotheses, and then write up the results, are no longer a thought experiment. Last year, a paper written entirely by Zochi, an AI system built by US start-up Intology, passed peer review in the main track of the Annual Meeting of the Association for Computational Linguistics, a top-tier AI conference.
Robin, an AI system built by the San Francisco nonprofit FutureHouse, proposed the hypotheses, designed the experiments and analyzed the results that identified an already-approved drug as a candidate for treating a leading cause of blindness; its work has since appeared in Nature.
Suppose that such a system delivers a new vaccine, catalyst or battery chemistry with no identifiable human author. The public is unlikely to care who did it, any more than patients ask which technician ran their blood test. What they will care about is who to hold responsible if something goes wrong.
A named author can be questioned, corrected, made to retract a claim or discredited; a model cannot. Human authorship was never a guarantee of reliability, as failed replications and retractions attest, but it did always give us someone to hold responsible for a result.
With a few exceptions, leading journals have refused to list AI systems as authors, on the grounds that authorship entails attribution and accountability. However, this merely preserves the old arrangement, rather than solving the new problem.
A survey found that nearly half of scientists (47 percent) use AI daily, and another 31 percent use it weekly. The more capable these systems become, the more widely they would be used. Because most people would not be able to tell good science from bad science, the human scientist would increasingly become the validator, the source of trust that shared knowledge requires.
However, the process of validation is already struggling to keep up. Nearly half of scientists who report saving time with AI then spend more than one-quarter of that time checking its output, while their backlog of untested hypotheses grows.
Moreover, peer review itself is being automated, with an analysis of submissions to this year’s International Conference on Learning Representations suggesting that about one in five reviews was fully AI-generated.
Such findings cast doubt on whether humans would be able to speak for science produced solely by AI. Who, then, would become the trusted party? One possibility is that trust attaches to where the technology was built, with the frontier AI companies becoming the de facto arbiters of what counts as sound science — much as pharmaceutical firms were before independent regulators existed to check their claims.
Another possibility is that the institutions that spent generations accumulating credibility — universities, academies, national laboratories and academic journals — would develop or certify scientific AI agents of their own, so that trust continues to flow through channels the public already has reason to regard as accountable.
Robin’s findings acquired their standing not from FutureHouse’s own announcement, but from peer review at Nature.
Nonetheless, this role would have to be earned, and quickly. Academic journals are already struggling with paper mills, machine-generated submissions and an overstretched system of peer review. The US National Institutes of Health accepts no more than six applications per year from any principal investigator and will not consider applications substantially developed by AI to be applicants’ original ideas.
The outcome would be decided by who moves first to build the infrastructure of accountability: tamper-proof records of every dataset, model version and inferential step behind each result; independent systems tasked with reproducing results without reference to the one that produced it; and, at the end of the chain, a named guarantor, be it a person or an institution, that is liable if the result fails.
The courts have already drawn this line elsewhere. The British Supreme Court has ruled that only a natural person can be named as an inventor on a patent, and a US appeals court has held that copyright requires a human author. Such an arrangement would resemble the way we certify aircraft or approve medicines far more than the way universities award tenure.
Every generation has had to decide who carries scientific credibility, and ours must now weigh this question again.
Sami Mahroum, founder and CEO of Spark X, previously held posts at INSEAD, the Organisation for Economic Co-operation and Development and Nesta.
Copyright: Project Syndicate