Scientific papers are designed to communicate knowledge, but using the methods inside them can require far more than reading. Researchers may need to find a code repository, install software dependencies, understand unfamiliar programming interfaces, locate data and reconstruct an analysis workflow before they can apply a published method to a new problem.
A new study published in Nature tested a different model for scientific communication. The researchers developed Paper2Agent, a multi-agent artificial intelligence system that converts a research paper and its associated code into an interactive agent that can answer natural-language requests and execute the paper’s methods.
The large-scale evaluation is notable because the system was not tested only on carefully selected demonstrations. Across 100 computational biology papers, Paper2Agent successfully converted 74 into working agents. Those conversions produced 599 proposed software tools, of which 593 passed automated validation. The study therefore provides both an example of what agent-based scientific publishing could enable and a useful measure of where the approach still fails.
From a static paper to an executable research tool
Paper2Agent is built around the Model Context Protocol, or MCP, a standard that allows AI systems to connect to external tools and resources. Rather than asking a language model to infer a paper’s method from prose alone, the framework attempts to turn the actual research materials into a structured MCP server.
The conversion pipeline first identifies and downloads the codebase associated with a paper. It then creates an isolated software environment, searches the repository for tutorials and examples, executes those tutorials, and records their inputs and outputs. Specialized agents extract generalizable functions from the working examples, parameterize values that were previously hard-coded, and package the resulting functions as tools.
Validation is central to the design. Generated tools are tested against reference outputs from the original code. Tests check whether expected files are produced, numerical results fall within specified tolerances and figures match reference results. Tools that repeatedly fail are excluded rather than being presented as functional.
This matters because a fluent AI response is not the same thing as a reproducible scientific analysis. Paper2Agent attempts to narrow that gap by grounding the agent in executable workflows and by locking validated tools after testing.
The first benchmark showed large gains in accuracy
One detailed case study converted AlphaGenome, an AI model for predicting regulatory effects of genetic variants, into an interactive agent. Paper2Agent generated 22 AlphaGenome tools and all 22 passed automated validation. The conversion took about 45 minutes on a personal laptop and cost approximately US$14, according to the researchers.
The resulting agent was then compared with two alternatives: Claude Code given direct access to the original repository, and Biomni. Two independent human experts graded the outputs using predefined rubrics, with inter-rater agreement of 96.7%.
Across five independent runs, the Paper2Agent version achieved 98.7% accuracy on 15 tutorial-derived queries and 100% accuracy on 15 novel queries. Claude with direct repository access reached 82.7% and 78.7%, respectively, while Biomni reached 37.3% and 56.0%.
The advantage narrowed when the questions became more open-ended, but it did not disappear. On 30 researcher-style queries requiring multiple tools and biological synthesis, Paper2Agent achieved 82.7% accuracy, compared with 56.7% for Claude with repository access and 72.2% for Biomni.
Speed also improved. For tutorial-derived queries, median runtime was 1.9 times faster than Claude with repository access and 3.1 times faster than Biomni. On novel queries, those differences increased to 2.9 times and 3.8 times, respectively.
The larger test is where the result becomes more consequential
A convincing demonstration on one well-documented research project would not establish that the approach generalizes. The researchers therefore processed three broader collections without manual cleanup, code modification or intervention: 100 computational biology papers, 26 data- and discovery-focused papers, and 10 non-biology computational papers spanning fields including artificial intelligence, statistics, econometrics, game theory and astrophysics.
Among the 100 computational biology papers, 74 could be successfully agentified. Across those papers, 593 of 599 proposed tools passed automated validation, equivalent to approximately 99.0% of the proposed tools within the successfully converted set.
That high validation rate should not obscure the other side of the result. Twenty-six of the 100 biology papers could not be successfully converted. Failures were associated with problems such as missing executable code, absent data or model artefacts, dependency and environment failures, and scripts that could not be generalized. In other words, Paper2Agent can automate substantial parts of research reuse, but it cannot manufacture reproducibility when essential research materials are unavailable or unusable.
On 300 tutorial-derived benchmark questions from the broader evaluation, Paper2Agent using Sonnet 4 achieved 91.2% accuracy. Direct repository access with the same model achieved 80.3%, while a newer Sonnet 4.6 baseline reached 86.3%. Both comparisons were statistically significant at P < 0.0001.
The economic and time differences were also substantial. Paper2Agent queries cost about US$0.20 and took 1.6 minutes on average in this benchmark, compared with US$0.38 and 4.3 minutes when Sonnet 4 worked directly with the paper and repository.
It was not limited to computational biology
The researchers also tested 42 execution-based tasks drawn from 10 non-biology computational papers. Across five independent runs, Paper2Agent achieved 98.1% accuracy. This does not prove equivalent performance across all scientific disciplines, but it provides evidence that the underlying conversion strategy is not specific to biological software.
For papers where executable tools could not be created, the framework could still expose research materials through a structured resource layer. Across 26 data- and discovery-focused papers, that resource layer achieved 89.0% accuracy on 100 synthesis questions. A browser-use baseline achieved 82.0%, with the difference reaching P = 0.03. The Paper2Agent approach was also reported to be 34 times cheaper and 15 times faster in that comparison.
These results broaden the significance of the study. The proposed model is not simply an automated code wrapper. It separates executable tools, static research resources and workflow prompts, allowing different parts of a paper to remain useful even when full automation is not possible.
Paper agents could also work together
The study went beyond reproducing existing analyses by connecting agents derived from different papers. In one case study, agents representing AlphaGenome, an MPRA-coupled single-cell CRISPR interference study and a Perturb-seq study were used together to investigate a psoriasis-associated genetic locus.
The AlphaGenome agent ranked GPR137 as the top affected gene, with an RNA-seq quantile score of 0.997. The combined analysis then compared gene-expression signatures across independent experimental datasets. GPR137 knockdown showed significant agreement with the regulatory-element perturbation signature under stimulated conditions, with Spearman correlations of 0.613 at eight hours and 0.630 at 48 hours. The corresponding P values were 3.79 × 10−3 and 4.71 × 10−3, and both survived false-discovery-rate correction.
At rest, the correlation was weaker and not statistically significant, at 0.29 with P = 0.21. The researchers therefore interpreted the evidence for GPR137 as activation-dependent rather than universally present across cellular states.
This example is important because it illustrates a more ambitious possibility. If research outputs become interoperable agents, a scientist could potentially connect a methods paper to a dataset from another study and ask the combined system to test a new hypothesis. That is different from merely asking an AI system to summarize several PDFs.
What the study does not establish
The findings should not be interpreted as evidence that research papers can now be replaced by autonomous AI scientists. The authors explicitly retain a human role in open-ended reasoning, hypothesis selection and evaluation of evidence.
Benchmark accuracy also has limits as a measure of scientific validity. For questions with several defensible answers, matching a reference answer can demonstrate faithful execution without proving that the chosen interpretation is uniquely correct. The authors identify evaluation of multiple defensible answers as an important future problem.
The 74% conversion success rate in computational biology is another practical constraint. Research repositories vary widely in documentation, dependency management and data availability. An agent built today may also require maintenance as software packages, APIs and external dependencies change.
Security, intellectual-property and attribution questions remain as well. Turning code and data into callable tools changes how research outputs can be accessed and recombined, which may create risks that static publication workflows were not designed to manage.
A possible new layer of scientific publishing
The strongest implication of Paper2Agent may therefore be less about any single benchmark and more about the form of a research paper. Modern publications often point readers toward datasets, repositories and supplementary files, but the burden of connecting those pieces still falls largely on the reader.
Paper2Agent suggests that some of this integration can be automated. A publication could become both a document that explains a discovery and an executable interface through which other researchers can interrogate, reproduce and extend it.
The results also expose an uncomfortable but useful test of research infrastructure. If a computational paper cannot be converted because its code is incomplete, its dependencies cannot be resolved or necessary data are missing, the failure says something about the practical reproducibility of the original research. The authors suggest that the ability to create a functioning paper agent could eventually become another indicator of reproducibility.
For now, the evidence supports a more measured conclusion. Automated paper agents can reproduce and apply computational research with high accuracy in selected settings, can outperform general-purpose agents given direct repository access, and can lower the technical barrier to using published methods. But the approach still depends heavily on the quality and availability of the scientific materials it is asked to transform.
Source Information
Study: Reimagining research papers as interactive and reliable AI agents
Authors: Jiacheng Miao, Joe R. Davis, Yaohui Zhang, Jonathan K. Pritchard, James Zou and colleagues
Journal: Nature
Published: 16 September 2026
DOI: 10.1038/s41586-026-11044-y
Article type: Peer-reviewed research article, open access
