Writing with AI

Most of this manual is about using agents for analysis. But analyses end up in papers, and agents can help with the writing too. This chapter covers the many ways AI can contribute to a manuscript, the obligations that come with each, and the principles behind how we approach it.

This chapter is not a statement of what should or should not be written with AI. That is a decision for each investigator to make, in the context of their particular project and goals. How these tools are used in writing, if at all, will vary widely across labs and investigators, and it depends heavily on the project: a methods paper, a review, a dataset description, and a long collaborative study raise different questions.

What we describe instead is what we have learned about what is possible, and how to get the best results. Knowing how the tools work and what they can do matters whatever your position: if you have decided to use them for a project, if you are deciding whether to, if you are mentoring someone who is considering it, or if you definitely do not want to use them and are working out why.

Where analysis ends and writing begins

In practice, the line between using agents for analysis and using them for writing is very hard to draw cleanly, because the products of the analysis are in the manuscript:

Figures are generated by analyses of the data, usually by code an agent helped write.
Numbers in the text, such as sample sizes, percentages, and test statistics, come from the analyses.
Short text summaries of results often begin as the agent’s description of what an analysis found.
Methods sections should describe precisely what the analysis did, and the agent that built the pipeline is well placed to draft an accurate first version of them.

An agent that helped build the analysis has already contributed to the manuscript before anyone drafts a sentence. That is one reason the categories below overlap, and why clear criteria are hard to set.

Ways AI can contribute to a manuscript

Beyond the products of the analysis, agents can help with the writing itself:

Literature searches for the introduction and discussion, surfacing work to read.
Proposing outlines for each section.
First drafts of sections, which authors then rewrite.
Distilling a draft abstract from a nearly finished manuscript.
Copyediting text drafted entirely by hand, for clarity, concision, and consistency.
Feedback while you draft by hand, without incorporating any of its output into the manuscript.
Mock-ups for several journals: versions of the same manuscript adjusted in length, organization, tone, and emphasis, to judge which journal fits best.
Simulated reviews that look for likely problems before submission: unsupported claims, missing controls, unclear figures, the objections an expert in the field would raise.

These differ in an important way. Some put AI-generated text into the manuscript: drafts, abstracts, edited sentences. Others shape the manuscript without contributing any text: feedback, journal mock-ups, and simulated reviews inform your decisions, but every word that reaches the paper is yours. Both kinds are contributions, and both deserve to be discussed and reported, but they raise different questions about authorship and originality.

Choose the model carefully for prose. The ability to write good prose varies widely from model to model, more than the ability to write working code. Weaker models produce text that is generic, padded, or subtly imprecise, which takes longer to fix than to write yourself. For drafting, editing, and feedback on writing, it is generally best to use the most capable model you have access to, even if you use a faster one for routine coding.

The essentials still apply

Using AI in Research sets out the obligations that apply however a manuscript was produced:

You are responsible for everything in it, however it was generated. That covers correctness, originality, and citations. A sentence an agent drafted is your claim once it is in the paper.
Report your AI use clearly and specifically. See Reporting AI use for the frameworks that help.
Follow the policies of the journal, funder, and institution involved. Journal policies differ, for example on AI-generated text and images, and they change. Check the target journal’s current policy before you start writing, not after.

One policy deserves particular care. Simulating reviews of your own manuscript is fine, but manuscripts you review for others are confidential. Many journals forbid uploading them to AI tools. Follow the journal’s reviewer policy.

Talk about it openly

Because these uses have no clear boundaries, criteria are hard to set in advance. That makes open discussion more important, not less. Co-authors need to talk about how AI will be used in a project and agree on it, rather than leaving each person to decide privately.

Discuss it early, when the project starts and again when writing begins. Co-authors may have different expectations and comfort levels, and journal policy sets only the minimum.
Agree on what is acceptable for this project: which of the uses above, by whom, and for which parts of the paper.
Keep a record as you go. A statement of AI use written at submission from memory will be incomplete. Commits that name the model and version, as described in Reporting AI use, together with notes in the plan, make the record as the work happens.
Report what was actually done, in specific terms.

Hiding AI use helps no one. It leaves co-authors unable to vouch for work they are named on, prevents readers from evaluating the work appropriately, and stops the community from developing shared norms. Those norms are still being worked out, and they will be worked out better in the open.

Principles behind our approach

The lab encodes its manuscript conventions in the dunnlab-manuscript skill, and Dunn Lab Practices describes them. Several principles guided them, and they apply whatever tools you use.

Results come from code, not from the model. Every number, table, and figure that reports a result is generated by the analysis code and enters the manuscript automatically, never typed by a person or an agent. An agent asked to copy numbers from analysis output into the text puts the model on the manuscript’s data path: the reported results then depend on the model having copied them correctly. And a typed number is correct only until the analysis is rerun; nothing flags it when it goes stale. Generating results also means rerunning the analysis updates the paper. The agent’s role is to write the code that generates results, and the prose around them.

The text is plain source under version control. We write in plain-text formats, such as Quarto or LaTeX, that Git can track. Every change, whether by you, a co-author, or an agent, appears as a reviewable difference that can be undone. Edits from co-authors who work in Word are carried back into the source as one commit, so the change is a single diff to review.

Methods are written from the code. The workflow is the most accurate record of what was done. Have the agent draft the methods from the actual code, configuration, and software versions, then check the draft against them: every tool, version, parameter, and step. Methods drafted from memory, by a person or a model, drift from what was actually run.

Judgment stays with the authors. An agent can draft, condense, and polish. What the paper claims, how strongly, what it emphasizes, and how the results are interpreted are the authors’ decisions. Read every sentence that reaches the manuscript as if you had written it, because you are answerable for it as if you had.

Getting references right

Early language models were notorious for fabricating references, with plausible authors, titles, and DOIs for papers that did not exist. Current models, especially agents that can search the literature, are much better at finding relevant work and citing it accurately. Errors still happen, though: a garbled author list, a DOI that points to a different paper, or a real paper cited for a claim it does not make. A citation error looks correct until someone checks, so check every reference against a source of truth outside the model.

Use AI literature searches to find things to read. A search can surface relevant work you would have missed, but a paper becomes a citation only once you have checked and read it.
Validate against an external source of truth. Check every entry against an authoritative record, such as Crossref, PubMed, or the publisher’s page, rather than trusting either a model or your own memory. Confirm that each DOI resolves to the work cited, with the correct title and full author list.
Read the references. At minimum, read the parts that support your claim, and confirm the paper supports that claim, for that system, at that strength. Correct citation details do not make a citation appropriate.
Prefer primary sources for findings, and cite reviews as reviews.
Cite the software and data you used, with versions or accession numbers, not only the papers describing them.
Automate the validation. An agent can write a script that looks up every DOI in the bibliography in Crossref and compares the returned title, authors, journal, and year with your entries. Run it whenever the bibliography changes. It catches fabricated and garbled entries; it cannot tell you whether a paper supports your claim. Only reading does that.

The dunnlab-biblio skill encodes these practices for agents, including never filling in a missing field by guessing.