Sign in Subscribe
10 min read

The Argument We Won't Get to Have

AI can now write virus code from scratch, with no family tree at all. The frightening part isn't that someone could build something dangerous. It's that we would have no way to find out if they did.
Share
The Argument We Won't Get to Have
Researchers just recorded the first generative design of complete bacteriophage genomes using genome language models—AI.
The COVID lab leak fight is over. Not solved. Over. AI can now write a virus from scratch, and leave no trail behind.

By Matt Stone

Whether you think COVID came out of a lab in Wuhan or out of a bat in a market, it does not matter anymore.

Not because the question was settled. It wasn't. Because the question stopped being interesting. For five years that fight was about capability and evidence: could a laboratory have built or modified that virus, and did the genome show fingerprints of someone having done it. Both halves of that argument are now obsolete, and they were made obsolete by a paper about a virus that infects nothing but bacteria.

On August 6, a team from Arc Institute, Stanford, and Memorial Sloan Kettering published in Science the first working viruses whose genomes were written by a machine. Not edited. Not assembled from known parts. Generated, the way a language model generates a sentence, and then built and turned loose on living cells.

There is nothing left to believe or disbelieve about whether humans can compose a virus from scratch. We can. It is in the literature. It is peer reviewed.

That is the small part of this story. And something many do not want to contend with is the fact that Covid was actually mild compared to what is possible.

COVID killed roughly one in a hundred people it infected early on, and less than that once we learned how to treat it. That is a mild virus. That is the low end of what nature produces. And that mild virus emptied the shelves, stranded container ships for two years, put a chip shortage into cars that were still missing features three years later, and left hospitals rationing ventilators in the richest country on earth. Ninety-nine out of a hundred people lived and it still did that.

Now move the number. Not to something exotic, just to what already exists in nature. Something that kills five in a hundred instead of one, or that puts twice as many people in a bed for three weeks instead of five days. The people who don’t die still don’t go to work. That’s the part nobody models.

A refinery doesn’t need its workers dead to stop running, it needs them home. Same for the grid, the water plants, the truckers hauling diesel to the generators keeping the vaccine cold. COVID took the slack out of every one of those systems and we never put it back. We ran the drill on easy mode, failed most of it, and then spent five years arguing about where the virus came from instead of fixing the parts that broke.

The next one doesn’t have to be a movie virus. It just has to be slightly worse than the one we already couldn’t handle, and this time nobody will even be able to tell you where it came from.

What they actually did

Samuel King, Brian Hie, and their colleagues took Evo 1 and Evo 2, genome-scale language models trained on DNA the way ChatGPT was trained on text, and fine-tuned them on 14,466 genomes from Microviridae, a family of very small bacterial viruses. They pointed the models at a template: ΦX174, a phage that has been a laboratory workhorse since the 1970s and was the first DNA genome ever sequenced.

The models produced thousands of candidate genomes. The team synthesized 285 of them into physical DNA and dropped them into E. coli to see what would wake up.

Sixteen came alive. They infected the bacteria, replicated, and killed. Some carried between 67 and 392 mutations relative to anything in nature. The most divergent one sat at 93 percent nucleotide identity to its nearest natural relative, which is far enough out that it arguably qualifies as a new species. And when the team mixed the synthetic phages into a cocktail and threw it at E. coli strains that had already evolved resistance to natural ΦX174, the cocktail won in every case, within one to five passages.

That last result is the one the researchers care about, and it deserves to be taken seriously. The GRAM Project's global analysis in The Lancet put 1.14 million deaths in 2021 directly attributable to antibiotic-resistant bacteria, with another 4.71 million in which resistance played a role. Their forecast has annual attributable deaths reaching 1.91 million by 2050, and more than 39 million people dying of resistant infections between 2025 and 2050. Phage therapy is one of the few live ideas for getting ahead of that curve, and engineered phage cocktails that beat resistance in five passages are exactly what the field has been trying to build. This work was not done by reckless people and it should not be described that way.

The honest limit, up front: this is not a demonstration that AI can design a human pathogen, and anyone telling you it is has skipped the methods section. A 5.4-kilobase phage genome with eleven genes, templated on one of the best-characterized organisms in molecular biology, is a narrow and forgiving problem. The hit rate was 16 out of 285, under five percent, with every advantage stacked in the model's favor. The authors say plainly that larger genomes will pose additional challenges they have not solved. If your argument requires the jump from phage to pathogen to be a matter of scale alone, your argument has a hole in it.

It doesn't require that jump. Here is why.

The filter reads for resemblance

Designing a dangerous sequence was never the hard part. The genome of the 1918 influenza virus is public. So is polio's. Both have been reconstructed in laboratories from published sequence data, and that was twenty years ago, before anyone had a foundation model.

The thing that has actually stood between a bad actor and a synthesized pathogen is the DNA synthesis industry's screening layer. Order a sequence from a commercial supplier and software compares it against databases of known hazardous agents before the order ships. That system has real teeth. It is also, at its core, doing one thing: checking whether what you asked for looks like something we already know is dangerous.

Homology. Resemblance. Family.

In October 2025, Bruce Wittmann, Eric Horvitz, and a cross-sector team published the result of a quiet red-team exercise in Science. They used AI protein design tools to generate thousands of reformulated variants of dangerous proteins, ricin among them, and ran the sequences past the biosecurity screening software that commercial synthesis companies actually use. For some source proteins, up to 100 percent of the variants passed undetected through at least one screening tool.

They patched it before publishing. The vendors pushed fixes in July 2024, and the updated software caught 97 percent of the variants that computational metrics suggested would still function.

Read that number the way a security researcher would. Ninety-seven percent is a catastrophic pass rate when the remaining three percent is sufficient. And nobody tested any of these variants in a laboratory. Functionality was inferred from structure-prediction confidence scores, which means the true evasion rate is unknown in both directions.

Horvitz put the problem in one line: "This is about what the sequence does, not just how it looks."

That is the whole vulnerability. Our chokepoint inspects appearance. Generative design changes appearance while preserving function. The phage paper is the proof of concept for the second half of that sentence, running on genomes instead of proteins.

The part nobody is saying out loud

Screening failing on the way in is a containment problem. Serious, and at least legible: you can fund better filters, tighten the vendor consortium, write the sequences into law.

The thing I cannot find a fix for is what happens on the way out.

Every argument anybody made about COVID's origin, on every side, ran through the same instrument. Phylogenetics. How close is this to RaTG13. What does the furin cleavage site imply. Does the codon usage look optimized by a human hand. Are there restriction sites where restriction sites shouldn't be. The lab leak case and the natural spillover case were both built by comparing a genome to its relatives and arguing about the distance.

That argument was possible because SARS-CoV-2 has a family tree.

A generated sequence does not. When the most novel output of a model sits at 93 percent identity to its nearest natural relative and carries hundreds of mutations that no evolutionary process produced, you have severed the reference. There is no baseline to deviate from, which means there is no deviation to detect. The signature of engineering is the gap between what you found and what nature would have made. Erase the comparison and you erase the gap.

We spent five years fighting about whether COVID came from a lab. That fight only happened because there was a virus to study and a family tree to compare it to. AI can now write virus code from scratch, with no family tree at all. The frightening part isn't that someone could build something dangerous. It's that we would have no way to find out if they did.

Oliver Crook, Anemone Franz, and Aaron Maiwald laid out the state of biological attribution in the Bulletin of the Atomic Scientists last December, and it is worse than most people assume even before you add generative design. Their honest summary of the field is that investigators failed to determine the origins of both the 2001 anthrax attacks and COVID despite enormous effort. The most promising forensic tool they describe, an AI system that identifies which laboratory produced a given plasmid, hit 81.9 percent accuracy against a field of more than 1,300 candidate labs in a 2021 challenge. That is genuinely impressive and it is also nowhere near a standard of proof. And they note that some techniques leave nothing at all: modern assembly methods join DNA without scars, and serial passage can push an organism toward a trait using nothing but selective pressure.

So the next one would not be disputed. Disputed requires evidence to dispute. It would be unarguable, by construction.

To be fair, some researchers believe generative models leave their own fingerprints, statistical regularities in the output that betray which model wrote it, the way an AI-written essay has tells. If that holds, attribution doesn't die, it changes instruments. I think that is the strongest objection to everything I have just written, and it is unproven in either direction. It also cuts strangely: a model fingerprint tells you which architecture produced a sequence. It does not tell you who was sitting at the keyboard.

The safeguards were a choice

The Arc team did nearly everything right. They stripped every eukaryote-infecting virus out of the training data and then demonstrated the exclusion actually worked, that the model could no longer perform prediction or design tasks on human viruses. They built computational filters restricting their designs to laboratory strains, and confirmed it: not one of the 285 assemblies infected an off-target E. coli K-12. They chose ΦX174 and non-pathogenic E. coli C precisely because both have decades of safe use behind them.

Their conclusion is that existing biosafety systems adapt fine to generative design, "especially when designs are constrained by well-characterized natural genomes as templates."

Sit with that clause. It is doing all the work.

Every protection on that list is either a discretionary choice by one research group or a property of the specific toy system they selected. Not one of them is a rule. Not one of them binds the next lab, and there will be a next lab, in a jurisdiction of its choosing, with training data of its choosing, working from a template of its choosing.

Thomas Inglesby and Moritz Hanke of the Johns Hopkins Center for Health Security published a companion piece in the same issue, and they were blunt about it: "The ability to compose viral genomes using generative AI now exists; the governance to safely steer it does not." They also state, flatly, that designing new disease-causing viruses should not be pursued at all.

They are describing a gap. Understand what fills it right now. What stands between us and a designed pathogen is not a law, not an inspection regime, not a treaty. It is the judgment of individual scientists about what to leave out of a training set.

What we lost

The lab leak fight was ugly, bad-faith in places, and genuinely useful. It happened because there was something to examine. A genome, its relatives, a chain of custody, institutions that could be subpoenaed and researchers who could be asked questions under oath. We had five years of it and we still didn't reach consensus, which tells you how hard attribution is when everything is working in your favor.

Now imagine the same event with no family tree. No natural relative to compare against. No engineering scar. No evolutionary path that has to be explained. Just an outbreak, and a sequence that came from nowhere, and every party involved able to say with a straight face that nothing in the evidence points at them.

The frightening thing about generative viral design is not that it lets someone build a pandemic. The frightening thing is that it retires the only method we ever had for finding out whether someone did.

We spent five years fighting over what happened. We should have spent them building the ability to know.

The Grounded stands behind every factual claim it publishes with a $100 accuracy guarantee. Find an error of fact and tell us. If you are right, we pay.

Sources

Published by:

Matt Stone

Member discussion