AI Reads The Unannotated Genome

[THEO] Okay, picture this. You hand a computer the entire genome of an organism it's never seen — billions of As, Ts, Gs, and Cs, no labels, no annotations, nothing — and you ask it to just... read. Like handing a kid a library in a language nobody's translated and coming back a year later to find they've figured out the grammar on their own.

[DR. MARA] That's genuinely where the field has landed, and it's worth being precise about what "read" means here. We're talking about language models — the same architecture underneath the chatbots — but trained on DNA and protein sequence instead of English. The idea is that a genome has statistical structure the way language does. Certain letters go together, certain motifs recur, and if a model learns those regularities, it's implicitly learned something about function.

[THEO] And the reason anyone cares — the stakes — is that we can sequence DNA faster than we can understand it. We've got millions of genomes, most from organisms nobody's ever cultured in a lab, and the annotations are mostly guesses copied from a handful of model organisms.

[DR. MARA] Right. E. coli, yeast, human. If your microbe doesn't look like those, you're often flying blind. So the promise is that a model trained across enormous sequence diversity could tell you what a gene does, or how a protein folds, or where a regulatory element sits — without a wet-lab experiment for every single case.

[THEO] So let me set up the vocabulary for anyone coming from, say, immunology or ecology. A "language model" here just predicts missing pieces of a sequence. Mask out some letters, make it guess. Do that a trillion times and it builds an internal representation — an "embedding" — a compressed numerical fingerprint of what that sequence is.

[DR. MARA] And "coevolution" is the other key concept. When two proteins physically touch, a mutation on one side often gets compensated by a mutation on the other, across millions of years and thousands of species. Those correlated changes are a fossil record of contact. That signal is what powered the structure-prediction revolution.

[THEO] Which is honestly where I'd root this whole arc. But you picked an odd starting paper — 2015, mitochondrial mutations in cancer. No language model in sight.

[DR. MARA] Deliberately. I wanted to start before the machines, with a study that shows what "reading DNA" meant the hard way. This group called somatic mitochondrial mutations across 527 tumors, 14 cancer types, comparing the DNA sequence to the RNA. Massive coverage — over five thousandfold. And the DNA and RNA allele frequencies agreed beautifully, correlation of 0.91.

[THEO] Except for a handful of rebels.

[DR. MARA] Fifteen outliers. And twelve of those were transfer-RNA mutations, all sitting in the stems of the tRNA — the base-paired regions that hold the cloverleaf shape. The mutations broke the predicted fold, and the mis-folded tRNAs piled up as unprocessed precursors, up to a hundredfold. The lesson: what the cell reads is the structure, not the letters. The secondary structure is the processing signal.

[THEO] So a decade before the deep-learning wave, you've got this clean demonstration that biology reads shape, not just sequence. Which is exactly the thing a model of raw letters might miss.

[DR. MARA] That's the through-line. Hold that thought, because it comes back.

[THEO] So jump to 2024 and the models arrive in force. The one that grabbed me is RoseTTAFold2-Lite — they took a structure predictor, made it twenty times faster, and pointed it at nineteen human bacterial pathogens.

[DR. MARA] And crucially, they trained it on both interacting and non-interacting protein pairs. That negative signal matters — a model that only ever sees real complexes will hallucinate contacts everywhere. They screened seventy-eight million candidate interactions and resolved over 3,600 high-confidence complexes, more than a third never seen before.

[THEO] Including — and this is the callback — tRNA modification machinery.

[DR. MARA] The same molecule from the 2015 story, now being understood through predicted protein structure. Plus urease virulence factors, outer-membrane biogenesis. Real drug-target-shaped biology in pathogens where we had nothing structural before.

[THEO] But that's still proteins. The next turn is when people said, why not read the DNA itself as language?

[DR. MARA] And there the arc splits into interesting tension. One 2024 paper does something clever with interpretation — nucleotide dependency analysis. Take a DNA language model, substitute pairs of positions, and watch how the model's predictions shift. If changing base A forces a compensating change at base B, the model has learned they're coupled.

[THEO] Which is the coevolution trick, but inside the genome instead of across species.

[DR. MARA] Exactly, and it recovered transcription-factor binding sites and RNA structures — four novel E. coli RNA structures they validated — as accurately as models trained directly on experimental data. So the language model learned structure it was never explicitly taught. That's the 2015 lesson vindicated by a neural net.

[THEO] But it also exposed blind spots, right? Architecture matters.

[DR. MARA] It does. Which sets up the real fight. The OMG dataset paper built gLM2 — trained on 3.1 trillion base pairs of metagenomic data, and here's the innovation, it reads mixed modality. Amino acids and nucleotides, across multiple genes in a locus, at once.

[THEO] So instead of a protein alone, it sees the protein and its neighbors and the DNA between them — gene order, regulatory syntax, the whole neighborhood.

[DR. MARA] And it picks up coevolutionary interface signals a protein-only model simply can't access, because those signals live in the arrangement, not one sequence.

[THEO] Then 2025 throws a punch. The WGRL paper — whole-genome representation learning — beats protein domain presence-absence on twenty-three of twenty-five bacterial phenotypes. But it's flagged as being in tension with the genomic foundation model idea.

[DR. MARA] That's the honest friction in the field. WGRL didn't do the giant undirected foundation-model thing. It modeled ordered conserved elements — the sequence in order — and got whole-genome embeddings that outperformed the standard feature approach. The tension is philosophical: do you need a trillion-parameter model swallowing everything, or does a focused representation of the right structure win? The 2015 paper would nod at that. Order and arrangement carry the signal.

[THEO] And the newest one, CodonTransformer, flips reading into writing.

[DR. MARA] It learns codon usage — the fact that different organisms prefer different synonymous codons for the same amino acid — from a million gene pairs across 164 species. Then it generates sequences matching an organism's natural codon distribution. And emergently, without being told, it avoids negative cis-regulatory elements.

[THEO] Which for anyone building tools in a non-model organism is the dream. You want to express a gene in some weird environmental microbe, and the model writes DNA in that organism's dialect.

[DR. MARA] That's the destination this arc is pointing at. From painstakingly reading structure off RNA pileups in 2015, to models that read genomes unsupervised, to models that write functional sequence in an organism's own idiom.

[THEO] And the open question is the one you keep circling — how much do these models really understand structure versus statistics.

[DR. MARA] The nucleotide dependency work says they can capture it. WGRL says arrangement still matters more than raw scale sometimes. That argument is where the field lives right now, and it's a good place to leave it.

[THEO] We'll pick up the mailbag after the break. Stay with us on The Dish.