Decoding Life's Unwritten Language

[SOFIA] Okay, so here's a question I want you to sit with. What if you could hand a computer the raw genome of some organism nobody's ever studied — no annotation, no experiments — and it could just... read it? Tell you where the genes are, what the proteins do, which ones touch each other?

[DANIEL] That's the dream, yes. And I'd want to see the error bars before I believe it.

[SOFIA] Of course you would! But that's the arc we're tracing today — teaching machines to read DNA. And I mean read in the way you read a language, not just spell out the letters.

[DANIEL] Which is the right analogy to start with, actually, because the whole field borrowed it from natural language processing. The core idea: you train a model on enormous amounts of sequence — protein or DNA — with no labels. You just ask it to predict the next token, or fill in a masked one. And in doing that, it has to learn the grammar.

[SOFIA] The grammar being — the statistical rules of what goes with what. Which codon follows which, which amino acids sit at a fold, which regulatory motifs cluster together.

[DANIEL] Right. And why does that matter to a working biologist? Because most of life is unannotated. We've got beautiful tools for E. coli and yeast and human. But the vast majority of organisms — the ones in soil, in the ocean, the non-model bugs — we have their sequence and almost nothing else. The experiments don't scale to millions of genomes.

[SOFIA] So the pitch is: let the model do the reading experiments can't. And to get everyone on the same page — a couple of terms. When we say "language model" here, we mean the same architecture family behind the chatbots. Transformers. Attention. Trained on sequence instead of English.

[DANIEL] And "MSA" — multiple sequence alignment. You take the same protein from hundreds of species, stack them, and look at which positions change together. If two positions always mutate in tandem, they're probably touching in 3D. That coevolution signal is the fuel behind structure prediction.

[SOFIA] Which is where the story really opens. AlphaFold and RoseTTAFold cracked single-protein structure off exactly that signal. But proteins don't work alone — they grab each other. So the first paper in our arc pushes into interactions.

[DANIEL] This is the 2024 work on human pathogens. And what I appreciate is the engineering honesty. AlphaFold-style prediction is accurate but slow. They built RoseTTAFold2-Lite — a surrogate, roughly twenty times faster.

[SOFIA] Twenty times! That's the difference between screening a handful of pairs and screening everything.

[DANIEL] And they trained it on both interacting and non-interacting pairs — which matters, because a model that's only ever seen things that bind will happily tell you everything binds. You need the negatives to calibrate.

[SOFIA] They screened seventy-eight million candidate interactions across nineteen bacterial pathogens. Landed just over 3,600 high-confidence complexes — and 37% of those were previously unknown. Including the machinery for tRNA modification, urease virulence, outer-membrane biogenesis. First structural look at those in these bugs.

[DANIEL] The speed is the turning point there. It converts a boutique calculation into a screen.

[SOFIA] Okay but then the field asks a harder question — can we do this for DNA itself, not just protein? And this is the good stuff. The nucleotide dependency paper.

[DANIEL] Hm. This one I find genuinely clever. Genomic language models — trained on raw DNA — are black boxes. You get embeddings, you don't know what the model learned. So they did a pairwise substitution trick: mutate one position, mutate another, and ask whether the model thinks those two positions depend on each other.

[SOFIA] Which is the coevolution idea, but now the model plays the role of all those species. You interrogate what it internalized.

[DANIEL] And out fall the functional elements. Transcription factor binding sites. RNA structures — they validated four novel ones in E. coli. As accurately as models trained directly on experimental data. That's the part that survives skepticism for me — an external, wet-lab-validated readout.

[SOFIA] And it also exposed blind spots, right? Some architectures just can't see certain structure.

[DANIEL] Which is the honest footnote the field needed. Not every model reads equally well.

[SOFIA] So now zoom out. Protein models see amino acids. DNA models see nucleotides. Real genomes are both — genes next to regulatory DNA next to other genes. And that's the OMG paper. The Open MetaGenomic corpus.

[DANIEL] 3.1 trillion base pairs, pulled from JGI and MGnify. And the model, gLM2, is mixed-modality — it reads amino acids and nucleotides in the same context window, across multiple genes in a locus.

[SOFIA] Which no protein-only model can do! Because the regulatory syntax lives in the DNA between the genes. gLM2 picks up coevolutionary interface signals and the regulatory grammar in one pass.

[DANIEL] One detail I liked — embedding-space deduplication. Metagenomic data is wildly redundant; you'll have the same organism a thousand times. If you don't dedupe, the model just memorizes the common stuff. They deduplicated in embedding space rather than by exact sequence match. Cleaner training signal.

[SOFIA] And this is where the arc turns from reading to writing. CodonTransformer, 2025. Because once you've learned an organism's grammar, you can generate in it.

[DANIEL] This is the synthetic biology payoff, and it's squarely in your wheelhouse, Sofia.

[SOFIA] It is! So — the problem. You want to express a gene in some host. The amino acids are fixed, but you get to choose the codons. And every organism has its own codon preferences — its dialect. Get the dialect wrong and expression tanks.

[DANIEL] They trained a masked encoder — a BigBird transformer, with a scheme they call STREAM — on a million gene pairs across 164 species.

[SOFIA] And it generates sequences that match the natural codon distribution of whatever host you pick. But here's the part that made me laugh out loud — emergently, it learned to avoid negative cis-regulatory elements. Nobody told it to. Those are little sequence motifs that sabotage expression — cryptic sites, terminators. It just... stopped putting them in.

[DANIEL] Because they're rare in real genes, so a model matching the real distribution avoids them by construction. Emergent is the right word, but there's a mechanism under it.

[SOFIA] Party pooper. But yes, exactly.

[DANIEL] The 2025 review on enzyme annotation frames where all this is going. The field moved from hand-crafted features to deep learning that extracts its own — for predicting EC numbers, what reaction an enzyme catalyzes. And the argument is: the next frontier is generative. Not re-labeling enzymes we already know, but proposing functions we've never seen.

[SOFIA] And then the bookend — the human proteome interaction paper closes the loop with the pathogen one we opened on. Same problem, human scale.

[DANIEL] Thirty petabytes of unassembled genomic data to build MSAs seven times deeper. Depth matters — more sequences, sharper coevolution signal. They screened 200 million human protein pairs, predicted 17,849 interactions at 90% precision. 3,631 of them uncharted.

[SOFIA] So watch the through-line. Deeper data, faster models, from protein to DNA to both together, from reading to writing. Every paper hands the next one a bigger vocabulary.

[DANIEL] And the honest tension across all of it — the models are only as trustworthy as the validation behind them. The four RNA structures, the 90% precision, the calibration on true negatives. That's what keeps this science and not autocomplete.

[SOFIA] Reading the genomes we could never afford to study by hand. Daniel, thank you — we'll pick up the writing half of this next time. Stay with us.