Genomic Grammar Unlocks Non-Model Organism Mysteries

[THEO] Okay, so here's a sentence that would've sounded like science fiction fifteen years ago: we are teaching machines to read DNA the way they learned to read English. Not just scanning for a gene, but learning the grammar of it.

[DR. MARA] And that word "grammar" is doing real work there. Language models — the technology under ChatGPT — learn by predicting the next word, or a masked-out word, over enormous amounts of text. Nobody tells them the rules of syntax. They infer them. The bet in biology is that genomes have a grammar too, and if you train on enough sequence, a model will pick it up.

[THEO] Right, and biology actually is a sequence problem at heart. DNA is four letters. Proteins are twenty. Those are alphabets. So the leap people made was — what if we treat a genome like a document?

[DR. MARA] But I want to be careful before we get excited. The reason this matters isn't that it's a cute metaphor. It's that most organisms on Earth are not model organisms. We've got maybe a handful we understand well — E. coli, yeast, a few others. For everything else, a non-model microbe pulled out of soil or seawater, we have the sequence and almost no idea what most of it does.

[THEO] It's like getting a library in a language nobody's ever translated.

[DR. MARA] Essentially. And the classical approach to that was homology — you line up a new gene against things you already know, and if it matches something labeled, you copy the label over. That works until you hit a sequence that matches nothing. Then you're stuck.

[THEO] So the dream is a model that's read so much biological sequence it can tell you something useful about a gene it has never seen before. Function, location, how it folds — without a lookup table.

[DR. MARA] That's the arc. And it's worth saying where it starts, because the roots aren't in AI at all. Our earliest paper here is from 2015, and it's not a language model. It's a cancer genomics study — somatic mitochondrial mutations across 527 tumors.

[THEO] The TCGA one. Walk me through why that's the root of this story, because on the surface it looks unrelated.

[DR. MARA] They had DNA and RNA from the same tumors, deep coverage — over five thousm-fold. Mostly the mutation frequencies in DNA and RNA agreed beautifully, correlation of 0.91. But there were fifteen outliers, and twelve of them were transfer-RNA mutations. Every one sat in a stem of the tRNA's folded structure, broke the predicted fold, and the result was unprocessed precursor RNA piling up — up to a hundredfold.

[THEO] So the mutation didn't change the message, it changed the shape. And the cell's machinery reads the shape.

[DR. MARA] Precisely. The processing signal was the secondary structure, not the sequence. That's the quiet lesson that echoes through everything after: the information in a biological polymer is not just the letters in a row. It's the structure the letters fold into, the context around them. You cannot read mitochondrial tRNA by sequence alone.

[THEO] Which is exactly the problem a plain sequence-matcher runs into. So the field needed models that capture context, not just letters.

[DR. MARA] And that's the next turning point — 2022, DeepLoc 2.0. This is where protein language models show up doing something concrete. They take embeddings from models called ESM1b and ProtT5 — these are networks trained on hundreds of millions of protein sequences — and use them to predict which of ten compartments in the cell a protein ends up in.

[THEO] Embeddings being — the model's internal summary of a protein, a long list of numbers that somehow encodes what it's "about."

[DR. MARA] Yes. And the lovely detail is attention. The transformer architecture has this mechanism where it decides which parts of the sequence to focus on. In DeepLoc the attention peaks landed right on the sorting signals — the little address tags that actually route a protein to the mitochondrion or the nucleus.

[THEO] So the model rediscovered the shipping labels on its own. Nobody pointed at them.

[DR. MARA] Nobody did. That's the emergent-behavior theme, and you'll see it again. Now jump to 2024 — the OMG dataset and a model called gLM2. This is where it gets ambitious. Three-point-one trillion base pairs of metagenomic data, scraped from two giant public repositories.

[THEO] Metagenomic meaning — you sequence a whole scoop of environment, all the organisms at once, mixed together.

[DR. MARA] Right, mostly uncultured microbes. And gLM2's trick is that it's mixed-modality. It reads amino acids and nucleotides in the same model, across a stretch of genome with multiple genes in order. So it sees not just one protein but the neighborhood — the genes next door and the regulatory DNA between them.

[THEO] That's the piece a protein-only model physically can't see. In bacteria, genes that work together sit together, in operons. The arrangement is information.

[DR. MARA] And they report it learns coevolutionary signals — which residues on two different proteins are touching at an interface — plus regulatory syntax. Things you only get from reading the whole locus. That's the direct descendant of the 2015 lesson: context carries meaning.

[THEO] Then 2025 is kind of a pile-up of papers all pushing on scale and context. There's WGRL, the whole-genome one.

[DR. MARA] This is my favorite tension in the set. WGRL builds an embedding for an entire bacterial genome and uses it to predict phenotypes — does this bug do X. And it beats the old reliable method, protein-domain presence-absence, on 23 of 25 traits.

[THEO] Which sounds like a clean win for the foundation-model crowd.

[DR. MARA] It does, and yet it's filed as in tension with genomic foundation models — because it's a pointed argument about how you represent a genome. Not just throw raw sequence at a giant model, but model ordered conserved elements. It's a disagreement within the family about what the right unit of "reading" is.

[THEO] A family argument. Love that. And then CodonTransformer is almost the mirror image — instead of reading, it's writing.

[DR. MARA] Right. Trained on a million gene pairs across 164 species, it learns organism-specific codon usage — the fact that different species prefer different synonymous codons for the same amino acid. And when it generates a sequence, it emergently avoids negative cis-regulatory elements. Nobody told it to. Same theme — the model picks up rules nobody wrote down.

[THEO] Which matters enormously if you're trying to express a gene in a non-model host. You want the sequence tuned to that organism's dialect.

[DR. MARA] And the last paper is a review of enzyme annotation — EC numbers, GO terms — arguing the field is moving from hand-built features to generative AI that could propose genuinely new functions. Not relabeling what we know. Finding what we don't.

[THEO] So the through-line, start to finish: structure and context carry the meaning, machines keep rediscovering that on their own, and we're going from reading the library to writing new books in it.

[DR. MARA] With the honest caveat that generating a plausible sequence and generating a functional one are not the same thing. The wet lab still has the final word.

[THEO] A perfect place to hand off. After the break — the mailbag, and someone wrote in to ask us what a genome actually smells like. I'm not kidding.