Decoding Genomic Dark Matter

[THEO] Okay, picture this. You hand a computer a genome — just the raw letters, A, C, G, T, billions of them, no annotation, no labels — and you ask it to learn the language the way a kid learns English, just by reading a mountain of it. No grammar book. That's more or less where a big chunk of biology is right now, and it's kind of wild.

[DR. MARA] It's the same trick that powers the large language models everyone's been talking about. You mask out part of a sequence and train the model to predict what's missing. Do that a few trillion times and the model starts to internalize structure — which stretches of DNA go together, which don't.

[THEO] And the promise is huge, right? Because most of the genomes we've sequenced are basically dark. We know the letters, we have no idea what most of them do.

[DR. MARA] That's the core problem. Sequencing got cheap a long time ago. Interpretation didn't. We can read a bacterium's genome in an afternoon and still not tell you what half its genes are for, let alone how they interact or what phenotype they'll produce. The gap between sequence and function is the whole ballgame.

[THEO] So let me set the table for anyone who's a sharp scientist but not in this corner. A few terms. A "language model" here — it's a neural network trained to predict sequence from sequence. A "foundation model" is one of those trained broadly enough that you can reuse it for lots of downstream tasks without starting over.

[DR. MARA] And "self-supervised" is the important qualifier. Nobody's hand-labeling the training data. The sequence supervises itself — the answer is always right there in the part you hid. That's what lets you scale to trillions of bases, because you're not paying humans to annotate.

[THEO] Right. And then there's this beautiful old idea underneath all of it that I think is the real root of this story — coevolution.

[DR. MARA] That predates the deep learning by decades, conceptually. The insight is simple. If two positions in a protein are physically touching, they have to stay chemically compatible. Mutate one, and evolution pressures the other to compensate. So when you line up the same protein across thousands of species, positions that change in a correlated way are probably in contact.

[THEO] It's like — if every time one dancer steps left, their partner steps right, you can infer they're holding hands even if you can't see the hands. Evolution's been running that experiment for a billion years and writing the answers into the sequences.

[DR. MARA] That coevolutionary signal is exactly what AlphaFold and RoseTTAFold learned to read for structure. Which is where I'd actually start the arc, even though our earliest paper isn't about language models at all.

[THEO] Yeah, the 2015 mitochondrial one. Why put that first?

[DR. MARA] Because it makes the deep point that the machines eventually learn without being told: structure is the signal, not sequence. That study called mitochondrial mutations across 527 tumors, comparing DNA and RNA at over five thousandfold coverage. DNA and RNA agreed almost perfectly — correlation of 0.91.

[THEO] Except for a handful of oddballs.

[DR. MARA] Fifteen outliers. Twelve of them were tRNA mutations, and every one sat in a stem — the paired region that holds the tRNA's cloverleaf fold together. Break the fold, and the cell can't process the tRNA. It piled up as unprocessed precursor, up to a hundredfold.

[THEO] So the cell isn't reading the letters. It's reading the shape. A mutation that keeps the shape is fine; one that collapses the stem jams the whole assembly line — including the neighboring genes in the cluster.

[DR. MARA] Which is a preview of the whole field's thesis. Function lives in structure and context, not in the linear string. Hold that thought, because it's what the language models are eventually forced to rediscover.

[THEO] Okay so fast forward almost a decade and the machines show up in force. 2024 is the turning point year. Start with the protein interaction one.

[DR. MARA] RoseTTAFold2-Lite. They built a faster surrogate — about twenty times faster than the full model — and trained it on both interacting and non-interacting protein pairs, which matters, because you have to teach it what "no" looks like too. Then they screened 78 million candidate pairs across 19 human bacterial pathogens.

[THEO] Seventy-eight million. That's the kind of number that only makes sense once the thing is fast.

[DR. MARA] And they resolved over 3,600 high-confidence complexes, more than a third of them previously unknown. Including — I'll note with some satisfaction — the first structural models of tRNA modification machinery in these pathogens. The 2015 paper told us tRNA processing is fragile and structural. This one starts drawing the machines that do it.

[THEO] That's a nice through-line. But that's still protein structure. The leap I love is when people point the same tools straight at the DNA. The nucleotide dependency paper.

[DR. MARA] This one's clever. They take a DNA language model and, instead of trusting it blindly, they probe it. Substitute one base, see how it changes the model's prediction at every other position. Positions that depend on each other light up.

[THEO] So it's the coevolution trick, but run inside the model instead of across species.

[DR. MARA] Exactly. And what lights up are real functional elements — transcription factor binding sites, RNA structures. They validated four brand-new RNA structures in E. coli that way, matching what experiment-trained models find. It's interpretation. The model isn't a black box if you can ask it where its attention hangs together.

[THEO] And it also caught the models being blind in spots.

[DR. MARA] It exposed architecture blind spots, yes. Which is honest, and useful.

[THEO] Then the OMG dataset pushes the scale — 3.1 trillion base pairs of metagenomes, and gLM2, which reads amino acids and nucleotides in the same breath.

[DR. MARA] Mixed-modality. That's the move. A protein-only model never sees the DNA between the genes — the regulatory syntax, the operon arrangement. gLM2 reads the whole locus, protein and the nucleotide context around it, and picks up coevolutionary interface signals a protein-only model simply can't reach.

[THEO] Which brings us to the fight in the room. The 2025 whole-genome paper.

[DR. MARA] WGRL. They model ordered conserved elements across a genome and get embeddings that predict bacterial phenotypes — beat protein domain presence/absence on 23 of 25 traits. But it's listed as in tension with the foundation-model idea, and I think that tension is real. Does scale-it-up-and-let-it-read actually beat a focused, structured representation? On these phenotypes, the focused approach won.

[THEO] So the field's arguing with itself in a healthy way. Bigger versus smarter.

[DR. MARA] And CodonTransformer, also 2025, shows what "smarter" can buy you. Trained on a million gene pairs across 164 species, it learns each organism's codon preferences and generates sequences that match the natural distribution — and, without being told to, learns to avoid the cis-regulatory elements that would gum up expression.

[THEO] Emergent good manners. Nobody wrote the rule; it read enough genomes to absorb it. Which is exactly the 2015 lesson wearing new clothes — the meaning was never in the letters alone.

[DR. MARA] Structure and context. The machines are converging on what the biology told us ten years ago. The open question is whether reading more genomes, or reading them more cleverly, gets us to genomes we can actually design.

[THEO] And on that cliffhanger — after the break, the mailbag. Someone wrote in about tardigrades again.