Stealing LLM Reasoning Without Traces

[THEO] Okay, picture this: you've got this incredibly smart friend, a real whiz at puzzles. You ask them a tough question, and they instantly give you the right answer, maybe even a quick summary of *how* they got there. But they don't show you all the scratch paper, all the little steps they took in their head.

[DR. MARA] And you want to know those steps, not just the solution. Because knowing *how* they solved it would help *you* solve similar problems.

[THEO] Exactly! That's the vibe I'm getting from this paper we just got sent in by a listener: "How to Steal Reasoning Without Reasoning Traces" by Zhang, Morris, and Shmatikov, out of Cornell Tech on arXiv. It's looking at large language models, or LLMs, and how even if a model *hides* its internal thought process, you can still figure out its reasoning.

[DR. MARA] This gets to the core of what's often called "distillation" in machine learning. You have a large, powerful "teacher" model, and you want to transfer its capabilities to a smaller, "student" model. Typically, you'd want the teacher to provide not just the final answer, but also its "chain of thought" – the detailed, step-by-step reasoning process it used to arrive at that answer. This is like getting the full solution manual, not just the answers in the back of the book.

[THEO] Right, like if I'm trying to teach a baby AI how to calculate protein folding, I want it to see all the intermediate energy states, not just the final folded structure. But many of these big, proprietary LLMs, especially the commercial ones, don't give you that full chain of thought. They just give you the answer, and maybe a brief summary. It's a black box.

[DR. MARA] Precisely. The authors of this paper wanted to see if that "black box" nature actually protects the reasoning capabilities of these models. Their approach involves what they call "Trace Inversion models." These are models trained to *synthesize* those detailed reasoning traces, even when only given the input, the final answer, and any brief summary from the target LLM.

[THEO] So they're essentially reverse-engineering the thought process? Like, if I just see the ingredients and the finished cake, their model tries to write down the recipe?

[DR. MARA] A good analogy, Theo. They train their inversion model to predict these detailed steps. And what they found was quite striking: the synthetic traces they generated had a high overlap with the *actual* ground-truth reasoning traces, when those were available from open-weight models.

[THEO] Whoa. So they can guess the scratch paper contents really well. And then, here's the kicker, they took those *synthesized* traces – the ones they essentially made up based on the black box's output – and used them to fine-tune smaller "student" models.

[DR. MARA] Yes, and those student models, trained on these inverted traces, showed substantial improvements in their own reasoning capabilities. Crucially, they could even do this with proprietary, black-box models like GPT-5.4 mini, demonstrating that even when a model's full reasoning is hidden, its core logic can still be effectively "stolen" and transferred to other models. It implies that simply withholding the full chain of thought isn't a robust defense against knowledge distillation.

[THEO] So, the secret sauce might not be so secret after all. This has huge implications for how we think about the intellectual property of AI models, doesn't it? If the reasoning can be inferred, what exactly are you protecting? Fascinating stuff.