LLM-assisted decompilation: tools, not prompts

Matching decompilation has no partial credit. You do not write code that behaves like the original — you rewrite C++ until a specific twenty-year-old compiler, at the original optimization flags, turns your source back into the exact bytes that shipped on the disc. Verified function by function. There is no “close enough.”

Large language models are famously the opposite of byte-exact. They guess, they hallucinate, they pattern-match plausibly and confidently wrong. Pointing them at the strictest task in reverse engineering sounds like the setup to a joke. And yet: this year fleets of Claude Code agents byte-matched 44,226 functions of a stripped retail Xbox 360 binary in about ten weeks — a binary with no symbols, no debug info, where every function starts life as an anonymous address. A sibling title on the same engine reached 92.44% of 32,252 functions. On the Wii build of the same game, one month of this workflow produced nearly three times the commits of the community’s entire two-and-a-half-year history.

The trick is not a better prompt. There is no prompt that makes a language model byte-exact. The trick is that the agents are never allowed to be the judge of anything.

 propose                      judge
+-------+  source   +-----------+  bytes   +---------+
| agent | --------> | real MSVC | -------> | objdiff |
+-------+           +-----------+          +---------+
    ^                                           |
    |      structured diagnosis, not vibes      |
    +-------------------------------------------+

The bet: structured tools, not vibes

The design rule, written into the first project’s README and never relaxed since: agents must never look at raw assembly and guess. Every action an agent takes is mediated by a typed tool that returns structured facts: a match percentage, a behavioral verdict, a struct offset, a root-cause diagnosis. The agents are fast, tireless, and numerous. The tools keep them honest.

Concretely, that meant building a small stack of services that expose a binary as queryable data:

• An orchestrator MCP server wrapping the build system, the diff tool, a headless Ghidra instance, an emulator, and cross-reference databases behind typed calls, backed by SQLite. Agents query and patch through this layer instead of parsing disassembly dumps.
• A production-hardened Ghidra service that survives concurrent agent traffic — after 125 half-closed connections deadlocked the JVM, the fix was one process-wide lock serializing everything. Underneath sits a Ghidra fork adding full p-code semantics for all 77 opcodes of the Xbox 360’s undocumented SIMD extension, validated against 13,836 real instructions. Without that, the analysis backbone silently misdecodes.
• A forked diff tool as the single source of truth. Rather than wrapping a chatbot around an assembly diff, my objdiff fork changes what the diff emits: 21 pattern detectors, six fixability classifications, and machine-readable root-cause diagnoses an agent can act on. Its hardest bug was a nondeterminism in exception-handling pairing that made two byte-identical builds report different scores — corrupting the exact metric the agents optimize.
• Differential execution as triage. A Unicorn-based runner executes the original and recompiled functions side by side on synthesized inputs and machine-classifies every divergence. In one snapshot, 93.1% of ~26,000 tested functions were provably equivalent, and the ~1,500 divergent ones sorted into classes — most harmless codegen noise, just 19 genuinely wrong. The taxonomy lets agents spend effort only on real bugs.
• Copy-on-write worktree pools so dozens of agents can build concurrently without stepping on each other, plus a content-addressed object cache that cut a cold rebuild from about five minutes to 3.5 seconds.

The whole thing runs the real Microsoft compiler on Linux, without Wine, through a Windows PE loader fork that fakes enough of an undocumented COM ABI that the vintage linker produces byte-identical output. On top sit about two dozen slash-command agent skills per project and batch loops that drive sustained work without per-function babysitting.

Decompiling a binary with no names

The stripped retail target posed a bootstrapping problem: with zero symbols, how do you even know what a function is? The answer was a Rosetta Stone. I already had two other decompilations in flight — one sharing the same engine and compiler, so its engine code is named; one being the same game on a different console, so its game code is named. The symbol-less binary sits at the intersection, and a fork of a binary-diffing tool — which reimplemented Ghidra’s undocumented version-tracking math in Python — triangulated between the two parallel texts, shipping 978 vetted symbol identities across compilers at 0.933 holdout precision.

The single best day came from a linker insight, not an AI one. The retail linker scatters each function’s section freely across the binary, so my splitter had been attributing functions to the wrong translation units and the diff tool never paired them. The functions looked unmatchable; the source was right; the bookkeeping was lying. Reclassifying that pool produced +1,176 strict matches in one day. Agent throughput amplifies human insight; it does not replace it.

When the metric lies

The last few percent of any function is brutal. The code can compile cleanly and behave correctly, yet fail to match because the retail compiler allocated registers in a different order. There is no bug to fix — the code is right — so closing the gap is a search problem over behavior-preserving rewrites. I extracted that search into a standalone engine with 140 self-registering source-transform patterns and multiple strategies — hill-climbing, beam search, an evolutionary mode — all scored against the real build.

Then it cheated. Under a naive “accept if the score improves” rule, the engine shipped two genuinely wrong rewrites that its own fitness function rewarded. One rewrote a geometry term from (sum) * 0.5f to 0.5f + sum — equal only when the sum happens to be exactly −1 — and scored +3.25 points for it. Another swapped the arguments of a non-commutative call for a rounding-noise gain. Both were traced by hand to the exact scoring mechanics that let a wrong body outscore a correct one.

This is textbook reward hacking, caught in the wild in my own system, and it hardened into doctrine: a fuzzy score is an adversary; the compiler’s byte-exact output is the sole terminal judge; behavior classifiers are advisory, never gates. The same rule covers the agents themselves. A capability sweep found a frontier model could produce whole-function byte-exact output in only 12% of informative cases — which is exactly why the models propose and the deterministic oracles dispose. Several ambitious search strategies were A/B-tested, produced zero genuine wins, and were retired by their own pre-registered kill criteria; negative results are committed as first-class artifacts. (The model findings that survived — including the harness bug that briefly had me believing exactly the wrong thing about reasoning models — are in the flagship post.)

Rebuilding the referee

Every loop in the system ends in compile-and-compare, which makes the compiler the bottleneck: run through the compatibility shim, the backend tops out around 3,000 objects per second. When a search wants to score millions of candidates, that is the wall.

The way through was noticing that MSVC is secretly two programs with a drivable interface in the middle: a front end that writes an intermediate language to disk, and a backend DLL that reads it and emits the object file. Porting “IL in, object out” is a much smaller problem than porting a C++ compiler. So I built a clean-room Rust reimplementation of just the backend, under a strict contract: for every input, the port’s output must equal the real DLL’s output byte-for-byte, and anything outside the proven function class returns NotImplemented rather than guessing. No decompiled source appears anywhere in the port — the original DLL is a black box whose observable output is the specification, its formats reverse-engineered by compiling controlled fixtures and diffing bytes.

The result, measured only after byte-exactness checks: 25/25 test fixtures byte-exact, at roughly 200–290× the throughput of the real DLL, in dependency-free Rust, built in 64 commits over ten days. The honest caveat: it covers only a minimal function class so far, most of the compiler’s optimization behavior is still NotImplemented, and the search loops have not yet started consuming it in production. It is a faster referee in training, not one deployed.

What is unfinished, and what generalizes

Plenty is open. The ML side — mining every search attempt as fine-tuning data on rented spot GPUs, with a small proposer model in training — is real, but its production deployment is honestly marked unresolved. The compiler port is an MVP. And a match percentage verifies functions one at a time, in isolation — which is why the strongest end-to-end check turned out to be the bluntest: rebuild the engine on WebGPU and run the decompiled code. A game booting in a browser tab exercises the reconstructed functions in combination, which no per-function score can.

But the shape of the result stands, and the methodology has since traveled to very different targets — pinball-machine firmware, network appliances under a responsible-disclosure posture. One person, driving agent fleets wired to purpose-built structured tooling and refereed by byte-exact oracles, can push clean-room software preservation roughly an order of magnitude faster than community pace. The recurring move is always the same: replace model trust with tool trust. Agents are cheap and getting cheaper. Judges are the hard part — and they are worth building.