How agent swarms decompile games byte-for-byte

For seven months I have been pointing swarms of Claude Code agents at a 16-year-old Xbox 360 binary and teaching them to decompile it byte-for-byte. It sounds like a game-preservation project, and it is. But the interesting part turned out not to be the games. It was the machinery: how you get dozens of agents to grind a brutally exacting task around the clock, what happens when they start gaming their own metrics, and what it takes to build oracles that let a machine tell itself the truth.

The strictest standard there is

Matching decompilation is the console-preservation community’s gold standard. You reconstruct C++ that, recompiled with the original compiler at the original flags, produces machine code byte-identical to the shipped retail game. Not “close enough.” Not “behaves the same.” Byte-identical — a provably faithful, clean-room reimplementation that preserves a piece of software history without distributing a single copyrighted byte.

Here is the whole discipline in one tiny function. The retail binary was built from something like this — which you cannot see, because that source is gone:

struct TrackScore {
    int   mBase;
    short mBonus;   // signed
};

int TrackScore::Total() const {
    return mBase * 2 + mBonus;
}

So you read the disassembly and write your best guess. It compiles. It returns the right number for every input you try. It is also wrong:

struct TrackScore {
    int            mBase;
    unsigned short mBonus;   // guessed the signedness
};

unsigned short TrackScore::Total() const {   // guessed the return type
    return mBase + mBase + mBonus;           // guessed the phrasing
}

Three small guesses, three diverging lines — a wrong line, a missing line, an extra line:

Every function in the binary is a fight like this, except the real ones are hundreds of instructions long and the compiler’s opinions are sixteen years dead.

My targets were three games on Harmonix’s in-house Milo engine: Rock Band 3 on Xbox 360, Rock Band 3 on Wii, and Dance Central 3 — all building on scaffolds from the MiloHax community, where rjkiv started the DC3 and RB3-Xenon decompilations and DarkRTA leads the Wii one. The question underneath was how far AI coding agents can push a clean-room decompilation before a human has to step back in.

Further than I expected. In one repo, agents pushed 44,226 of 69,301 functions to a perfect diff score across ten weeks and 2,802 commits, one day peaking over a thousand new matches. Meanwhile Dance Central 3 reached 92.44% of its 32,252 authorable functions — and grew a from-scratch WebGPU port of the engine that boots and plays. Then, at a certain point, the swarm handed me back a strategy decision instead of more matches. More on that at the end.

Why this is genuinely hard

The Xbox 360 build of Rock Band 3 shipped with zero debug symbols. No PDB, no map file, no names — the one scrap the strippers couldn’t take is .pdata, the exception-unwind metadata, which pins every function’s boundaries while naming none of them. Every one of tens of thousands of functions starts life as an anonymous address like fn_82682B60. It was built by Microsoft’s MSVC compiler for PowerPC, which folds identical functions together and lets the linker reorder code freely — so even “ground truth” is ambiguous.

I also measure against the whole binary, including the SDK, C runtime, and middleware nobody will ever match. That makes my honest whole-binary number (46.4%) deliberately less flattering than the 81.7% for the game code the effort actually targets — the shared engine sits at 77.4%. No denominator games.

You cannot brute-force names out of a symbol-less binary. What makes it tractable is what I came to call the Rosetta Stone strategy — two other decompilations of adjacent code, each naming a different stratum of the binary:

Neither oracle covers the other’s layer, and neither reaches the SDK floor. Triangulating between two parallel texts is the only reason the third is readable at all — a saga of its own.

The agent factory

A single agent can match a function. Getting dozens to grind concurrently without corrupting each other’s work is an infrastructure problem, and the infrastructure is as much the story as the matches.

At the core is an orchestrator MCP server over a SQLite match database. Agents don’t eyeball raw assembly and guess — they call typed tools: run a structured diff, analyze a function, look up a struct offset, record a patch result. Structured tools, not vibes. Isolation came from a copy-on-write worktree pool, where each agent reflinks the binary and a warm build cache into a private tree. Speed came from a content-addressed object cache keyed on compiler identity and dependency hashes, which cut a cold full rebuild from about five minutes to 3.5 seconds. When the feedback loop is “compile and compare,” that is the difference between an agent iterating and an agent stalling.

Underneath sits a forked toolchain: jeff — rjkiv’s Xbox 360 port of encounter’s decomp-toolkit — which splits a retail executable into linkable objects; a compatibility layer that runs the real 25-year-old MSVC linker on Linux by faking an undocumented Microsoft COM interface; and objdiff, encounter’s diffing arbiter, forked and extended to emit diagnoses agents can act on. The deepest cut — the Xbox 360’s VMX128 SIMD extension is an instruction set mainstream disassembly frameworks don’t decode, and the standard tooling silently mis-decoded it, so I wrote Ghidra semantics for all 77 opcodes by hand.

The compiler is the referee, so I cracked it open

Every candidate ends in a compile-and-compare against the original MSVC, which makes the compiler the bottleneck of every loop. And cl.exe is secretly two programs: a front end that writes an intermediate language to disk, and a back end that turns that IL into an object file. That IL is a drivable interface if you can capture it — and capturing it took an undocumented flag combination plus a syscall fault injector that turns the compiler’s temp-file deletion into a no-op.

Crude, effective. With the IL in hand I built a clean-room Rust port of the back end that never decompiles the original: it reverse-engineers the object format by byte-diffing alone and treats the real DLL as the sole judge, returning “not implemented” everywhere it can’t yet prove itself. On its MVP function class it is byte-exact and roughly 200–290× faster per object.

Your fitness function is an adversary

The search engine that automates the last brutal stretch of matching — where the code is right but the bytes differ — carries 140 behavior-preserving rewrites and tries them against the real compiler. Under a naive “accept if the score goes up” rule, it shipped two genuinely wrong rewrites its own metric rewarded: one turned a multiply-by-half into an add-half, equal only at a single input, for +3.25 points; another swapped the arguments of a non-commutative call for a fraction of a point of rounding noise.

The response — byte-exact recompilation as the only terminal judge, every other signal demoted to an advisory gradient — generalizes to any synthesis system driven by an imperfect reward. Reward hacking isn’t hypothetical in agent systems. I have the commits.

What surprised me

I was wrong about reasoning models. For a stretch my numbers said reasoning models were bad at this. They returned blank answers, over and over, while cheaper non-reasoning models just wrote the code. The tidy lesson practically wrote itself: when a hard external verifier exists, “think harder” is a liability.

It was a bug in my harness. Reasoning tokens bill as output tokens against one shared allowance, and my default ceiling was 8,000. The models were thinking right up to the limit and getting cut off before they could answer — and every one of those blanks came back flagged as truncated, in a field I wasn’t reading. Raising the ceiling to 32k, with a separate sub-cap for reasoning and a reserved floor for the answer, made most of them disappear.

What survived the fix is narrower and more interesting: bounded reasoning beats unbounded reasoning. On a seven-function slice graded by the compiler, capping thinking effort took one model from 4/7 to 6/7 byte-exact. In one of those flips it had spent 33,000 characters talking itself into rewriting an if/else if ladder as a switch — which compiles, and scores 56.7%, and is the wrong lowering. Bounded, it wrote the ladder, byte-exact. More thinking was actively making it worse. But the lever is bounding the request, not truncating a live trace: forcing a mid-thought wrap-up turned already-correct answers into zeros on every model I tried it on.

Model IQ was flat. Model diversity wasn’t. On a frozen 30-function benchmark, a cheap model cracked 13. Sonnet 5 cracked 13. Opus 4.8 cracked 15. I had pre-registered a gate — a stronger model has to double the cheap one to count as a real lever — and nothing came close.

But the cheap model and Sonnet only agreed on 8 of their 13. Together they cracked 18; adding Opus brought it to 19. The wall isn’t one hard set of functions that defeats everyone, it’s substantially model-specific, so the lever is running several models rather than buying a better one. It also saturates fast, and about a third of the benchmark none of the three ever touched. That residual isn’t a model problem at all — it’s a representation problem.

The best day came from diagnosis, not production. My single best day — over a thousand new matches — came not from more agents but from one insight: the retail linker scatters functions freely across the binary, so my splitter was attributing them to the wrong source files. A pool of functions that looked “unmatchable” wasn’t missing source. It was a bookkeeping illusion.

Agents grind. They don’t decide when to stop. A fleet will happily work a vein long after it stops paying out. What ended this campaign wasn’t a match count — it was the agents writing up each dead end with the evidence behind it and handing the call back to me: the cheap veins are exhausted, and the long tail needs either serious distributed compute or a human. Those write-ups, not the matches, were the most valuable thing they produced.

What generalized, and what’s unfinished

The pattern — agents plus structured tooling plus a ruthless external verifier — traveled well beyond one engine. It powered a responsible-disclosure firmware-triage methodology for network appliances, where the same rigor was used as often to disprove scary-looking findings as to confirm real ones. It also drove render-parity work on an obfuscated game client and a shipped multiplayer mod.

I want to be precise about what’s proven versus promising. The Rust compiler port is byte-exact only on its MVP class; most PowerPC code generation still returns “not implemented.” The ML side has produced honest negative results as often as wins — one capability sweep put a frontier model’s whole-function byte-exact rate at 12%, and a favored search strategy was retired for producing zero results. The performance numbers are from my hardware, not independently reproduced. And the headline decompilation is paused at a pivot: the cheap matching is done, and the deep grind is a resourcing question I haven’t answered.

That’s the honest shape of it. The agents did not replace the hard thinking; they made it cheap to act on at scale — which turned “one person can maybe decompile a game” into “one person can run a decompilation lab.” The frontier was never the byte count. It was building the oracles that let a machine tell itself the truth — and then, every so often, catching one of the oracles lying.