Forking the decompilation toolchain
What it took to make nine open-source reverse-engineering tools — from Ghidra on down — fluent in a console ISA none of them were built to read.
Over about six months I forked nine open-source reverse-engineering tools — Ghidra, objdiff, decomp-toolkit, m2c, wibo, and several others — and extended each one until a fleet of AI coding agents could use them to reconstruct the source code of two Xbox 360-era games, byte for byte. None of these tools was built for that job. Most of them had never seen the Xbox 360’s CPU at all.
This is a story about what happens when your entire toolchain is someone else’s project, and the target you care about sits just outside what any of it supports.
The problem: a console nobody’s tools can read
Matching decompilation is the practice of writing new source code that compiles to the exact bytes of an original binary. It is how communities preserve games whose source is long gone: the recovered code is original, written from the compiled output, and verified instruction by instruction against the retail build. The tooling for this is mature — but it grew up around the GameCube and Wii, targeting GCC-family compilers, and it assumes a human sitting at the controls.
My targets were Harmonix’s Dance Central 3 and Rock Band 3, built with Microsoft’s MSVC compiler for the Xbox 360, alongside their Wii counterparts. That combination breaks the existing tools in three compounding ways. The Xbox 360’s CPU carries a proprietary 128-register SIMD extension called VMX128 that no mainstream disassembler decodes — not Ghidra out of the box, not even Capstone. MSVC’s PowerPC code generation looks nothing like the GCC idioms the community’s analyzers were tuned against. And the executables ship in Microsoft’s XEX container format, which most tools cannot open.
On top of that, I added a constraint of my own: the consumers of these tools would not be humans at GUIs but swarms of Claude Code agents running around the clock. A tool that renders a pretty diff for a person is useless to an agent that needs a structured verdict it can act on.
So every fork had to answer two questions. Can you teach this tool the target? And can you teach it to talk to machines?
Teaching Ghidra a new dialect
Ghidra — the NSA’s open-source decompiler — became the backbone. A community contributor had already produced disassembly-only VMX128 support, enough to name the instructions but not to reason about them. My first substantial commit added full decompiler semantics for all 77 VMX128 opcodes: 86 p-code operation definitions covering the load/store, logical, and floating-point families, plus a correctness fix declaring the vector registers at their true 128-bit width. I validated the result against 13,836 VMX128 instructions in a real binary. The payoff is a new processor language for the Xbox 360’s Xenon CPU, so vector-heavy code decompiles instead of vanishing into undefined-instruction noise. A second language definition covered the GameCube and Wii CPUs.
The most satisfying fix was smaller and went back upstream. Ghidra’s PowerPC jump-table analyzer silently failed on MSVC-generated switch statements, leaving large regions of control flow unrecovered. Digging in revealed three independent defects: an access check that unconditionally rejected table reads beyond 4096 bytes, a target list that leaked entries from one switch into the next, and a predecessor walk too shallow to span MSVC’s multi-block pattern. I filed the diagnosis as an issue and the fix as a pull request to the NSA’s repository, with a synthetic-pattern regression test. A maintainer triaged and assigned it; as of this writing it remains open and unreviewed. That is simply how upstreaming to a large project goes, and I would rather say so than pretend otherwise.
Scale was the third front. Running Ghidra’s function-matching and similarity-search engines on a game with roughly 65,000 functions surfaced performance cliffs everywhere — one degenerate case stalled the CPU for over 70 minutes. Fixing them meant parallelizing carefully inside a codebase I did not own, so every parallelization commit carries a written argument for why the output stays deterministic. The honest caveat: I quantified the before state in each commit but did not record measured after numbers, which I regret.
Rebuilding tools for machine consumers
The more interesting transformations were about who the tool serves.
objdiff is the community’s standard object-file diffing tool, built to show a human where their reattempted code diverges from the original. I rebuilt it to diagnose instead of display: 21 pattern detectors that recognize common mismatch causes, six fixability verdict classes, and structured JSON output an agent can consume directly. The fork grew by about 19,000 lines across 76 files. Its nastiest bug was uniquely an AI-pipeline bug — a nondeterminism in how MSVC exception-handling funclets were paired meant two byte-identical builds could report different match percentages, quietly corrupting the exact metric the agents optimize against. Deterministic pairing over a masked byte signature fixed it, across three days and four patch releases.
pyghidra-mcp, which exposes headless Ghidra to agents over the Model Context Protocol, needed a different kind of hardening. Its worst failure only exists under swarm load: 125 half-closed connections deadlocking a single-threaded JVM as sync and async handlers raced it. The fix was one process-wide reentrant lock serializing every public method — unglamorous, and exactly right.
Meanwhile decomp-toolkit, the Rust CLI that splits console binaries into linkable objects, forked into two deliberate lineages. One stays within eight lines of upstream, PR-ready. The other carries a full platform rewrite teaching it to split retail Xbox 360 executables — cut over like a zero-downtime database migration, with staged phases, shadow execution, parity digests, and a rollback flag, promoted to default only after it processed 69,336 functions with byte-identical output and zero crashes. Keeping a clean fork and a heavy fork of the same tool sounds like overhead; in practice it is what let me hack aggressively without burning the bridge back to upstream.
Throwing away the clever version
The fork that taught me the most was wibo, a minimal Win32 compatibility layer. To rebuild these games verifiably, I needed the 2010 Xbox 360 linker running on Linux — which meant faking Microsoft’s private, undocumented PDB COM interface. My hand-written stand-in for mspdb80.dll runs about 810 lines and implements eight COM vtable object types; the linker’s output is byte-identical to a run under Wine, apart from timestamps and an embedded GUID.
My first working version generated x86 calling-convention thunks at runtime. It worked. When I opened a pull request, the upstream maintainer called it “technically impressive but incredibly scary” — and he was right. I threw it out the same day and rebuilt it as a boring, reviewable cross-compiled DLL. That PR is also still open, but the lesson shipped immediately: in someone else’s codebase, reviewability beats cleverness every time.
What’s finished, and what isn’t
The status board is honestly mixed. The Ghidra fork, objdiff, the heavy decomp-toolkit lineage, and the MCP server are actively maintained; they carry the pipeline daily. The m2c decompiler fork — which grew a from-scratch VMX128 disassembler for an ISA extension Capstone still cannot decode — has been dormant since March. Two upstream pull requests remain open. The heavy toolkit fork is mid-triage on a third round of binary-splitting edge cases.
Much of this work was itself AI-assisted, and the commit history says so: agent co-authorship is disclosed in trailers and in the upstream PRs, with human direction and review throughout. If there is one takeaway, it is that “fork and extend” is not a lesser mode of open-source work. Reading nine unfamiliar codebases deeply enough to change what they fundamentally do — while keeping each fork honest, minimal where it can be, and pointed back at upstream — turned out to be the hardest and most transferable engineering in the whole project.