From emulator to real console

Every emulator is a claim about what the hardware does. Most of the time the claim holds, and nobody checks it. This year I spent five months checking it — taking Rock Band 3 from a debuggable boot inside a modified emulator, through a from-scratch native container writer, to a shipped mod running on a physical Xbox 360.

xenia fork         xex-patcher        RB3 Overdrive
headless 360       XEX2 containers    same-instrument
emulation oracle → a loader accepts → multiplayer, shipped

Nearly every hard bug across those three repositories lived in the same gap — between what an emulator does and what the console actually does. Here is the arc, through its three hardest problems.

An emulator that can’t play games, on purpose

The work started with a fork of Xenia, the open-source Xbox 360 emulator, for an unusual reason: I didn’t want to play anything. I’ve been decompiling the Milo engine — the shared engine behind Rock Band 3 and Dance Central 3 — and a decompilation is only provably correct if you can run the rebuilt binary and show it behaves identically to the original. That demands an emulator that is headless, scriptable, and deeply observable. Xenia ships as a windowed, gamepad-driven application — the opposite of a test harness.

So the first commit was a big one: a headless mode for Linux — 3,328 lines across 61 files, adding a windowless Vulkan command processor, a no-op audio backend, scripted controller input, and frame-dump output. Landing that in one shot into an unfamiliar codebase with seven thousand commits of history — then keeping it alive through five months of churn — became the foundation everything else sat on.

The months after were a slow detective story. Getting Dance Central 3 to boot meant root-causing heap exhaustion in the guest, then discovering that config parsing was pathologically slow under emulation — a bump allocator cut one parsing pass from 15,273 ms to 236 ms — and ended with the main loop stable at 175 fps. Along the way the fork grew the instruments the later work would need: an interactive guest-PowerPC debugger, and a function entry/exit tracer woven directly into the x64 JIT.

The segfault at exactly four gigabytes

In July the target shifted to Rock Band 3 and the work got genuinely strange. A host-side segfault kept landing at exactly the guest memory base plus 4 GiB — the telltale signature of a guest NULL-pointer dereference mapped into the emulator’s own address space. It hit both games. The obvious conclusion was a game bug, or a bug in my decompilation-derived understanding of the engine.

It was neither. Tracing the fault against guest disassembly and the retail kernel’s behavior revealed a hardware-fidelity gap in the emulator itself: real Xbox 360 hardware backs guest page zero with actual memory, and certain architecturally special guest code legitimately touches it. Xenia’s policy treated any access there as fatal. The game was fine; the claim was wrong. The fix was a hardware-faithful zero-page backing behind a default-off flag, so the working configuration was never at risk — and the investigation fills ten cross-linked wiki pages that read like a lab notebook, wrong hypotheses included.

That investigation set the pattern for everything after it: root-causing faults that span four layers at once — host x64 JIT, PowerPC guest, kernel emulation, and the game engine — from nothing but fault addresses and logs. It also surfaced a bug I have to own. A later crash in the community-modified Rock Band 3 Deluxe traced back to a regression in my own earlier kernel-emulation change, which had been silently wedging its splash screen. Owning your regressions is part of the discipline; so is A/B-ing draw counts, frame rates, and crash counts on nearly every change, so a fix for one game never quietly breaks the other.

Writing containers a real console will accept

Emulation proves correctness. It does not get code onto a real console. The mod at the end of this chain ships as a DLL that a development Xbox 360 must accept through Microsoft’s proprietary XEX2 container format — undocumented, and enforced by a loader whose only diagnostic is binary: the module loads, or it doesn’t.

the XEX2 container, as the loader verifies it
┌────────────────────────────────┐
│ headers + import table ←── bug │
│ RSA-2048 signature             │
│ per-page SHA-1 chain, ×134     │
│ LZX-compressed image, 8.38 MB  │
└────────────────────────────────┘

For months the working hypothesis was that the container itself was broken — the compression, the integrity hashing, the signature envelope. One control experiment collapsed all of it: recompressing and re-signing a known-good stock DLL, with no content change, still loaded fine. The container pipeline was never the problem. The real defect was in the packer-synthesized import table — one kind of import thunk demands an exact byte encoding, and the packer was emitting one of two wrong forms. That narrowing, verified as a five-module load matrix on live hardware, became both the proof and the regression suite.

From there I reproduced the console’s signing and integrity pipeline from scratch as a native Linux tool: a reverse-chained per-page SHA-1 hash chain and a 2048-bit development-console RSA signature, matching the reference output bit for bit. The hardest piece was the compressor. The format uses LZX, and the open encoder I vendored had a single-buffer limit of roughly 2 MB against the mod’s 8.38 MB image. Generalizing its sliding window to stream past that limit produced output whose decompressed image and all 134 per-page digests are byte-identical to the closed-source reference tool. The compressed bytes differ — but every value the loader actually verifies matches, which is the only claim that matters, and the honest way to state it.

Shipping a feature the game never had

The end of the chain is RB3 Overdrive, a mod players download and run on their own console. Its headline feature — multiple local players on the same instrument, which retail Rock Band 3 never supported — required coordinated hooks across part selection, track binding, and player-slot assignment, all inert behind a default-off flag so stock behavior is untouched unless you opt in.

Getting it working meant debugging closed-source retail game code live on real hardware: two days of numbered diagnosis cycles, converting each crash into a named root cause — a slot index of exactly negative one, a null dispatch, a missing vtable pointer — rather than a surface patch. The subtlest bug wasn’t input handling at all but a shared display asset, and it led to the part of the project I’m proudest of as engineering ethics: rather than ship a modified copy of a copyrighted game asset, the mod reconstructs it from the player’s own game files at first boot, using a tiny embedded splice plan, with SHA-256 gates and automated tripwires that make shipping a stock byte impossible. It went out as version 1.0.0 in a twelve-day sprint, full source included, verified on the same console the container work targets.

What’s done and what isn’t

Honesty about status matters more in a writeup than in a changelog, so: Dance Central 3’s boot-to-gameplay path is stable in the emulator fork; Rock Band 3’s emulated bring-up is still the live frontier. The container writer’s offline validation is complete and hardware-confirmed, but a live load test of its natively compressed output is still pending. The mod’s 1.0.0 release is shipped and hardware-verified. And none of the three forks has an upstream pull request open yet — several of the emulator fixes are written to upstream quality and gated behind default-off flags precisely so they could go back, but that work remains ahead of me.

The through-line, if there is one, is that an emulator and its hardware disagree in small, unforgiving ways — a page of memory one backs and the other doesn’t, a byte encoding one loader checks and the other ignores. Each of the hardest bugs here was one of those claims quietly failing. Finding the gaps from both sides, and closing them with control experiments instead of guesses, turned out to be the whole job.