Chasing a Phantom Compiler

I set out to rebuild a 1990s DOS game’s executable byte for byte. Not a working clone. Not “close enough.” The same bytes. Feed my reconstructed source through a compiler and get back a file whose SHA-256 hash matches the original, all thirty-three thousand-odd bytes of it, exactly.

That goal is how I ended up spending weeks chasing a compiler that, it turned out, never existed.

Why Byte-for-Byte

There’s a reason to set the bar that high. A clone that behaves the same only proves you understood the behavior. A byte-for-byte match proves you understood the toolchain: the exact compiler, the exact flags, the order it laid things out in memory, every quirk of how it turned Pascal into machine code. You can’t hand-wave it. Either the hash matches or it doesn’t. It’s the most honest test of understanding I know, because the binary keeps no secrets and grants no partial credit.

The game was written in Turbo Pascal, the compiler every DOS hobbyist had a pirated copy of. The runtime library stamped into the executable read 1990, which pins it to Turbo Pascal 6.0. So I dug up a copy of TP 6.0, reconstructed the source one function at a time, and started compiling. Function by function, the bytes lined up. It was slow and tedious and occasionally thrilling, in the way that watching a diff shrink to nothing is thrilling if you’re the kind of person this post is for.

And then the bytes stopped lining up. In exactly two places.

The Copy With the Seatbelt Cut

The two holdouts were both string copies.

In Turbo Pascal, a string is a length byte followed by its characters. When you assign one string to another, the compiler emits a little copy routine. Stock TP 6.0 always emits the careful version: it pushes the destination’s maximum length onto the stack and calls a bounds-checked copy that refuses to overflow the buffer.

; the careful copy stock Turbo Pascal emits
mov  ax, 4          ; the destination can hold at most 4 bytes
push ax
call StringCopyChecked

; the copy in the original binary
push dest           ; no maximum length pushed at all
push src
call StringCopyBlind

My target called a different copy routine. One that took no maximum length and just blindly moved the bytes. A string copy with the seatbelt cut.

I could not get the stock compiler to emit it. Not with any combination of optimization switches. Not with a tiny string[3] or a full string[255]. Not with typed constants, not with function results, not with concatenation. I tried every shape of source I could invent, and TP 6.0 produced the careful, checked copy every single time. The original used the blind one. Two places, both wrong, and no source I could write would close the gap.

The Wrong Compiler

So I assumed I had the wrong compiler.

This is a perfectly reasonable assumption, and I chased it hard. I tracked down the 6.01 beta. Turbo Pascal 7, and Borland Pascal 7 after it. TPCX, the protected-mode “Professional” compiler that’s a pain to even run today. I compiled the same probe with each one and classified the output. Every last one emitted the careful copy. None of them did the blind one.

Then I found the detail that should have been a much bigger clue than I let it be. There was an older build of the game, from 1990, compiled with Turbo Pascal 5, a whole major version earlier. It had the blind copy too.

So whatever produced this wasn’t a feature of one compiler release. It spanned at least two major versions, across a span of years. That killed the entire “I just need the right Turbo Pascal” theory in one stroke. The thing I was looking for was older and more stubborn than any single compiler.

The Tool That Wasn’t There

Which left a more romantic explanation, and I’ll admit I liked it: the author had a secret tool.

Picture it. Some sharp assembly programmer in 1990 writes a little post-processor, a program that walks over the finished executable and rewrites those safe string copies into fast unsafe ones, shaving off the bounds check because he knows it’ll never overflow. People absolutely did things like that back then. Cycles were scarce and bragging rights were real. It fit.

So I went looking for the tool. Archive.org. Old shareware CD dumps. The Garbo and SimTel mirrors. Usenet threads from the early nineties. I actually found things, which was encouraging and then deflating: a couple of genuine period Pascal optimizers, including a string-aware peephole optimizer named Sally that did almost exactly the right kind of thing. I got it running and fed it the case I cared about. It optimized the copy, all right, by inlining it directly instead of calling the blind routine, and it ignored the one situation that mattered to me entirely. Wrong mechanism. Every tool I found missed.

Meanwhile the binary kept quietly insisting it was legitimate. DOS executables carry a relocation table, a list of addresses the loader has to patch when it places the program in memory. I checked where the blind-copy calls sat in that table. They were threaded in, in load order, perfectly interleaved with everything else, exactly the way a linker writes them. Not bolted onto the end the way a patch tool leaves its fingerprints. Whatever did this did it before the linker ran, not after.

And the rebuild itself argued against a general-purpose optimizer. Every other byte in the file was plain stock TP 6.0 output. Only this one transform was alien. If the author had run some broad optimizer over the whole program, it would have rewritten the boring code too, and nothing would have matched. The weirdness was surgical. It knew about Turbo Pascal’s string routines specifically and touched nothing else.

I’d been at this for weeks. I had a wall of evidence and a culprit I couldn’t name. To keep moving, I wrote my own little post-processor that replayed exactly that one transform, just so my rebuild would match, and told myself I’d identify the real thing eventually.

The real thing was a compiler flag.

It Was a Flag

Here’s what I’d missed, and it’s embarrassing the way the best bugs are. Every test I’d run, every probe, every “does the stock compiler emit this” experiment, was a plain standalone program. The actual game wasn’t. The game used overlays.

Overlays are a memory trick from when memory was the constraint that governed everything. A DOS program had roughly 640 KB to live in, and a big program didn’t fit. So you split the code into chunks that took turns. Only the chunk you needed right now sat in RAM; the rest waited on disk, and an “overlay manager” swapped them in and out as you called between them. It’s virtual memory, hand-rolled, years before the hardware would do it for you.

Now think about a string constant living inside overlaid code. That constant sits in a code chunk the overlay manager is free to evict at any moment. If you pass a pointer to it across an overlay boundary, you’re holding a live grenade: the manager might swap the chunk out from under you mid-call, and your pointer now aims at whatever got loaded in its place.

Borland knew this. So when Turbo Pascal compiles overlay-aware code and you pass a string constant to another routine, it doesn’t pass the pointer. It first copies the constant into a temporary on the stack, which is always resident and never swapped, and passes that. And because the temporary is freshly made and exactly the right size, there is nothing to bounds-check. So the compiler uses the blind copy.

That was it. That was the whole mystery. The “impossible” instruction was stock Turbo Pascal doing something completely ordinary and correct, in a context I had never once tested.

It got sharper still. You don’t even have to actually overlay the program. The trigger is a single directive at the top of the source:

{$O+}   { overlays allowed }

Flip that on and the compiler, not knowing whether a given unit will end up overlaid, plays it safe and emits the resident-copy-then-blind-copy everywhere. The author had built the game’s shared library units with overlays allowed. That one directive, inherited by everything that linked against those units, was the entire fingerprint I’d spent weeks chasing.

I added the flag, rebuilt, and the alien instructions appeared on their own. Stock compiler, no tricks. My custom post-processor, the one I’d written to forge the mysterious transform, ran and found nothing left to do. It had become a no-op. The phantom tool I’d hunted across twenty years of shareware archives was a checkbox in the compiler I’d had open the entire time.

The part that stings a little: it wasn’t even rare. Once I knew the fingerprint, I could see it in other DOS games of the same vintage built with the same compiler. They all carried the blind copy, for the exact same reason. It was never a secret. It was a documented feature behaving precisely as designed.

The Anticlimax

There’s a particular flavor of anticlimax in reverse engineering where the answer, once you finally have it, makes the entire preceding hunt look a little foolish, and is somehow more satisfying for it. I’d built up this whole story about a clever author and a lost tool. The truth was that the compiler was being careful and I was being ignorant. The mystery was never in the binary. It was in my assumptions about how I’d tested it.

The byte-for-byte match works now, which is its own quiet reward: a hash that comes out identical, proof that there’s nothing left in those thirty-three thousand bytes I don’t understand. But the thing I’ll actually keep is the shape of the mistake. Every experiment I ran was clean, rigorous, reproducible. Every one of them was also quietly testing the wrong thing in the same way, for weeks, without my noticing. The hardest bugs aren’t in the code. They’re in the variable you didn’t know you were holding fixed.

Some flags cast very long shadows.