This is fun to watch in part because of the context – you see various strange states, people thrown around with GTA’s signature ragdoll physics, random explosions. But what I found satisfying was trying to figure out what might have gone wrong. Some of these are mistakes in the rendering engine, others in the scripting logic, and in many cases you can imagine a simple if statement that could have prevented the problem.
Mostly, it’s all a testament to the game’s complexity. Most of these are edge cases, corner cases, or whatever comes after corner cases (3D corner cases?).
Grand Theft Auto Online was to Grand Theft Auto V what World of WarCraft was to WarCraft 3: a 2013 release that allowed people to play online together, cooperatively or competitively.
But the game had a problem. The single-player mode booted up quickly, but the online mode took over 5 or 6 minutes to load.
It was a surprisingly lengthy load time, and it stayed this way for years. Eventually, in 2021, a player going by t0st got so bothered by it that they decided to try to fix it themselves.
This is a write-up of how they approached it. It’s pretty technical, but there is something really interesting and even inspiring seeing an amateur just go after code this way:
The culprit was one particularly inefficient, unoptimized loop. t0st not only managed to fix it for themselves, but their essay and fix did indeed reach Rockstar; the company patched the game for everyone within a few weeks, and then awarded t0st a $10,000 bounty.
As you already know from Unsung, I won’t let this remain just a cute warm-feeling story. When I think of this, I think of 5 or 6 minutes, times 8 years of this bug existing, times however many users started the game each year. This is awful. I wonder what happened in Rockstar’s culture that allowed this kind of a simple thing to remain unfixed, despite them likely knowing very well about people’s complaints (as is obvious from the beginning of the post above).
Here are two good comments from Hacker News showing it likely wasn’t even just bad user experience, but also a bad business decision:
But how could Rockstar not do this? GTAV is so full of great engineering. But if it was a CPU bottleneck then who works there that wouldn’t just be irked to try to nail it? I mean it seems like a natural thing to try to understand what’s going on inside when time is much higher than expected even in the case where performance is not crucial. It was crucial here. Almost directly translates to profits. Unbelievable.
I am absolutely shocked about this finding. The amount of money on microtransactions Rockstar lost because of this single issue must be gigantic. The amount of people that got turned off by the loading times over the years is massive. It’s mind boggling.
But I did also like this comment:
Honestly, while this horrible code is mildly offensive to me, I’m pretty impressed by this person’s persistence. It’s one thing to identify a bug in a compiled program, but it’s another to patch it without fully understanding what’s going on. Caching strlen was a particularly clever trick that sidestepped a bunch of more complicated solutions.
Grand Theft Auto: San Andreas followed in the footsteps of 2002’s GTA: Vice City, and was released in 2004. (Subsequent versions were IV from 2008, V from 2013, and VI, launching right now.)
It all started with a last-minute decision to remove some sex scenes that were originally meant to be published in the game:
The question now was how to extract the explicit material without breaking the wider game’s functionality so close to release. An emergency meeting was called so that the senior managers could discuss a solution. During the meeting Houser explained: “You can’t always take a thing out of a game.”
He had an alternative solution: rather than remove the content, the team could ‘wrap’ the sexual scenes so that while the relevant code would still be present on the game disc, it would be impossible for the player to access from within the game.
As he put it in an internal email sent at a later date: “We locked it away because there was no other way to get the game done on time – safely. The code is very interwoven in [GTA] and everything reacts to everything else. The impact of yanking something late is too scary.”
The decision was made. The sex would remain, but the sex would remain hidden.
The idea was perhaps to unlock the scenes later via an official patch, but we all know what happens to content hidden in game’s code – inevitably, a curious nerd will find it and then talk to other nerds about it:
The video showed that the content wasn’t a hidden cut-scene but a pair of playable mini-games. In one, dubbed the “spanking game” by Wildenborg, players were invited to smack CJ’s girlfriend on the backside, rhythmically tapping a button to raise an “excitement” bar. In the “shagging game”, players moved the analogue stick in time with the characters’ bodies to increase excitement. Both characters remained fully clothed and there were no genital shots. But this was still the most explicit sexual content Wildenborg had seen in a video game.
“It was at this point I decided to release the patch to the public,” he says. “I tossed the name «Hot Coffee» on the file, based on the fact that the girlfriends would ask CJ in for some «coffee» as a euphemism for sex. Hot Coffee was the first modification for San Andreas.”
Despite not owning a copy of the game himself, with two days still to go before the game’s European release, Wildenborg had spilled Hot Coffee.
It’s an interesting story that contains some good observations – “Houser understood the great contradiction at the heart of Western culture: tolerance towards violence versus intolerance towards sex” – although I am somewhat doubtful that the game creators were interested in pushing the public debate as much as they were in generating marketing-friendly controversy. If you agree with that correction, it becomes an even more interesting story in that it doesn’t portray anyone involved in a particularly positive light.
The original Grand Theft Auto – known in early production as Race ’n’ Chase – had a problem: it just wasn’t too fun to play. The driving was there, the buildings were there, the missions were there, but the joy wasn’t.
What injected the right amount of joy was… an unexpected bug.
When the player earned a “wanted” status, the game dispatched some cop cars to chase them. As they do, the cop cars had a particular pathfinding algorithm where they meant to converge on a player. But within that algorithm, there was a mistake, as described by one of the game’s creators:
[…] the other thing that was a problem was the handling – the car handling was appalling. There was a point in it where you used to have a button for opening the doors and it was just rubbish. I can’t remember if this is true because we used to joke about it that you even had to start the engine. It was awful, it was too sim-y.
And there was a whole living city thing that crept into it. That kind of inspires people to be more realistic and simulate more often. That steers you in a certain direction.
But there were people on the team constantly trying to get that fixed. So the core of playing was fundamentally broke. There were steps being taken to fix that, but it wasn’t really gelling together. The police behaved really badly, the way they originally worked was just rubbish.
Then one day, I think it was a bug, the police suddenly became mental and aggressive. It was because they were trying to drive through you. Their route finding was screwed I think and that was an awesome moment because suddenly the real drama where, “Oh my God, the police are psycho – they’re trying to ram me off the road.”
That was awesome, so that stayed in. It was tweaked a little bit, but that stayed in because that was great fun. Suddenly the game got more dramatic and it’s no longer boring – the police trying to pull you over. They’re after you, they’re trying to ram you off the fucking road. Everybody suddenly went, “Hey, this is actually pretty cool. There’s something in this, this is working.”
It was less about the mission stuff, which we always thought was another mess, and more about just general play – just being able to piss around.
I don’t think we’ve ever seen the code or the bug in question. I found the story interesting, not only because it teaches us that you can’t really work your way around bad mechanics (this can be as true for serious apps as it is for videogames), but also because it highlights the interesting push and pull between realism and fun.
How do you squeeze a city that occupies over 50 megabytes into the 32MB memory of the console? You simply do what The Truman Show did, and construct the city around the player as they’re moving around:
This has, as you can expect, a lot of technical and even game-design consequences, and the video goes into a lot of detail on these – including Brown rebuilding the Grand Theft Auto 3 source to visualize things better.
This technique is also used in interface design, for example if you have a really long list of things that would take too much memory or GPU power to render. What the video calls “streaming” is, in the context of UI, often called “virtualization”: instead of having a full long list (or an entire world), you abstract it away – or, virtualize – into something nimbler.
Some of the challenges and techniques used by Grand Theft Auto 3 apply pretty directly here, as well:
you can use UI skeletons as “low poly” models,
in some contexts, you can guess the user is more likely to move in one direction (for example, going through fonts in a font picker), and more eagerly preload where they’re going to look next, rather than symmetrically in both directions.
On the other hand, “speedy players” and “pop in” can’t ever be solved because any UI list is random access, and slowing users down is not typically appropriate; better to make loading as pleasant as possible than introduce any roadblocks, even if figurative ones.
San Andreas was released in 2004, but the game started breaking only after Windows got updated… in 2024. Turns out the bug was sort of a ticking time bomb just waiting for the right set of conditions. We covered one similar bug before, in Half-Life 2 – but this investigation goes deeper, and shines a light on the difficulty of making Windows, whose backwards compatibility comes at a price.