Total War: WARHAMMER II – Development Update

It’s been a very busy year at CA, what with WARHAMMER II’s launch, a string of announces for new historical games, and not forgetting Halo Wars 2 in February as well. In line with our long-term plans to make more games in parallel, we’ve also welcomed many new members to the CA family. It’s going to be another interesting year ahead.

It’s quite exciting to go back to ROME II with Empire Divided. We’ve seen pleasantly surprised reactions from Total War players across the board, especially from historical fans who may have been feeling a little left out over the course of WARHAMMER I and II. This is precisely why Empire Divided exists; to provide some hearty historical snacking for players to enjoy between main meals.

Our first Saga title, Thrones of Britannia, will be more significant still. It’s incubating in the care of Jack Lusted and his Historical New Content team who made the Age of Charlemagne expansion for Attila. They’re working on some very nice designs which we’ll start sharing in more detail shortly.

Plus, we also have the next full-blown historical era title in the works too. More on this next year.

Work on Warhammer II continues of course. Mortal Empires has not been as smooth for all players as it should have been, and we feel some explanation is in order. I’ll hand over to Brand Director Rob Bartholomew, and Creative Director Mike Simpson, to tell you more –

Rob:

There are rather more moving parts to the Mortal Empires campaign than we’d first anticipated, that’s for sure. At present, we do feel it’s playable and enjoyable, though not under all circumstances. It’s taking time for us to stage implementation of the Norsca content, then tweak that content so it’s relevant to Mortal Empires. There are also some genuine bugs, and rest assured we’re in the process of addressing these. It feels like these two things get conflated a lot, and as a result it’s perhaps getting harsher treatment than it deserves. But for sure, our update schedule for WARHAMMER II is behind where we (and likely you) would like it to be.

Chiefly, integrating Norsca into Mortal Empires has given us no end of trouble. If you’re interested in the detail, Mike has some more in-depth commentary on the development challenges behind this.

Mike:

Things have gotten complicated since I started making games. About a hundred million times more complicated. The first game I wrote for the Sinclair Spectrum back in the 80’s fitted into 41 kilobytes. An uncompressed WARHAMMER II build is about 112GB of data, spread across about 800,000 files. And at any point in time there are more than 30 different branches of Total War in development, each of which needs its own copy of those 112GB, so work on one aspect of the game in one branch doesn’t cause bugs in another.

There are branches for different games and DLCs, different builds and releases for each, small progressive changes and long-term rewrites. They constantly branch, progress, complete and merge back together again. ARENA uses streams – they’re the speed-dating version of branches. Shorter lived, more casually started and dropped, and very numerous – there are about a hundred of them.

So TW as a whole is about a hundred million times bigger than that first game I wrote in 1984.

“So what?” you may say. “Sure, it’s complicated, but there are hundreds of you now, trained to the teeth, with cutting edge tools and a thousand machines the equivalent of 80’s supercomputers. It’s like the Matrix down there in Horsham now.

It’s a fair point.

The truth is we screwed up. We got so good at branching and merging builds, we thought we had it mastered. Merging is easy when the things that have changed in one branch don’t coincide with the things that have changed in the other. It gets more difficult when the changes overlap – you most often have to manually intervene to pick which branch is “right”. Typically, merges get exponentially more difficult the longer branches stay apart, as the chance of changes clashing increases.

We planned to take the Norsca branch from WARHAMMER I, along with a variety of other WARHAMMER I improvements, and merge it in to WARHAMMER II. Because the content in Norsca didn’t exist in WARHAMMER II, even though a long time had passed, the merge should have been quick and easy.

There was one tiny problem with this plan, which a bit of deeper might have discovered. It was bollocks.

Although the data itself didn’t clash, improvements to the infrastructure of WH2 meant the containers the data was going into had changed significantly. We also wanted to make Norsca really interesting and unique to play as, so there was also more in the way of bespoke code for Norsca than we originally intended. The first attempt to integrate Norsca in the routine way caused an immediate tsunami of bugs. We smacked our heads against it, realised it was unsolvable in a short span, and had to back out. We started on a second, more careful approach, which has been glacially slow, with deeper and deeper issues with the integration tools and process becoming apparent at each stage.

Modding Norsca into the final game, just as a modder might, would be much quicker and easier. But adding content to the final game rather than the source, which takes a great deal more careful implementation, would produce the same problems as modding; every new game update may break it. It would be a bodge.

The changes to the database structure between WARHAMMER I and WARHAMMER II and the ongoing code and database changes for Mortal Empires made it very difficult to un-tangle the Norsca content, once it had been copied across to WARHAMMER II. As the weeks passed, we realised that it would probably be easier to re-implement Norsca in WARHAMMER II, rather than straighten out the Norsca content copied over from WARHAMMER I. And so, having lost months, we’ve now decided to pick the next point where all the branches are back together (after the next major content update), and then re-implement Norsca from scratch in WARHAMMER II. This won’t be quick, but it is guaranteed to work.

And so we add one more thing to the long list of dumb mistakes we’re never going to make again…

Rob:

As you might expect, these difficulties have had a knock-on effect on all our upcoming WARHAMMER II DLC, and the re-implementation of WARHAMMER I content into WARHAMMER II. Due to the divergent codebases between WARHAMMER I and WARHAMMER II, these are not simple copy-paste problems, as the phrase goes. However, we’re now in a stronger position to share our plans for that campaign’s content changes and updates, and they’re going to look something like this:

December: Reprisal Update rollout for Mortal Empires. This contains equivalent Foundation Update skill/campaign improvements to Old World Legendary Lords and Chaos, and will also include some new changes and fixes for Mortal Empires. We’re also bundling in a free, new (and highly experimental!) game mode which we’ve had in our back pocket for a while now. More news on this later in the week.

January: Our first Campaign Pack was initially planned for this year, but will slip to January. Not to seem too… cryptic… but It’s a hugely characterful and significant addition to both Eye of the Vortex and Mortal Empires, and we’re pretty sure people are going to enjoy it. We’ll also be rolling out another Mortal Empires update alongside this release to address any further issues post-Reprisal Update.

May: Yes, this is a bit of a gap, but this is how long our painstaking re-implementation of Norsca will take to get added to Mortal Empires. This update will include Norsca and its full roster, updates to various aspects of the race (eg techs, New World race interactions) to make them appropriate and relevant as well as the 30th birthday Regiments of Renown. There will also be further updates to other aspects of Mortal Empires.

You’re likely going to have some questions about all this, so I’ll try and anticipate them, if there’s further questions we can answer, we will follow up with a quick-fire Q&A.

Why are you launching a DLC Campaign Pack before Norsca makes it into Mortal Empires?

Because Warhammer’s New Content Team began work on it before WARHAMMER II’s launch, and we’ve a great deal less to do on it than we have to get Norsca in the game. Plus, those who already own Norsca have likely played/completed campaigns already. We also know that both the Vortex Campaign and Mortal Empires will benefit from an injection of fresh talent, just as the Old World did over time.

Why don’t you just pull people in from other Total War projects to help out on Norsca?

There’s an element of Brooks’ Law at play here, especially when it comes to code-level integration issues. Project-switching like this also requires time for people to get up to speed with its very specific challenges, which steals time from both the project they were working on and the one they’re switching to.

Why can’t you do more frequent patches?

Our update methodology is to roll as many fixes into one patch as we can, and time them to coincide with new content launches, as it’s the most efficient and effective way of scheduling test-cycles, and results in higher-quality updates overall. Even then, bugs can sometimes get through.

We take this approach because Total War games are uniquely complex and incredibly time-consuming to test. More frequent, incremental patches may seem sensible to a player, but we’d ask you to look at it from a tester’s perspective. He could spend weeks testing a couple of fixes, when he could be doing the exact same playthroughs and test twenty in parallel. In extreme cases we’ll work on a hotfix – but only in extreme cases, as we did with the Chaos doom-bug.

I hope this explains where we are, and gives you some reasonable expectation for the coming months. It comes with the very firmest assurance that, just as we did with WARHAMMER I, WARHAMMER II and Mortal Empires will continue to grow, evolve, and improve over time.