Jafar Soltani - Adopting Continuous Delivery in AAA Video Games - PIPELINE conference 2018 - friv2018games.net

Jafar Soltani – Adopting Continuous Delivery in AAA Video Games – PIPELINE conference 2018

Pipeline Conference
Views: 9506
Like: 463
“My talk is about how Rare changed its culture and approach for developing ‘A Sea Of Thieves’ by adopting Continuous Delivery.

As the result of this change, we’ve minimized crunch and have been sustainably delivering new features to players for the past 2 years. We’ve had over 120 releases so far, which means we can get feedback from players, evolve and improve the game. Our bug count has been very low (less than 50 high priority bugs) throughout the project. As we have invested heavily in automated tests, we’ve had a very small manual test team which has reduced cost significantly.

Adopting Continuous Delivery practices isn’t trivial and whilst we are several years into the journey, we are continuously working to improve our capabilities. I’ll discuss the challenges we’ve faced and the principles behind the techniques and practices we’ve used, such as:

– Changing culture so that everyone including Developers, Artists, Production and Test teams are responsible for the quality of the game.
– Changing culture to adopt extensive automated testing.
– Working in small batches.
– Adopting Continuous Integration practices such as Trunk based development, Feature Toggles and locking the depot when the Commit stage fails.
– Continuously improving the Commit stage to ensure we get fast and reliable feedback. I will cover the challenges in building AAA games in C++, transferring large game packages between agents, running 40,000 automated tests, identifying the most relevant and valuable tests to run as part of the Commit stage and dealing with flaky tests.
– Ensuring the game is always shippable. I’ll discuss prioritising fixing bugs and broken tests over developing new features. I will describe how we release regularly and how we’ve been releasing at least once a week over the last two years.
– Applying a continuous improvement culture, where we’ve been collecting telemetry from players and how we’ve used the data to help us better design the game. ”

Jafar joined Rare in January 2008. During his time at Rare, Jafar has worked on Banjo-Kazooie: Nuts & Bolts, Viva Piñata: Trouble in Paradise, Kinect Sports Series, Rare Replay. He’s most recently been leading the deployment pipeline team working on Sea of Thieves helping Rare adopting Continuous Delivery.


  1. 14:53 Could you elaborate on this last one? Do you mean you had them make a feature, and when done, had forced them to basically start over in the real code base?more clarification wanted: was this so that they could try out ideas without cluttering anything? I don't understand though. if the goals were force them to re-think about the quality as they put it in the real deal, why not just create a branch and never let them merge it? was it that you wanted to be sure they didn't just copy paste?

  2. Incredibly valuable information! Thank you for sharing

  3. Did the technical practice also have an effect on the business side of things?

  4. So this is why games as a service is so popular now? I'm not sure it works.

  5. Games as a service is a marketing and financial decision rather than an engineering one. Persuading customers to become subscribers, microtransactions, online delivery, and licensing games to customers rather than selling them was a decision Microsoft implemented with MS Office. Office 365 promised for an annual fee access to a constantly optimised version of Office in exchange for which the consumer lost any property rights over the software. MS wanted that because it guaranteed a regular cashflow, plain and simple. (Much like it, Samsung and Apple are adopting this model with the next generation of hardware too.) Yet, MS have reversed their decision this year because not enough people wanted it . Perhaps there are limits to the Rentier Economy that is damaging us.

  6. So many idiots linking CD with 'live service'. It's a gaming terminology relating to monetization, leave the engineering quality learnings alone

  7. Well what happened then, cause the game is absolutely boring and from what I can tell very little has been updated.

  8. That's Nice but every Time i look at a vidéo of funny game bugs there's sea of thieves in it…

  9. Over complicated bullshit ideas and bullshit talk in this video.
    When you have a idea of a game, get a team, be a player to understand what players really want and simple create that said game. Thats it.
    And Sea of Thieves is not a AAA game… ffs.

  10. why do people keep coughing in the background? lol

  11. Graph at 6:37 explains Cyberpunk. They are the red line, and only about halfway along it too

  12. the bug fixing phase is the deadline phase? wow…how fckin backward is that? If thats the case every single game might aswell be early access.

  13. But can this Continuous Delivery model be used for proper full-on Singleplayer games that do not operate under the GAAS model & will be released only once to the consumer to experience the campaign once?

Leave a Reply

Your email address will not be published. Required fields are marked *