(September 23, 2015) The success of sports games relies on engendering a sense of reality – not just in the ebb and flow of the sport itself, but in selling the player on the fantasy that they really are in the sneakers of their favourite players.
To capture that sense of authenticity in its famous basketball series, 2K relies on the fidelity of Faceware...
Video game facial animation has come a long way, from the first twitch of Mario’s moustache to the near-Hollywood production levels of titles like Grand Theft Auto and Halo. Today’s characters can emote with a visual fidelity that’s long since leapt over the Uncanny Valley and entered the realms of true-to-life realism.
But dramatic impact isn’t just the domain of narrative-led action titles. A sense of believability is also key to the next generation of sports games – games like NBA 2K16. Not only must these games accurately reflect the strategy and physicality of the sport that they emulate, but also the intensity, the passion, and the all-out drama experienced by the players.
That’s where Faceware Technologies
“I’ve been working on the NBA series for six years now, and our end goal hasn’t changed in that time – we just want to get better and better at selling the emotion of basketball,” begins Anthony Tominia, performance capture stage manager at 2K. “We’re working on 2K16 now, and we want everything to be as realistic as it possibly can be. That means everything from mo-capping the basketball itself to realistically representing the real-life players in digital form.
“This year we’re trying to get more expressive emotions; better-looking lip sync in our story mode; distinct reactions that mirror what happens on the actual court – Faceware Technologies is an important contributing factor to all of that.”
Capturing every wrinkle
2K has used Faceware’s hardware and software solutions for it facial performance capture for many years, and utilised them once again to bring NBA 2K16 to life. For Tominia, the technology was nothing less than integral in capturing the raw kinetic energy of the sport.
“Basketball is a story, and emotion helps you to tell that story,” he says on his team’s approach to NBA 2K16. “Those physical facial expressions add to the drama of the game. The player might get irritated if they miss a shot, or reach for the skies when they make that last-minute alley-oop, and seeing that makes your actions all the more engaging. We really wanted to delve deeper into those little narrative movements this time and sell what’s happening on an emotional level.
“So, in 2K16 we’ve really exploited our rigs to the extreme so we can match every nuance of the person,” continues Tominia. “We’re making sure that not only does each digital character look like their real-life counterpart, but they also show the signature emotion of that person. When you look at Shaquille O’Neal this year it’s not just a representation of him – it has every tic, every subtle nuance that makes his face uniquely ‘O’Neal’. It’s not just a celebratory face, it’s Shaq’s celebratory face, based off his real-life performance. And that’s thanks to Faceware.”
Faceware’s technology was used to capture the facial performance of over 30 different NBA athletes, including O’Neal. Actions both on the court and off were recorded, with players undergoing full performance capture for their in-game animations and also recording dialogue for the My Career story mode. Once this acquisition was complete, it was time to transplant the performances into the game.
To make the next steps of the process more streamlined, the 2K team dived into the Analyzer and Retargeter scriptability.
“The great thing about Faceware’s script interface is that it’s really simple, so we can easily develop scripts that make our job easier,” explains Tominia. “In this case, we wrote a bunch of Python scripts that allowed us to automatically solve everything that little bit faster. There were over 232 lines spoken by each of the 30 players, and we found that we could easily batch, through Analyzer, 232 lines per player without having to even touch the data.
“We did the same thing with Retargeter,” he continues. “In our pipeline the Python script does the retargeting, creates play blasts and saves those files immediately for review. The process doesn’t have to be manned, so we could capture all day, then leave the system to analyze and retarget overnight. We would then just come in the next morning and quickly identify which files needed the most touch up and pass those on to the team. The speed could not be better.”
Speed and power to rival LeBron
The ease of use of the software, along with the custom development done to streamline the pipeline, meant that after analyzing and retargeting the data, clean-up time on NBA 2K16’s animation was down to three-four minutes per second of data.
“At the end of the day, this kind of work will always rely on the touch of a human animator to flesh the capture out – that’s just a matter of fact,” begins Tominia. “However, in our line of work we want to get as far into the pipeline as we can before a human has to touch that data, and Faceware delivers excellent results in that regard. Four minutes per second wasn't really that much time, because there was nobody touching the file until that point, and that’s a huge advantage in a production such as this.”
Combined with the raw power of Faceware’s solutions, this speed makes for a particularly lethal combination. “Last year for NBA 2K15, we recorded 120 hours of facial animation – that’s massive amounts of data,” says Tominia. “In comparison, most feature films are two hours long and have a maximum of three CG characters on screen at any time, so that’s just six hours. But we did 20 times that in the space of a year.
“When you put that into context, it’s insane,” he marvels. “A keyframe animator might be able to give you five minutes of animation a day, and that’s pushing it. That would take one animator nearly four years of nonstop work to achieve, and if you’re paying them $30 an hour it’s going to total up to an incredible amount. Really, there’s no number of animators on the planet that could be assembled to put out 120 hours of animation in a year. But Faceware’s tech enables that, so you’re making an incredible amount of savings, both in time and money. Faceware really is just so good at what it does – in my experience, its solutions are the fastest that are out there.”
Working as one
When you’re depending on a solution such as Faceware to complete work of such high quality, you need to ensure that you have the support in place to keep things running smoothly. With this in mind, 2K has developed a strong and lasting relationship with Faceware, one that keeps the development of the series on schedule even during challenging times.
“If you have any questions or run into any problems with anything, from scripting to hardware, it’s as easy as picking up the phone and calling Faceware – they’ll have your answers straight away,” beams Tominia. “A great example of this was when we were shooting scenes with multiple helmet characters, and the director decided that we needed four more within two days. We’re up here in Novato and Faceware is down in LA, but we picked up the phone, told them what we needed, and they got the extra cameras built and sent here in time.”
Faceware also went as far as to build a custom camera for the NBA 2K team – one that could capture data at the higher, non-standard framerate the team utilises for the game’s production.
“When it comes to the amount of help Faceware extends, it really feels like you’re dealing with a small company, but that’s not the case,” says Tominia. “I know Faceware deals with a lot of people and works with many different companies, but you really wouldn’t know. It’s impressive that they can make you feel like you’re their only customer, even with all that other stuff they have going on.”
The next level
The end result of this collaborative relationship is some of the most convincing facial animation in modern sports games. Every wince, every cheer, every determined glance towards the net has been captured in absolute detail, and all of it, no matter how subtle, feeds into that all-important sense of raw emotion.
“Faceware really has given us limitless volume in our facial animation,” concludes Tominia. “What we’ve achieved with NBA 2K couldn’t have been done without its solutions.”
Copyright © 2015 Grammatik, All rights reserved.