Visual Effects Supervisor Simon Jung and Animation Supervisor Dennis Yoo talk clickers, bloaters, giraffes, and fungi, some of the captivating and ghastly visuals they helped produce for HBO’s critically acclaimed and wildly popular series, nominated for 25 Emmys including Outstanding Visual Effects in a Season or Movie.
There are zombie series and there are zombie series; there are video game adaptations and there are video game adaptations. And then there’s The Last of Us. Acclaimed as among the greatest video game adaptations of all time, The Last of Us was created by Craig Mazin and Neil Druckmann for HBO, based on the 2013 video game developed by Naughty Dog, where Druckmann is co-president. The series has been nominated for 25 Primetime Emmy Awards across virtually every category, including, not coincidentally for our purposes, Outstanding Special Visual Effects in a Season or a Movie.
That the show should be nominated in the VFX category is not entirely surprising given that a substantial portion of the shots were the work of the leading New Zealand-based studio, Wētā FX. Since its founding by Peter Jackson and his associates in 1993, Wētā has worked on well over 100 major movies and series, chalking up seven Academy Awards along the way, and has been consistently among the foremost innovators in the digital entertainment domain.
For The Last of Us, Wētā’s work included a dizzying array of creatures, both real and imaginary (think clickers and bloaters), as well as highly detailed fungus. We spoke with Visual Effects Supervisor Simon Jung and Animation Supervisor Dennis Yoo, who were only too happy to share some of the secrets of creating hyper-realistic toxic fungi, as well as some of the less unsettling aspects of their endeavors.
AWN: Why don't you start by describing the scope of the work you did on the show?
Simon Jung: I think we delivered 482 shots total, or thereabouts, and the work spanned across at least six or seven episodes. It ranged from really simple things like weathering or overgrowing buildings a little bit, all the way up to full CG shots with hundreds of characters in them. A big focus of ours was the character work, which included the clickers in the Bostonian and in the battle scene, the bloater, the child clicker, the giraffes in the end sequence, the deer that Ellie shoots, the monkeys in the lab and the campus. So a lot of creature work and a bunch of the environment work as well.
AWN: Because we see so much creature work, it seems that’s probably become a lot easier than it used to be; but I know, from talking to VFX folks like you, it's never easy. Can you tell me a little bit about what’s involved in creating the CG deer and other photoreal animals on the show?
SJ: As you said, things become easier – but then the expectations rise. As with computing power, computers get better all the time, and then we keep loading them harder and harder. So, in the case of the deer, we would traditionally look at behavioral references, how they generally move and all of that. And then, obviously, also the materials. What does the hide look like? How long is the hair? How much wetness is there on the nose? For the one in The Last of Us, they actually had a stuffed animal on set, which was surprisingly good. So we had to combine that with our research as well, and then build a creature based on all of that. There's actually one shot that's a mix of the stuffy and our animated creature.
Dennis Yoo: The industry's standards have changed since we worked on, say, Lord of the Rings. The whole paradigm has shifted from when you wouldn’t see everything come together – the textures, the shading, the animation – until the renders. But, these days, you see a first look in animation. So right in our animation solves, you see it with textures, you see it with shading, you see it with lighting, you see it with certain effects. And we never used to see that before. So, you can have problem-solving within the animation solves. The animation solves and the renders line up way more than they used to, when the solves used to just be gray-shaded, no textures.
AWN: That fundamental change in being able to see a much more finished version earlier in the work – as opposed to waiting for various stages down the pipeline – is that solely because of the advances in technology? And does that ultimately mean you can just do things faster, or does it mean that you can do things better?
DY: I think that change mainly came from gaming, because gaming engines and all that tech got thrown in there, so we can see things faster and clearer. And, as Simon was saying, the expectations just rise. And even if the animation is not that close to what the renders are going to look like, we do get a better idea of it. So, in that sense, we do solve a lot of problems earlier, but it obviously doesn't solve the major problems in rendering itself.
SJ: I think it's also really helpful for some directors who aren't that visual that, instead of looking at a gray render, which has all these faults and folds in it, they get a much better idea early on what a shot might look like. And I think that accelerates the creative process a lot – it might tell us that a look isn’t going to work when it's still really preliminary.
AWN: Let's move on to the clickers and the bloater. These aren’t based on a real animal, these are brand new creatures. Did you have any concept art or any finished assets when you start working on those?
SJ: For the clickers, we leaned really heavily on the practical ones that they had built, which looked amazing already. We had scans of those, and we did our development work based on those scans. Especially in the Bostonian, there was a mixture of live-action, and sometimes we had partial replacements, like head replacements. There were others where we only replaced the fungal growth on the head, just to make it look a little bit nicer. We also had a full digital copy that Dennis and his team were able to animate and use. Once we had the two built that we used in the Bostonian, we then built variants, like gender variants and scale variants. We varied the growth of the cordyceps on their heads, wardrobe, ethnicity. We basically built our library up like that, using those Adam and Eve ones.
The bloater was a little more involved in terms of the design process, because the prosthetic that we had looked great, but it didn't quite have the performance that we needed. But once we knew that we were going to free up that creature and that we had to go CG with it, that also opened up the design and the materials. It got us pretty excited, because obviously that's a really great opportunity to make some changes and optimize it in a way that helps the performance.
And it was a real collaborative process with the filmmakers, as well as with Neil and the Naughty Dog crew. They sent us really good references. And on our side we had Gino Acevedo, a long-time character developer, who was involved in Gollum and the Goblin King. He has been doing this for many, many years. So it was a really nice collaborative effort to get the best-looking creature out of that process, and also make sure that we could make that creature move in a way that was helpful for Dennis and his team.
DY: I just want Simon to talk a little more about that fungal work, because it looked amazing. The prosthetic was great, and when Simon got his hands on it, man, it looked even greater – all that detail and the different variations.
SJ: It's hard to know when to stop, and when to keep going into the weeds, but that's a really great point. What's limiting with latex is that you don’t get any light penetration, or any subsurface. We were able to add tiny little fuzzy bits in the crevices, and add slime in places. We had a great time just making these creatures as gross and as scary as possible, especially in that fungal area.
AWN: Was there any reference for the fine detail that you were adding?
SJ: We used real-life reference from fungi, and we used game references. At some point, Neil even gave us a little vault of the reference that they had used for the game, in order to get to the look that they got with the clickers. It was amazing for us to follow that trail, to see how they did their look development, because that broke it down in a really nice granular way for us, so that we were able to learn from that and go down a similar route.
AWN: Was there a creative mandate with regard to how far to take it and the tone you were trying to achieve?
SJ: The main thing was to match the practical first, because we had great reference for it, and from there on, together with the filmmakers, work out how to make it basically more scary. And again, using materials to get it that little bit further, so it holds up in a really close-up, 4K TV show, which is pretty unforgiving nowadays. Everything is so in-your-face and so sharp, you can't really hide much. So every little hair and every little detail had to be just right.
AWN: What would you say were the biggest challenges for you and your teams on this show?
SJ: A big challenge was the compressed timeframe – we had to do a lot in a very, very small amount of time, which meant that our scheduling had to be super-tight. The number of creatures we had to do was a challenge, I would say. Within that, there were some creatures, like the child clicker, that had its own little problems. It was really hard to convey that it's a child, because you only have a very small canvas to sell it on, since three-quarters of the head is covered in this giant fungus.
DY: The biggest challenge for us was the performances. One of the things that we discovered early on was that we couldn't deviate in the performances because everyone was used to the game. There's this huge fan base for the game. So we based a lot of it on the game, even though we did capturing of different performers. We set up these little pictures of reference of the game within our animation, and we showed Craig and Neil, this is what we're referencing from, I know it's rudimentary, but this is what we're trying to achieve. And then trying to push that even further, so that it looks like the realistic motion that you'd see with live action, which the game never had.
So it was quite difficult, especially within scenes where they actually had a performer act out the scene, and then they decided, "Wait, actually this isn't holding up," or things are changing where they want to change the direction. So even though we had motion-capture, we keyframed a lot of it, sometimes even changing it completely from the motion-capture, especially that clicker scene, the one in the Bostonian, the really long shot where Pedro Pascal is hiding behind the glass cabinet. That was tough. That's actually all keyframed until the end, and then there's a little bit of motion-capture in there.
And then the other challenge was the bloater, where there were questions whether they'd keep the live-action performer in there with the prosthetic. And then when we started seeing it in the cut, we all realized that the weight, the gravitas that he needed to have, wasn't there. You have a big man in a huge rubber suit, and it looks like a big man in a rubber suit.
That was actually an interesting process, because a lot of the performances that we tried in the beginning was based off of that suit. It was a really asymmetrical design, where he had one leg that was extra big, extra tree trunk-like, and the other one was thin. And so we tried to animate that, but having somebody so asymmetrical brings about this disability look, which they didn't want. They didn't want a person like Captain Hook stomping around. They wanted something that portrayed strength. That's where Simon changed the design with Gino and the Naughty Dog team.
AWN: Even if you end up replacing the performance, is it always better to have something shot live, to have something in a plate that you can work from? Or are there some instances in which it's not necessary and may even be distracting?
DY: I actually always think it's better to have something in the scene, so that the actors can see somebody, and act with them. When the actors don't have anyone to act with, it's really difficult, because they're not looking the right way, they're not even acting the right way. So having somebody in the scene that they can act with is always a plus.
SJ: It's a two-edged sword, because if the performance changes quite drastically, it means we have to paint it out. There were some scenarios where the bloater smashes a soldier out of the way. That performance changed completely, and the bloater now smashes down on a soldier. That means we have to paint that one soldier out, restore what's behind it, and then add a CG soldier that the bloater can interact with. So it's a two-edged sword. At the same time, it's awesome to have a perfect lighting reference, for example. And, as Dennis said, it's great for the actors to have a real person to interact with.
Lastly, it's also maybe good to have something that we all agree doesn't quite work for us. That's a good starting point, I think, where we can say, "This performance doesn't work because he doesn't have the weight, he's not managing enough, he's probably a little bit short, and whatever.” That already gives us a few good hints as to how we can start with our CG replacements.
AWN: Is there anything else that you’d like to share about the project?
DY: I just want to say that no one ever questioned whether our giraffes, the ones that we made, were real or not. They thought they were real. They were always questioning the real giraffe. So, high five, Simon!
SJ: The one thing I would like to share is that, for a lot of people that we had on our crew, working on The Last Of Us was an amazing opportunity, because a lot of people just love the game. We had a lot of people begging to work on it. And that made for an incredible team, because everybody was excited, everybody put their best foot forward. Not that they don’t always do that, but it was just really noticeable, the enthusiasm and the skill that everybody showed, and everybody constantly trying to push everything a little bit further. And then also seeing it being received so well, because, when it started airing, we were still working on episodes. So that was a really uplifting feeling.
DY: It's great when projects are like that. As Simon was saying, having fans for a crew, building an episodic TV show like this, was inspiring. It kind of brought us back to Lord of the Rings, because we were all fans of the stories. It totally helps when everyone gets excited about things.
