Using customized Nuke tools, Sony Pictures Imageworks created the film’s signature graphic look almost entirely in comp.
Spider-Man: Into the Spider-Verse wowed audiences around the world on its way to picking up an Oscar for Best Animated Feature this past February. Much has been written about the unique look of the smash hit film, which evokes the feel of a comic book (right down to the Ben-Day dots). Central to the film’s groundbreaking visual design was how Sony Pictures Imageworks (SPI) customized and harnessed the power of Nuke, from Foundry, to provide the technological edge needed to bring the studio’s breakthrough creative masterpiece to the screen.
Rewriting the Visual Rule Book
Spider-Man: Into the Spider-Verse is by far the most comp-heavy animated feature SPI has ever produced. The signature Spider-Verse graphic look was created almost entirely in comp, which meant every shot had an extensive amount of work done in Nuke.
The comic-inspired look of the film involved a complicated production process. It required a completely new toolset -- the SPI team did a considerable amount of work developing and refining their process into a series of granular tools and templates artist-friendly enough to be utilized on every shot in the movie.
Geeta Basantani, lead compositor on the project, explains, “We created over 25 individual tools for the compositors, and probably just as many templates. The most widely used were the Hatcher and Thresher tools, which created the most recognizable printed comic look of the film, in the halftone dots and hatch lines. But we also had tools for offset colors, the accent ink lines, outlining, smearing, brushing, streaking… the list goes on! Pretty much every part of our traditional pipeline was reworked in some way to achieve the unique look of Spider-Verse.”
“Because so much of the look of the film was found in comp, we gave the artists many toys to play with, and they were always finding new exciting things to do with them,” Basantani continues. “For instance, ChromaShifter was a tool originally created for handling the color offsetting effect we used for depth of field, but we found that by swapping motion vectors for depth and pushing it to 11, we got that unique motion-trail look on things. The final look of the film really comes down to the creativity of the artists working with all of these tools.”
Custom Tools That Empowered SPI Artists
Much like the original comics, Spider-Man: Into the Spider-Verse involved an inordinate amount of artistic effort. The extensive range of new tools employed on this show was developed with one aim in mind: to empower the artists to be as creative as possible. Armed with a new array of artistic tools, the SPI production team was able to design and create animation with a highly stylized, hand-crafted look employed throughout each shot of the film.
According to lead compositor Marco Recuay, the customizable nature of Nuke made it the perfect package for the job. “Developing tools in Nuke has always been extremely easy -- from simple artist-created gizmos, to elaborate Nuke Developer Kit (NDK) nodes, there is support for whatever needs we have. On Spider-Verse, we took advantage of mixed toolsets. Some of the more complicated tools were written in a very granular way in BlinkScript or C, and then wrapped up into multiple artist-facing gizmos or templates, to present them with the specific controls needed for the task they wanted to accomplish. This was useful for things like the multiple looks on the show. In Gwen’s world, for instance, things are very streaky and painterly, so while the underlying tools were the same as Miles’ world, we had gizmos and templates that presented and used them differently.”
“In other areas, we would take a complicated setup created by our lookdev artists and boil that down into a gizmo, just exposing the controls the artists would need to work with those looks across a sequence,” he adds. “This was a really important process for our show, as we had to make the tools accessible not only to dedicated compositors, but also lighter-compositors who aren’t always as comfortable working with complicated setups.”
Creating a Truly Handcrafted Look
Basantani also notes that these tools were integral to the entire visual effects production. “Pretty much the entire movie was done with artists dialing shots on an individual basis using Nuke with the tools we developed,” she says. “Like for hatching and halftones, using the Thresher and Hatcher tool to dial them thick and thin based on the shading from lighting. The artists had freedom to dial the size, angles and space between the screentones as well as dial depth of field using our tool sets. For instance, in the train station, the flares had halftones which scaled with light and changed the appearance of them based on how the realistic flares reacted. Halftone dots were sprinkled to represent the glows.”
Recuay, elaborating on the process, explains, “The Spider-Verse look is a fairly complicated process to pull off, and required a lot of artistic work to dial things just the way the directors wanted them. For instance, take the halftone dots you see in rim lights on a character. In some instances, you want those to track along with the character, such as when Miles is swinging by on a web, otherwise he’ll appear to be swimming through a pattern.”
“Other times, you want them to be a little looser, so they feel like lights and not a texture,” he continues. “Being able to adapt the tools to whatever the shot needed meant we had to give the artists a lot of control, and train them on different ways to use the tools. So, while there were a lot of new things to learn, the benefit was that we were able to create a truly handcrafted look on every frame of the film.”
Using BlinkScript to Prototype Tools
The team used BlinkScript extensively to experiment with the different tools they might need to achieve the desired look. “We used BlinkScript as a way to quickly prototype tools in Nuke,” Recuay describes. “The ability to essentially code directly to the viewer was great, and allowed us to create more complicated results quickly. In our workflow, we would then convert the BlinkScript to a compiled node that would be used as part of the artist-facing toolsets used across the teams.”
