The visual effects studios recreated 1960s New York City, complete with city street and subway chases – and digital horse - and an ancient undersea trireme shipwreck dive that included digital doubles, digital creatures, and dry-for-wet effects, respectively, on James Mangold and Lucasfilm’s final ‘Indiana Jones’ cinematic franchise adventure.
The action gets fierce both on land and beneath the sea in Lucasfilm’s Indiana Jones and the Dial of Destiny, which sees Harrison Ford once more reprise his iconic swashbuckling archeologist persona, this time under the direction of filmmaker James Mangold, who takes over from Steven Spielberg for the fifth and final installment of the cinematic franchise. “There was every challenge on this show,” states Julian Hutchens, VFX Supervisor, Rising Sun Pictures. “Large scale environments, choreographing complex action, demanding effects simulations and crowds, complex face replacements, nuanced DMPs, CG horse, trains and cars, rain, and storms. The mantra was if it feels real and as though it could have been captured in camera and supports the story, its job done.” Collaborating alongside RSP was Important Looking Pirates, which viewed the project as a dream come true. “At a personal level the opportunity to contribute to such an iconic franchise felt incredibly fortunate,” remarks Nicolas Hernandez, VFX Supervisor, Important Looking Pirates. “Indiana Jones brings back some fond memories from my childhood and holds a special place among my all-time favorite movies. The entire team at ILP shared a similar sense of gratitude and appreciation.”
Extensive reference material was provided to the VFX studios by Production VFX Supervisor Andrew Whitehurst. “We received wonderful concepts and period reference photography prior to our shot turnover from production as well as an extensive New York photo shoot of buildings and streets,” remarks Hutchens. “These concepts really established the mood and energy for most of our key sequences, like the moon landing parade, Idlewild Airport, train chase and Indy’s neighborhood. The mood, texture and detail within those frames set a benchmark for how dirty and rich the visual palette was going to be. We also had a stable edit of each sequence which allowed us to plan our builds with the big picture in mind.”
For ILP, storyboards were first created for the wreck dive sequence they handled. “We collaborated with their internal team to previs the entire sequence,” states Hernandez. “Given the tight schedule and workload, it was a logical decision to divide the tasks, allowing us to work on the assets simultaneously. In addition to this, we took on the task of concepting the majority of our assets, such as the trireme wreck, eels, creepy crawlies, and environment extensions.”
According to Hutchens, a variety of creative and technical challenges were anticipated and encountered along the way. “Each sequence had its own creative and technical challenges to resolve. One example was the subway chase sequence, which was editorially made up of onset photography shot within the practical station built onset, bluescreen stunt work, and full CG shots. The challenge creatively was to choreograph all these elements via layout postvis to add maximum drama and energy to the near escape from both trains within the tunnel. We built a long section of tunnel based on onset reference to stage the drama between Indy and the two trains. Every shot had to aid the story and build up the drama across the edit.”
ILP had to design most of the shots above water as the protagonists sail through the Aegean Sea. “Continuity of CG sky and sea, boat extensions and wake FX, during the progression of the sequence, required precise attention,” remarks Hernandez. “Their adventure continued underwater, in the search for an artefact within a sunken Roman trireme. The wreck dive necessitated the creation of complex and intricate CG elements, including the underwater environment, the trireme wreck, digital doubles with diving equipment, underwater creatures such as a swarm of eels and fishes, and a wide range of underwater effects. It was evident that the wreck dive would become the most challenging sequence of the entire project.”
RSP was the second largest VFX contributor on the film after ILM. “In total we were responsible for 301 shots and were in post-production for a little over a year,” states Hutchens. “The majority of our work took place in the second act within New York and RSP was the only major vendor on that portion of the film, aside from a small inhouse VFX team. However, we did collaborate with ILM by providing additional supporting shots for the final act, including shots looking out from the Heinkel aircraft windows toward the foreboding storm and into the clear sky as we journey into the past.”
Extensive R&D went into the crowd pipeline and streamlining workflows for ingesting onset 3D captures of extras that retained as much clothing folds and texture detail as possible without having to do extensive additional modelling. “Meaning we could create more visual variation and complexity at a much more efficient rate,” Hutchens shared. “And as that detail was all stored in vector displacement maps, we could reduce the detail as required for computing efficiency.”
ILP worked on 250 shots over a period of 15 months and did not have to alter its software and pipeline. “One significant decision worth mentioning is the approach we adopted for underwater lighting,” observes Hernandez. “After thorough evaluation and testing, we opted to replicate the underwater lighting directly in 3D instead of going into full deep renders and achieve the look in comp. This workflow played a pivotal role in the project's success, enabling us to do faster iterations and more accurate previews.”
Current day Glasgow had to be transformed for the film into New York City circa 1969. “The key to transforming Glasgow into New York was to really embrace what was in the plate photography, however minimal,” explains Hutchens. “We knew we wanted to keep at least the bottom story of every building, complete with its set dressing and adornments to ensure our extension was grounded and based on something tactile. Pushing us to live up to that level of detail and imperfection in our extended world. Once we had believable top-ups extending out of frame to New York City heights the remaining street extension was approached in a more procedural way. By blocking out buildings based on existing New York survey data and making sure we kept the heights period accurate prior to adding the additional architectural adornments and details via procedural means. We wanted to make sure each shot said, ‘New York’, from iconic yellow traffic lights hanging out over the street to rooftop water towers.”
For most of the wreck dive sequence, stunt divers performed in a water tank situated at Pinewood Studio. “The art department constructed a small section of the wreck for this purpose,” states Hernandez. “Later, this physical build was seamlessly integrated into our digital wreck environment and adorned with underwater plants and additional props. As the adventure continued to the Cave of Dionysus, the actors were filmed climbing a practical rock facade with a greenscreen background. We were tasked to extend and enhance the environment, and as they ventured deeper within the cave, we added CG creepy crawlies plus enhanced the practically built rock passage leading to Archimedes’ sarcophagus.”
The heaviest simulation for the Apollo 11 tickertape parade was the thousands of cheering fans that had to be created because COVID-19 restrictions reduced the number of onset extras available. “To achieve this, we conducted several motion-capture shoots to make sure we had a diverse library of actions and behaviors to help bring the parade to life,” states Hutchens. “There was cheering, ribbon throwing, holding balloons, banners, and waving flags to name a few of the nearly 100 unique actions. All our crowd agents were built from the enormous library of extras which were captured onset. We procedurally extracted the vector displacement map detail with accompanying colored textures and applied that to a number of varied low poly ‘body socks’ to ensure detail was only engaged when required at render time and based on needs to camera.”
Dry-for-wet photography was predominately utilized for closeups to ensure the performance of the cast came through onscreen. “The talent wore a minimalist diving suit costume filmed at high frame rate on a bluescreen stage,” reveals Hernandez. “We had the challenge of matchmoving their actions, replacing and simulating their hair, adding visors to their goggles, breathing cables and making props and features appear as if they were underwater [such as mask straps and other objects]. Additionally, we ended up simulating FX breathing bubbles, particulate, sediment, volumetric beams from their torchlights and obviously integrating them with our CG underwater environment and sea life. As the dry-for-wet and water tank source material were filmed in a controlled environment, there was a lot of fine tuning and color grading required to mimic the color wavelength absorption relatively to the depth level. Volumetrics and murkiness required a lot of balancing and refinement in comp. Storytelling and visibility were key, so we had to find the best compromise to keep the shots as realistic as possible without damaging the action clarity. Additional effects such as animated caustics [synchronized with the 3D sunlight volumetrics] and underwater lens aberrations were key to blend the divers into the CG environment seamlessly and realistically.”
Not everything In New York City happens at street level; a subway chase sees Indiana Jones attempting to outrun an oncoming subway car while on horseback. It was important for RSP that enough detail was built into the CG extension to ensure a seamless integration with the practical set. “All wide shots of the horse were captured in camera, so we decided on a CG build to replace the onset buck used in the mid to closeups, which came with a number of complex challenges,” reveals Hutchens. “The run cycle had to feel natural whilst the head movements also had to carefully cover the onset buck to avoid additional complex paint when overlapping hero talent. We brought life to the horse by adding subtle blinks, twitches, and flowing hair where the practical buck was lacking this life. By using a digital replacement, we were able to control the lighting to better integrate the horse in its surroundings, in particular within the subway tunnel.”
“We had a lot of fun scaring Helena with the centipedes crawling towards her face, or placing loads on her backpack,” laughs Hernandez. “The spider on Indy’s hat trying to get to his face was also a fun addition. References inspired us. For instance, we particularly liked a video of moray eels aggressively attacking fish. The client appreciated this idea, and we incorporated it into a few shots accordingly.” A rig template was developed for the CG divers which enabled a variety of props to be attached to each of them. “The animation of the breathing tube and the different props involved a combination of both animation techniques and effects,” Hernandez continues. “To mimic the feeling of buoyancy and current-like slow-motion underwater movements, we simulated many parts of the characters, including hair, vests, props, and mask straps. Reducing gravity and volume advection with added turbulence were key in achieving realistic underwater motion. Additionally, we computed tension-based attributes to create subtle wrinkles when the diving suits compressed and stretched. Proprietary procedural rigging tools were used to build eels and creepy crawlies. “Due to the large numbers of creatures we wanted to be able to populate our shots without having to hand animate everything. We developed an intelligent particle system to solve the creatures motion path, using a combination of goal seeking, flocking and proactive collision avoidance. Results were most impressive, and it was difficult to distinguish between animated and the simulated ones.”
“When I first looked at the various sequences my initial reaction was, yikes how are going to do that crowd,” admits Hutchins. “There was a very thin layer of extras on set and therefore a much heavier reliance on hero digital crowd extras. Making sure we had enough detail and variation in look, movement and action was the biggest challenge for us; that is one of the things I love about visual effects. Starting with, ‘How on earth are we going to do this?’ and then working together as a team to figure it out and then executing an impressive final shot.”
Revealing a complex shot that occurred during the wreck dive sequence, Hernandez notes, “One of the dry-for-wet shots, became ‘almost’ full 3D as the actor's performance was not working in the edit. As a result, we had to enhance and refine a few assets for close-ups; our digital doubles were not designed to be showcased so up close. Additionally, the eels featured prominently in frame and the effects were quite intricate and required special attention as the breathing bubbles and flare ignition are scale dependent.”
Hutchins is looking forward to people seeing the entire second act. “It takes place in New York City, and hopefully audiences enjoy the story and are not even aware visual effects were applied throughout the sequence.” Picking one favorite moment is not easy for Hernandez. “If I had to choose, the wreck dive sequence stands out in my humble opinion. The part where the eels' nest is disturbed is spectacular and pays a nostalgic homage to Indiana Jones' deepest fear of snakes.”
Trevor Hogg is a freelance video editor and writer best known for composing in-depth filmmaker and movie profiles for VFX Voice, Animation Magazine, and British Cinematographer.
