Star Wars: Episode IX – ILM uses the Force of Substance

Star Wars fans behold! From space icebergs to the crashed Death Star, today we share how the ILM Singapore team textured environments of galactic proportions, using a combination of Substance Designer and the Substance Automation Toolkit in their assets pipeline.

Introductions

Elvin Siew: I was one of the asset and environment supervisors on Star Wars: The Rise of Skywalker. I was responsible for the asset texture, look development, and hair pipelines, as well as managing a team of artists from building the assets to finishing them in shots. 

Sam Schwier: I was one of the environment and generalist leads on the show, responsible for doing visual development for a few key sequences before the teams ramped up, as well as leading a team across our sequences to finalize those environments.  

Working on the Star Wars Franchise

Elvin: It was both exciting and daunting! We got the The Force Awakens team back together to revisit a galaxy so familiar and yet with many new worlds to explore in this last installment. When the team and I first read the script descriptions and saw some initial concepts, we were blown away, and left the meeting asking, “How are we going to pull that off?” That is why I love my job. 

Sam: The scope of the show was huge. When I was brought onto the production team, we initially looked at concept art and some shots — it got me really, really excited to work on the final movie of the saga. Huge environments, incredibly complex FX work, and the culmination of one of the biggest stories in film!  

Shots ILM’s Singapore studio worked on

We were responsible for four main sequences: the Millennium Falcon chase in the space iceberg, the hangar sequence, the escape from the Star Destroyer, and, finally, the epic duel on the sunken Death Star. 

Art Direction

Elvin: Led by Doug Chiang, Lucasfilm’s Art Department provided a comprehensive set of artwork for virtually all of the sequences and key shots. There’s an aesthetic to Star Wars that is practically baked into all of our cultural DNA at this point. The challenge has always been to find the look and style that fits each film but that also pushes things forward. From the actual model kit greebles used to build the iconic ships to the overall weathering and aging of every single surface, the team had incredible points of reference to create these new assets and environments. 

Sam: We used a lot of references from the original films. The crashed piece of Death Star in the pier battle was all built off a matte painting from the original trilogy, and the main crashed piece of the Death Star that you see in the sea from the clifftop was all based off photos taken of the original model from A New Hope.  

Our use of Substance

Elvin: When faced with the shots, we knew we had to populate these scenes with a lot of complex hard surface geometry. We needed to have both the artistic control and the ability to texture and shade hundreds of unique assets in a short amount of time to give the director a visual representation of the CG environments. One of the best solutions that came to mind was Substance Designer and the new Substance Automation Toolkit. 

Sam: When we first talked about Substance, it was to create a load of Star Wars-esque base materials that we could quickly drop onto our assets. We had a Nix scanner, so we could get the accurate colors of the actual physical models built for filming and use those where applicable. As soon as Substance was mentioned, I remembered David Larsson’s GDC 2018 talk about the Substance Automation Toolkit and pushed to use it for this project, as I saw it as a good opportunity to speed up our workflow.  

Elvin: Substance allowed us to process the enormous number of assets with very quick turnaround times to get them integrated into shots. We developed an asset pipeline where each time a model got checked in, it would generate material IDs, UVs, and send it to Substance Designer to calculate the utility maps like occlusion and curvature and then bake a complex Substance network onto maps ready for our technical directors to render. It also allowed one artist to maintain and own all the assets in a given environment. 

Sam: We often start off environments by using a lot of tiling and Tri-Planar mapping. The method is great but has its limitations, namely the repetition of the tiling, and also the fact that the Tri-Planar does not carry through our pipeline into the various DCCs, so Substance allowed us to have baked texture for our animation playblasts, for example.  

For future projects, we would use it more to process heavy assets or large quantities of data to help us work in an iterative non-destructive workflow. It has also gotten artists to think more creatively to develop systems procedurally in order to achieve visual complexity when there’s a lack of available photographic textures. 

One of the advantages we had with the Substance toolset was its ease of use, and a wealth of useful tutorials online to get you started. It meant we had artists up and running very quickly. Substance Automation Toolkit was a different story though!  

Substance Breakdown

The whole sequence of Rey in the Death Star wreckage was the first use of our new Substance pipeline, and all the assets were finalized with Substance. This sequence was started with SAT but actually was finished through Substance Designer and Substance Painter in the end. We also used Substance for the Ice Tunnel Chase sequence and the epic duel on the Death Star wreckage. 

Substance Automation Toolkit

We treated pretty much all the hard surface assets on the Millennium Falcon Ice Tunnel Chase and the interior and exterior Death Star wreckage through the Substance Automation Toolkit. 

SAT provided the glue to be able to batch process assets with look updates and gave us the freedom to iterate non-destructively. We could make drastic changes in the materials and bake it down into maps to be rendered in various DCC packages. 

The main difficulty was to troubleshoot why an .sbs would not be processed by SAT. The lack of traceback and debugging tools made it tricky to go through complex networks to find out which node or relationship was causing the automation processes to break.  

Substance Designer

We started by making sure we had a one-to-one match with our proprietary shader so we could seamlessly integrate it into our various DCC rendering packages. 

We then tasked artists to take the concept art and on-set references to start developing the various materials in Substance Designer, and then once it was visually approved to prepare it for the automation process.  

Tips for environment work

Always put your environment work in context. Establishing the context will help push the environment work forward in meaningful ways. Blocking, dressing, and lighting. 

Who is your favorite Star Wars character? 

Elvin: Darth Vader! 

Sam: Chewie, ARRRGGHHHH! 

All images © Lucasfilm Ltd. All rights Reserved. Courtesy of Industrial Light & Magic.