The Power of Layer Instancing for VFX Scenes

  • Film
  • Workflow

Hello! I’m Casimir. I used to work in the VFX/Animation industry back in Paris for 4 years and in September I had the opportunity to join Allegorithmic as a Texture Artist in Montreal.

Last month we visited Vancouver’s big VFX & Animation studios, and so to prepare for the trip I decided to work on a very large-scale asset. I found this Parisian building, composed of more than 2.5 million polygons, and I made 44 UDIMs for all the elements. This was going to be an interesting challenge, especially given that the demo would have to work live on a laptop.

I chose this asset because back at Illumination Macguff I made a lot of buildings for Despicable Me 3, and I used Substance Painter for that (2 years ago). This was often quite challenging from a technical perspective, and so I wanted to test the new version and see if the new features helped the artist to texture this kind of asset.

The experience was incredible: after struggling for two days with the UVs, I textured it in less than a day! The new instancing feature was key to this, and so in this post, I will explain the process I went through to texture and render the asset.

1. Working with UDIMs

First of all, to organize my workflow, I always prefer to arrange my UV tiles in rows: the first row will be the sidewalk, the second will be the restaurant, and so on.

2. Creating the Substance Painter project

I then usually start with a low-resolution project (as I can always go higher later if necessary). 1024 is a good choice when you have a lot of UDIMs and you’re working with a laptop like I was for this demo that we had to demo live at VFX studios.

3. Baking additional maps

When I bake textures for this kind of huge asset, I try not to increase the size of my project too much.

Here are a few tips to help with that:

  • The normal map is useless because I don’t have a low-res mesh.
  • The ID map isn’t needed either for most of the parts of my mesh.
  • Thickness isn’t needed because I’m working on a hard-surface model.
  • I lower the baking resolution (to 2K) for the world space & position maps.
  • However, I do need a higher resolution (4K) for the AO & curvature maps (which are needed for Substance Painter’s mask generators).

4. Instancing: the feature that changed the game

Like I said before, I like to be organized. So I create folders for each material that I’ll need for my asset: dirt, stone, metal, paint and so on.

Then I start to instantiate every folder to the right UDIMs (those where the relevant materials are assigned). Finally, I use fill layers, building materials with generators to add variations and wearing as needed, that then spread to multiple UDIMs at a time thanks to the instancing. With this technique, I can easily find what I want to modify.

Also, when working with fill layers I am very selective about the channels I use because sometimes normal, height or metallic aren’t needed. This reduces the computing time to process all of your actions.

I always start my look dev very freely. But when I’m happy with a result, I try to reduce layers and materials as much as possible. Often, just two or three materials are more than enough. For example, I have a single dirt folder instanced across all of the UDIMs for the overall look of the asset. Keep it simple.

When I’m working on a part of the asset, I like to increase the resolution and try to get the best quality/performance ratio. I was impressed at how fast Substance Painter could compute my channels even at 4K on 8 UDIMs at a time (on my laptop!). Sometimes you just have to wait 1 or 2 seconds to let Substance Painter process the generator on all the instances, but then moving the parameters is really smooth!

A quick tip when you want to remove all the instances of an instance: copy/paste the parent and then delete the old one. It will remove all of the instances on every UDIM but you still have your original layer (or folder) copied.

And another useful tip when working with UDIM-based assets: Ctrl – Alt – right-click to select a UDIM in the viewport!

5. Previsualization in iRay

When I’m done with my cleaning process, I usually start using iRay for test rendering. This path-tracing tool provides me with the closest result of what my offline renderer will ultimately render, especially for the roughness maps, where a more contrasted range is visible. This way I don’t need to export my textures multiple times.

6. Exporting textures

Having done my look dev for the asset, it’s time for the export process. Since I render my assets with Arnold 5, I use the default template provided by the software (Arnold 5 UDIM) and set everything to a higher resolution, usually 4K, sometimes 2K. The total export time took less than 10 min for 44 UDIMs ( 5 x 44 = 220 maps in total!).

7. Rendering in Maya

In my Maya scene, I just create one shader, load my maps and plug them into an aiStandardSurface.

Make sure that you have selected the UDIM option in the properties of each file node. This time I also used a SkyDome light using an HDR from Substance Painter (corsica_beach.exr). Then click render and away we go, it’s as simple as that!

I already loved using Substance Painter, but now with the instancing feature, I can honestly say that it is THE look dev tool every artist wants. Instancing is a very young feature; it is very powerful but does still have some limitations. Here’s what I think we can improve:

  • Ability to modify properties of multiple UDIMs at a time (resolution, channels, etc.)
  • Painting across UDIMs
  • Ability to bake maps based on the cage of the whole object and not by UDIMs (for a real seamless experience)
  • Ability to see all of the UDIMs in solo mode when you isolate a mask

But this really is a new turn for the software, and I love it! And rest assured that the team is well aware of the limitations and are working on it.

Read more