Nailing Photorealistic Facial Expressions for VFX

Creating a Bedouin Elf Character with Tomas Sackmann

When I was a kid, I was never good at drawing or painting like a lot of artists in the industry, but I always had a passion for creating in general. Maquettes with ice cream sticks, small sculptures, playing with Lego, and all these things always kept me entertained. But honestly, I would never think that this was going to be so important in my life. 

After finishing school, I studied industrial design at the University of Buenos Aires, and there I got introduced to my first 3D software. During the first year, while some of my colleagues were still doing a big part of the work by sketches and drawing, I did a short course in Rhinoceros and, from then on, I shifted all my design process to 3D. In some way, this was not the best from an industrial design perspective because I never developed my hand drawing skills. But on the other side, this introduced me to this amazing world. 

My passion for design and for 3D made me orient my career to experience and spatial design, always using 3D as my main tool. For the last few years, I have been working mainly on interiors, events, and products using 3ds Max and V-Ray. 

After working for years in these industries, I decided that I needed to push my 3D skills further as I realized that the part of the process that I enjoy more is the creation of this world in 3D.  A colleague from my previous work started studying at Think Tank, and after seeing his level of work and how much hard work he had to put into it, I decided that this was going to be my next challenge. 

I joined Think Tank 1 year ago, (where I got introduced for the first time to software tools like ZBrush, Substance Painter, Maya, etc.) with the objective of just using it to improve my 3D skills for my design process, but afterward, I realized that the worlds of film and gaming really fascinate me. 

Bedouin Elf Project

Bedouin Elf Tomas Sackmann

The Bedouin Elf was my final project for the advanced term at Think Tank. In this project we had the freedom to choose any concept that we liked to reproduce in 3D. 

Honestly, at the beginning of the term, I was planning to do a project with a mix of organic and some hard surface elements to work on both skills, but when I saw the original picture from Robin Moore, I fell in love automatically and I thought this would be a great character to design as an elf.  

I think the photograph tells a long story of the life of this man. Looking at how the character looks at the camera with that smile on his face, you can see in his wrinkles that this smile followed him in all his life’s battles.  

The idea was to tell the story of this Elf as best as possible with one image and mainly transmit the emotions of this character to the viewer through his expression. I wanted people to sympathize and feel a connection with him.  

Original photography “Smiling man”, by Robin Moore

Substance Painter

I love to work in Substance in general. It allows you to be creative with a straightforward workflow and removes a lot of tedious work. But until just a few days before I started this project, Substance had a strong weakness for my workflow: It did not work with UDIMS.  

In my previous project, I used Substance to develop the textures only for the hard-surface objects, because they were divided into separate pieces, but I could not use it for the character as it had a few UDIMS on the face and body. 

But just a few days before I started the Bedouin Elf, I got the great news that Substance added the option of painting across UDIMs, and I decided that I needed to try this new feature. 

3D Workflow

To create this artwork, I used a variety of software to try to mimic as best as possible what a VFX/feature film workflow is. I will include below a bit of a breakdown of it. 

Research of references: During this part of the process, I used pureRef to gather all the references, and I based my research on looking for head/skull anatomy from people from the Asian-African countries to start the sculpt. Also, I researched humans with a lot of wrinkles and, most importantly, cute smiles. 

On the other side, I researched pictures of people in similar living conditions as the character. Specially to understand the level of dust and dirt in those environments, the condition of the cloth and their skin, etc. 

Reference

Sculpt: All the sculpting was done in ZBrush. During the sculpt block out I found out that the project was going to be much more challenging than expected. At the beginning, I completely underestimated how difficult it is to make an appealing smile without making it look creepy. Getting the smile correct took me, more or less, half of the sculpting process. For this, not only the shape of the mouth is important, but also how the rest of the face moves when we smile. For example, if the eyes don’t get a bit smaller when we smile, the smile will look automatically fake. Also, a challenge here was that people from different ethnicities have different types of skulls. 

I started using a texturing XYZ map provided by the school for the skin details, but I was not getting the result that I needed, so I decided to sculpt all the wrinkles and details by hand and using alphas. 

Clothing: The clothing was done in Marvelous Designer and brought to ZBrush to do the final details.  

Retopology and UV: In the middle of the sculpting process I retopologized and UV-ed the character and the cloth in Maya, thinking like a feature film workflow, where the topology needs to hold as many details as possible and it is not a priority to keep the polycount low. 

Texturing: I will come back with a more detailed breakdown of my texturing process below, but, basically, I used Substance Painter and Mari for the character and Substance Painter for the clothing and the eyes. 

Grooming: The hair and beard were done with Xgen in Maya.  

Lookdev: The final scene was set in Maya, and all the renders and animation were done with V-Ray. Lighting was super important to achieve the final look.  

Post: All the postproduction for the renders were done with Photoshop. I started editing every render using the Camera Raw filter and after I was happy with the tones of the image, I used several render elements to adjust the final image. The Zdepth pass was the most important to give DOF and a photoreal look. 

Before / after DOF

With all the renders done, I did the final reel. I composed everything in Nuke. 

Texturing Workflow

All the renders were done in V-Ray, and I used the V-Ray Al Surface shader for the skin, a two-sided material for the clothing, a blend material for the eyes, and V-Ray Hair Next Mtl for the hair. For the skin, I started the wrong way, applying all the maps at the first time, as you can see in the image below. 

I got some great advice from my instructor, Raffael Frank. Basically, the core principle was to start simple and keep polishing and doing more complex material step by step.  

This was a really time-consuming and challenging part, as it took me a long time and a lot of interaction to be more or less happy with the look of the skin — even though I still think this needs a lot of improvement. 

I started doing some test renders with the V-Ray AI Surface material, just with the displacement on but with not any map applied, which allowed me to see if the sculpt was behaving as expected after applying the subsurface. It’s really important to understand that the SSS will eat a lot of the details of the sculpt, so it is key to start rendering early and then to do a back and forth, modifying the sculpt and checking how it looks with the material on.  

When I was more or less happy with the raw material, the next step was to apply, one by one, all the maps and check how each of them were affecting the material. I used Diffuse, Roughness, Specular, SSS, etc. But by doing it this way I realized that I didn’t need so many maps, and I ended with a much cleaner and efficient material and with a better result. 

For the clothing, as mentioned before, I used V-Ray 2-Sided Material to give a different tone to each side as it was on the reference. Also, to give a light, see-through look to the fabrics. 

The materials for the fabrics were done directly with the maps from Substance with the V-Ray Metallic/Roughness UDIM Material exporter.  

For the eyes I used a workflow learned from Magnus Skagerlund, my supervisor in the intermediate term, using a VrayFresnel to adjust the level of reflection depending on the angle of view. All the maps from the eye were done in Substance with a mix of photo projections and some vein brushes from Substance. 

Can you give us a breakdown of your texturing process in Substance? Please add wip visuals for every step to illustrate your process.  

After having the sculpt in a good stage (not final, as I kept retouching the sculpt until the last days), Substance Painter came on the game and I started to hand-paint the Diffuse of the skin. At this point, the new version of substance came out just at the right moment. As I mentioned before, the character was made for a feature film workflow so it has multiple UDIMS, and before the new UDIM support on Substance Painter’s last version, it would have been almost impossible to work so smoothly on a full asset like this.  

The matter of painting directly on the surface across UDIMs made this part of the process much faster and intuitive. At this stage, I focused on defining the color zones of the face and specific details of the character, like the freckles, moles, and birthmarks.  

An excellent feature of the newer version is also the option of masking different UDIMs so they do not compute during the process. This helped me a lot to keep the scene light and easy to handle.

As shown in the images above, the sculpt was not final yet at this point. To keep these textures working during the entire sculpting process, it was important to not modify the UVs. 

In a separate Substance file, I baked the Curvature, Ambient Occlusion, and Thickness maps and saved them externally. With all this in place, I jumped into Mari and continued refining the Diffuse using photo projections and the Curvature and Occlusion maps from Substance as masks to define the details of the geometry. 

After having the Diffuse map almost finished, I used it as a base for the other maps as the Specular, Roughness, SSS, and bump.  

For the SSS masks, I used the Thickness map from Substance with some curves to adjust the levels of SSS in each area of the body. At the end, I did not use the SSS maps to simplify the material, as I mentioned before, but it was great to develop them as a learning process as in case of having a different lighting set up these maps would be really important. 

The clothing was entirely done with Substance Painter. With a mix of a cloth smart material, smart masks, photo projection, and just a bit of hand paint, it allowed me to achieve a really similar result to my reference. 

First of all, I had to bake all the mesh maps on the “texture set settings” palette. 

After having all my maps baked, I started with the Fabric Flannel Tartan smart material and I modified it almost all procedurally to achieve the final look. 

When I use Smart materials, I always like to hide all the layers and start unhiding them one by one to understand exactly what the material is doing.  

Another good practice is to check the preview, not only in the material way, but also checking separately the diffuse, metallic, roughness, etc., to understand better what you are doing with it. 

The base was just a flat green color, matching the color of the reference. After that I started modifying the patterns from the smart material. All the patterns were just a plain color, basically, with a bit of bump and a procedural mask applied to determine where this color would be seen. 

It was key in my workflow to keep all the layers properly named and organized in folders. Without organizing correctly, it would be really difficult for someone else to take on a project or even to come back to it later, yourself, when working with complex materials with a lot of layers. 

I used a combination of different “Stripes” masks for all the patterns (original from Substance) and then used different procedural masks to break the lines and make them feel more organic. 

Here I added a subtle discoloration layer and the normal.   

After having all the patterns done, I think the key to achieve the realistic look was to add dirt and dust layers. Again, almost all of this was done with procedural masks from Substance, like the Dripping Rust, and some external black-and-white grunge textures to determine which areas were more worn than others. 

I added the stitches in Substance just to previsualize how they were going to look, but as I was going to have some closeup of those areas, it was not enough to have them on the texture. So I decided to model them in Maya and scatter them with Mash. 

The green fabric base color was done almost entirely procedurally, but for the Shemagh, I took a different approach: I projected an image from the internet and applied it with a planar projection. 

In cases like this it is not possible to project the texture with triplanar projections, so to have a nice set of organized and properly aligned UVs is key to making your texturing process easier.  

In my case, they were not perfectly aligned because the UV borders were going to be hidden, but they were close enough to allow me to match the directionality. 

After finishing all the textures, it was time to export the maps and integrate them in V-Ray. 

What did you learn from this project, and how do you see your use of Substance evolving on future projects?   

I think the main thing I learned in this project is how important and difficult it is to capture the expression or emotion of a character. Also, how every element of the pipeline is as important as any other to achieve realism. Even if you have a really good sculpt, if your textures and shaders are not at the same level, the artwork will never achieve the desired result. 

In this case, I think that the shader of the skin needs improvement, especially breaking up the specular more.  

On the modelling side, if I have the chance to come back to this artwork in the future, I will rework the topology to support better all the wrinkles of the face. In this case, my current topology would bring a lot of problems to animate. As this project was done only for a still image, this was not a big problem, but it brought me some issues with the displacement map that would be avoided if the topology would be following each big wrinkle. 

Also, I have learned how useful and powerful it is to work procedurally — but never forgetting that it needs to have a sense and always some touches of hand paint in specific areas. To tell the story, it is super important to not have a procedural look and feel, but to perceive it more naturally.  

Substance Painter is an amazing tool and does a lot for us, but it is really important to not forget to keep developing our observational skills, as that is what really makes a piece stand out. Seeing how the software is evolving, I see Substance getting more and more involved in my workflow in the near future.  

Anything we forgot to ask or that you would like to add?  

I would just like to give a special thank you to my instructors Raffael Frank, Magnus Skagerlund, and Rusty Currier, who helped me a lot during this amazing year at Think Tank. Also, to my TTO friends, staff, and, for sure, my family and wife, who supported me during this process. 

Please, if you have any doubts or suggestions about my workflow, please don’t hesitate to contact me on ArtStation or Instagram and I will do my best to help out and to answer any questions. 

Thanks a lot, Adobe, for inviting me for this interview and for continually developing this amazing software and raising the bar constantly.