From VR to 3D Print: Dylan Sisson on Sculpting, Texturing and Pixar’s Renderman

I started in 3D animation when you could get hired if you showed some kind of artistic aptitude and could use a computer. I heard about a studio that was making a CGI RPG, and I applied. I had some background in 2D animation, but no 3D experience. Instead of talking about 3D during my final interview, the owner of the company grilled me about Dungeons & Dragons for an hour, and I got the job. Turned out playing D&D was actually useful. It was a lucky break, and a great group of artists.   

Pixar’s Renderman

I created an animated short in 1998 called Object Lesson with Andrew Woods, which won a few awards, and that led to an opportunity to join Pixar in the RenderMan Products group working on plugins for the first release of Maya 1.0. Today, I provide creative direction for our marketing initiatives and deliver presentations and workshops at various conferences and events. I also get to try out features in early development, like the upcoming “Stylized Looks” in RenderMan 24, our solution for nonphotorealistic rendering (NPR). 

NPR isn’t new, of course, but it’s something that’s generally been computed during compositing. RenderMan has always supported that through a flexible shading language and secondary passes (AOVs and LPEs). What’s interesting about this upcoming implementation in RenderMan 24 is that it provides fast artist feedback and doesn’t require compositing, which makes creating stylized looks super interactive. We’re also leveraging all the built-in path tracing tools (material networks, lights, light filters, integrators, etc.), and supporting unique styles whether the final look is toon, anime, illustrative, or painterly.  

Play

Substance Painter

When I first got into VFX, I was drawn to the power of proceduralism, and I became a bit of a shader geek. Back in the day, RenderMan had a tool called “Slim” for visually authoring material networks, and I would render out procedural “shader swatches” at a large resolution (15,000 x 15,000 pixels) and print them out as giant procedural wall art (some have hung in the Art Gallery at SIGGRAPH). So when I first saw Substance Painter and its procedural approach toward material generation, and how it made the process artist-friendly in a way nobody had done before — I thought it looked brilliant. Plus, Substance made it simple to leverage surface topology (curvature, ambient occlusion, etc.) as signals to drive procedural textures, and bam: the materials looked natural, with dust, scratches, and whatnot all falling auto-magically into the right places on the model.  

Procedural shader art created in Pixar’s Slim

However, I really began using Substance in 2017 when I began creating VR sculpts in Medium and wanted to give them realistic materials as efficiently as possible (i.e., I didn’t want to paint a bunch of detail). After shading a couple VR sculpts with Substance, I haven’t gone back to creating materials in any other way. Instead of taking the better part of a day building a really nice, complex material network for a sculpt, I could use Substance to create a better-looking material in practically no time. After finishing a VR sculpt in Medium, I can export that, create a material in Substance, and have a final 2K render from RenderMan in less than an hour, which is perfect for my prototyping purposes.     

Medium VR

In 2017 I bought a VR headset and checked out all of the VR content creation tools, and I got hooked on Medium and VR sculpting. Picking up Medium itself was super intuitive. When I learn a new 3D DCC (like Maya, Houdini, Blender, etc.), at first, it’s always a hassle to understand how to rotate the camera, create a sphere, etc., but in Medium, you want to rotate the camera? Move your head. Want to create a sphere? Pull the trigger. It makes a lot of sense, and once I learned the basic tools, I could model much faster — and better — than I could before.      

Also, my background is 2D illustration, not sculpting. So the biggest surprise for me was that my 2D drawing techniques transferred directly over to VR sculpting. I didn’t have to “learn” how to sculpt. Another nice surprise was finding that my personal style transferred over to my 3D models (almost by osmosis), giving them my signature look, which is something I could never achieve with traditional DCCs. My very first VR sculpt (below), was a revelation, capturing the looseness of my reference sketch — after just getting into the tool.  

2D Style transferred into 3D with a material created in Substance Painter:

Finally, I wanted to integrate these VR sculpts into familiar VFX tools like Substance, Maya, and RenderMan, which turned out to be easy. The sculpts in Medium are made of voxels, which can be exported as polygonal meshes, so it was easy to prep them for previsualization.  

Bottle Opener

I sorted out a way to print my VR sculpts in solid steel, so I decided to create a functional sculpture: a bottle opener. I started with drawing a side profile of my “Dragon Opener.” I imported the finished drawing into Medium as a reference image, and because it is VR, the reference image can be placed anywhere, at any size. I placed it in front of me, like a drawing table, and drew the silhouette on top of it. Having centered the image at the origin, I enabled symmetry for sculpting and finished the model. (Notice the corkscrew was removed from final design due to functional reasons.) 

Original sketch & imported into Medium:

My tool of choice is the Move Tool; I modeled each individual scale with the move tool. I decided to add a uvula in the back of the Dragon’s throat, so scaled the model way up, so I was virtually standing on the dragon’s tongue while I sculpted it. For the final touch, I added my signature on the bottom using a subtractive stamp (that I had imported as an .obj).  

Final Pre-vis, Signature and Uvula:

With the model finished, I exported it to Maya, where I used a model of a bottle (sized to a real-world bottle) to determine the precise size of the Dragon Opener for the 3D print. I also tweaked the mouth geo of the Dragon Opener so it would grasp the bottle cap properly. 

Sizing the opener with real world units (cm):

With the model finalized, the next stop was Substance Painter to create some realistic materials. I started with a material from Substance Source (a Molten Metal 3D print material), and tweaked that until it matched the look of my steel 3D prints. I exported that material using RenderMan for Substance, attached that material in Maya, and rendered with RenderMan — and voilà, I have a photorealistic previsualization. From the pre-vis, it was clear the signature wasn’t deep enough, so with a couple adjustments the model was final. 

Final model and pre-visualization in RenderMan with materials from Substance:

To prepare the sculpt for printing, I took the model into Autodesk’s MeshMixer. First, I calculated the center of gravity so the Dragon Opener actually stands. Second, I hollowed it out, specifying the minimum thickness the steel material would support, so it would require less material to print and be more cost efficient. With the Dragon Opener prepared for 3D Printing, I submitted it to an online 3D print service and in a week I had a functioning Dragon Opener.  

Functional Bottle Opener:

The Dragon Opener may very well be the first product made in Medium that was sold online, back in 2017 — at least it is the first that I know of.   

Lost Wax Brass Cast:

Inktober

The purpose of this project was to test an upcoming feature of RenderMan 24 called Stylized Looks. This is an entirely new system for creating all sorts of NPR with RenderMan. For the test, I wanted to recreate my own illustrative style that I developed for “Inktober,” a challenge to create a new drawing each day of October. Started by Jake Parker in 2009, millions of people now participate in Inktober every year. I choose my series of drawings from 2019 to serve as the “artistic truth” and target for my Stylized Look.  

Inktober 2019 Series | @real_dylan_sisson

Because this is a loose style, I wanted the geometry itself to have the same loose feel. So I began sculpting the model in VR using Adobe Medium. I’ve found creating the sorts of shapes I might draw in 2D is quite natural when sculpting in Medium.  

To get started, I imported one of my drawings into Medium and used that as a reference, placing it just to the side of my workspace. It also served as a color palette, and I picked colors off of it for the objects in the scene.   

I used symmetry to quickly build out the base of the model, and then turned symmetry off and added variations to make each side unique. I added additional elements, the pens, the ink bottle, etc. I spent about seven hours on it. When I was done, I had created over 40 layers for all the objects, combined them in six layers, which I then exported as a collection of FBXs, with the layers preserved nicely for shader attachment. The final model has a “handmade” shape that worked well for the loose style.  

Next stop: look development in Substance Painter. First, I used the new UV automatic unwrapping tool in Substance, and it worked great. Really simplified my process. Second, I perused the online material library, Substance Source, which has lots of interesting materials and helped make quick work of look development. I just download several high-quality materials, and with a bit of artistic noodling, the model is good to go. In total, I used six materials to keep things manageable.  

With the look development finished, I used the plugin RenderMan for Substance Painter (https://github.com/pleprince/RfSP) to export the materials directly to the RenderMan Preset Browser in a single button-click, which makes the Substance to RenderMan workflow very straightforward.    

After importing the model to Maya and setting up the lights for RenderMan, here is the final PBR image:

Now to create the Stylized Look in RenderMan 24. Here’s a quick summary: It’s a matter of wiring in special NPR nodes to my Maya material networks, adding secondary outputs (AOVs), and dialing in the elements of the look using special NPR Display Filters.  

Five elements are combined to create a style 

The materials created in Substance contribute directly to the final stylized look, and the textures from Substance can be used as signals to drive arbitrary NPR effects. Also note, this stylized look is created entirely in camera, without any compositing, so these looks can be created during an interactive session.  

Building the look during an interactive session:

Play

And with that, I’ve created my unique “Inktober” look for RenderMan, which has a definite family resemblance to the original 2D art.   

For Inktober 2020, I’m going to try and complete this month’s challenge using VR Sculpting and these NPR Stylized Looks.

Medium sculpt and NPR image:

Tips & Tricks for Sculpting in VR

One word: Layers! Take advantage of using layers. When working on a complicated sculpt, layers can be used to organize the model in a way that keeps it manageable — and keep in mind that layers can be nested hierarchically, so a character can be posed by building in forward kinematics.  Beyond organization benefits, layers are useful also as an artistic tool: You can perform Boolean operations on layers — super useful. Layers can be duplicated, sculpted upon, and then recombined with the original layer, and reworked more. Simple example: By duplicating the model of a figure, scaling it up, deleting the head, arms, and legs, you’ve begun to make a shirt for the underlying figure. Layers offer a lot of creative possibilities.  

Personally, I like to nest the key light in my scene under the layer of my main sculpt; that way, however I reposition the sculpt, the light remains correctly positioned on the model.  

Future Projects

I’m making more functional art, including wall hooks and cocktail skewers. I also have an elaborate project I’m working on that I haven’t announced yet, but it’s going to have modular components and incorporate magnets and other doodads.   

Functional art hooks modeled in Medium and textured in Substance Painter:

Big Dragon with practical and fanciful Substance materials:

Looking down the road, it’s interesting to imagine how the new generation of VR creation tools will impact traditional games, animation, and VFX pipelines. There’s a lot of territory to explore and I’m excited to see what artists can do with Medium.  

All images courtesy of Dylan Sisson.
Follow Dylan on Instagram.

Thank you!