View Full Version : Basic composting help!!!

01 January 2013, 09:47 AM
Greetings CG society,

I come to you seeking information with regards to some basic composting questions. Really the main issue here is the use of ID passes in nuke. I recently started working in a new studio with a lot of technical and artistic experience with regards to programs such as Maya, Zbrush, etc. However, when it comes to comping all of us know only a little about setting up scenes. At the moment we are working on a project and have set up our renders with the passes necessary to composite a very professional looking piece, however bringing it all together is becoming an issue. We've been searching for answers all over the web through tutorials from DT and Gnomon videos, to forum posts on most major cg websites and we cannot seem to find a solid explanation of how to maximize the use of a custom ID pass from maya to make color corrections and composting masks in Nuke. for example we have a large crowd simulation in our video that we are coming into a 3D basketball stadium, things like reflection, beauty, ao, and shadow passes are really easy to set up, but when it comes time to start masking out sections and adding the crowd to the final scene we hit a wall. At the moment we've been attempting to use a form of keying and alpha masks, which does work, but only for one selection. I'm sure there is a very simple way to do this, but with the severe lack of sleep and time constraints I come here for help. If anyone can explain in detail how to set up and use the proper nodes for a id mask or a link with a good tutorial, that would be greatly appreciated.


01 January 2013, 12:32 AM
Not exactly sure what you mean by 'custom ID pass'; is it an actual matte, or something like a Z depth pass for the crowd? And the crowd is on it's own render layer?

Basically, you just connect the matte/image you want to use to the factor socket of a node (eg, blur). If it's a floating point depth pass then you can use the grade node to change the falloff of the image, perhaps to add in atmospheric effects, or to blur the BG more than FG.

It's a bit hard to answer your question without knowing exactly what you're trying do. Using mattes in Nuke is pretty much shuffling them out from an EXR (or reading them in as an image sequence, roto, keying, etc), then doing whatever you want with them....

01 January 2013, 01:24 AM
Sorry I was on the tail end of an all nighter when I made the post. What I meant was "using a color id pass out of maya", basically a surface shader of a bright color that represents and hold the objects shape.


We actually managed to figure it out using miLabel gizmo. We can use that to make an ID selection of the color from the pass and turn that into an aplha channel for a mask. Giving us the selection needed for whatever comping/correcting tools we need for it in Nuke.

01 January 2013, 09:24 AM
ID passes, like Object ID or Material ID, will always have some issue with edges. because the dont have antialiasing in it.
but on nukepedia you can find some tools which will help you selecting mattes from ID passes

depending on which renderer you use you can more or less easily set up RGB matte passes with correct AA, MB and DoF.

01 January 2013, 05:21 PM
Ok, New problem!

So we were working on the project and we managed to get the whole scene minus our action sequence comped together. The problem we are running into now is that with the action layer our masks come out to be about 3 pixels to large in certain places and right on the money in other places. We have messed with all the settings in nuke to shrink and blur the mask, including using a combination of nodes to achieve results. This unfortunately didn't yield a good result. We figured that it is probably a problem with the passes out of maya. We are using 2dmv, AO, beauty, front surface SSS, back surface SSS, diffuse SSS for two different characters, reflection, refraction, shadow, and a custom ID pass (the id pass also has 2dmv so frames line up) for control in post. We have tried numerous different things to attempt to fix this problem which include adding the character's displacement to the ID pass, changing its render format on the id pass to 16bit (down from openEXR 32bit), and checking both on and off the pre-multiply box in maya. We are a small team and we are really trying to avoid having to roto the project. Do you guys have any ideas?

If it helps we are using Maya 2013, NukeX 6.3v1, and After Effects cs6. Our settings in Maya include Final Gather and Unified Sampling.


01 January 2013, 10:03 PM
Have you tried premult/unpremultiplying the matte by other passes in Nuke itself? I've come across this many times, and I once had to premultiply the matte by the diffuse color pass to get it to work.

01 January 2013, 04:28 PM
Yea we messed with both the premult and mult nodes, maybe we weren't using them right but we didn't get the result we needed :/. We finally managed to notice a problem with the rendering out of mental ray. Apparently when you have large scene files and they get disorganized on OpenPipeline Maya decides to render things out weird. Hopefully when we make the switch to vRay after this project things will get easier. Thanks for all the help though!

CGTalk Moderation
01 January 2013, 04:28 PM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.