Maya Render Layers (Shadow Separation)


#1

Can anyone help me figure this out? I have a character on one render layer and a hat he’s wearing on another. I want to have the shadow information that the hat is casting onto the character to be in the hat’s render layer and not the characters. The character has 6 different versions, each a different color. I want to be able to lay the hat on top of any color version of the character and it looks accurate.

At first I was multiplying (in Nuke) the matte shadow pass from the hat render layer with a hold out of the prop on the character render layer, but I feel I am doing this incorrectly. I select the prop (hat) and create a Vray Object Properties with the following settings set with a Layer Override:
-Matte Surface enabled
-Alpha Contribution set to ‘-1’
-Shadows enabled
-Affect Alpha enabled
-Shadow Brightness, Reflection Amount, Refraction Amount & GI Amount set to ‘0’

This creates the hold out I then use to multiply against the matte shadow pass of the prop.
The matte shadow pass is achieved by placing the character into a V-ray Object Properties group and enabling these settings:
-Matte Surface enabled
-Alpha Contribution set to ‘-1’
-Shadows enabled
-Affect Alpha enabled

Is this correct or is this a cheated version of what I really am searching for?

How do I get the shadow information that the hat casts onto the character to be within the hat render layer?

I am using Maya 2020, Nuke 13.1 & V-ray 5.

Thank you!