I had this entry as a tutorial on my old blog website. I have since been restructuring my website and have not had a chance to rebuild the blog section. I have been getting emails from people asking me to repost the entry. So for now I will put some of them here. Hopefully you’ll find them useful.
This post is a little bit different from the normal drivel I provide world wide. As I have mentioned before and in the About section I earn my keep as a visual effects artist for film. One of the more interesting aspects of the job is the problem solving that goes into creating desired effects. Sometimes the search for answers to problems I encounter become a project in itself and even good resources can end up using cryptic wording that makes its hard to understand a procedures implementation in production. The long and short of this post will show you how to build occlusion passes using Mental Ray Final Gather that obey transparency maps. I have attached below a downloadable Maya 2008 .ma file in case my walk through does not register, or if you are like me and find it much easier to dissect the scene rather than read someone elses lecture. For the purposes of this post I will be referring everything based on Maya 2008.
The current mib_amb_occlusion node located in Mayas hypershade panel under the Create mental ray Nodes menu, is a great source for delivering an ambient occlusion layer or pass, but to my knowledge does not recognize transparency maps assigned to objects within a Maya scene.
[u][img]http://www.joshmossotti.com/TransparentOcclusions/images/mib_amb_occ_location.jpg[/img][/u]
[img]http://www.joshmossotti.com/TransparentOcclusions/images/normal_AO.jpg[/img]
I am going to move rather quickly past that node. If you are here you likely searched for something related to transparencies in ambient occlusions which should leave me to believe you are already comfortable using normal Mental Ray ambient occlusion shaders. If you want more info on that contact me through this post.
In order to run a transparency map into an AO (Im abbreviating now) shader, we need to use Mental Rays Final Gather renderer and a couple of Final Gather specific nodes. In the hypershade panel create a Maya lambert material, a Maya surface shader material and open the both of them in the bottom work area by shift selecting both of them and clicking the showUpAndDownstream connections button above.
[img]http://www.joshmossotti.com/TransparentOcclusions/images/showUpAndDownstream.jpg[/img]
We are now going to grab the necessary final gather nodes from the Create mental ray Nodes menu. The first one is the mib_fg_occlusion node located in the sub-menu MentalRay Lights. The second node is the mib_transparency node located in the Sample Compositing sub-menu.
[img]http://www.joshmossotti.com/TransparentOcclusions/images/mib_node_locations.jpg[/img]
I am going to generate a transparency map using a Maya based procedural Ramp texture to complete this walk through. You will substitute this for the map you need based on your production. Lets start by connection the mib_fg_occlusion.outValue to the surface shader.outColor by middle mouse click and drag from the mib_fg_occlusion node to the surface shader node. This will bring up the connection editor, connect the forementioned attributes to be connected. Connect the surface shader.outColor to the mib_transparency.input. Connect the transperancyMap.outColor to the mib_transparency.transp. This will automatically connect the transperancyMap.outAlpha to the mib_transparency.transpA. Lastly, connect the mib_transparency.outValue to the LambertSG.miMaterialShader. Be sure when you are connecting that last attribute to be connecting to the [b]shader group[/b] of the lambert material. You can now connect the shader to the desired object.
[img]http://www.joshmossotti.com/TransparentOcclusions/images/fg_AO_connections.jpg[/img]
Before all the magic unfolds before your eyes you will need to bring up the attribute editor for the transAO_LambertSG shading group. By default there is an attribute Export with Shading Engine which is flagged On. Uncheck this box for the transparency to work. It will not function properly if this attribute is not turned off.
[img]http://www.joshmossotti.com/TransparentOcclusions/images/exportWShadingEngine.jpg[/img]
We are almost there and your patience will be rewarded I assure you. We need to turn on Final Gather in the Mental Ray render globals. By turning on Final Gather and ensuring that raytracing is turned on this shader now will recognize the transparency maps in and ambient occlusion shader. I would also suggest making sure that your quality presets are set high and adjust the settings in the Final Gather menu to achieve the desired results.
I will close this post by saying that this will likely serve the needs you may have for generating AO passes containing transparency maps. I do prefer the control settings in the mib_amb_occlusion node to the mib_fg_occlusion but this does the job for now. I hope this helps and is an easy approach to building this shader. You can download the Maya 2008 scene used in this walk through below.
[img]http://www.joshmossotti.com/TransparentOcclusions/images/fg_AO_example_01.jpg[/img]
[b]Maya Scene Download[/b]
[[b]JM Transparent Occlusion Tutorial[/b]](http://www.joshmossotti.com/TransparentOcclusions/jm_AO_tutorial.zip)