View Full Version : Render passes output black for transparency

12 December 2009, 04:28 PM

Have been desperately trying to fix this for 2 days now. Maybe I am just doing something stupid but when I render passes in Maya 2009 anything with transparency(eyes, hair cards eyelashes) just comes out black in diffuse and as a white plane in AO. Only works right on master beauty pass. Attached images below to illustrate what I mean.

This can't be right can it? If I'm rendering a diffuse pass I still expect it to take account of transparency on objects. Is there a setting to make this work or a clever way of doing it with frame buffers?

Help please, thanks :)



Agent X
12 December 2009, 05:32 AM
Hi Finbo,

I know what you are talking about and it is correct. It's stupid, but it is correct.

From looking at your scene, i gather that you have two spheres for the eye, one below with the texture and one above with reflection/refraction. If this is not true, then please disregrard the following.

OK, I will go through the theory first, then possible solutions. This going to get a little complicated, but here goes:

The mental ray shaders are physically correct, which means that when light hits and object, it can do one of three things (it could be more, but i am simplifying for the explanation):

It can be spread across the surface (diffuse)
It can be bounce off it (reflection)
It pass through it and bend(refraction)

Now the colour of object can only 1. so if diffuse is 1, the refraction and reflection have to equal 0.

if reflection equals .5, and refraction = .25, then diffuse equals =.25. and so on and so forth. i could go through every possibility here, but it would take a while

So, enough of the theory, and more of the answering your question.

You are looking at the diffuse pass of your render, but the top eyeball sphere doesn't have any diffuse. It has refraction and reflection. No diffuse on the 'diffuse' render means that is black.

I know that you are thinking "That's fine, Jerk Hole, but what about my textured eyeball underneath it, why doesn't that show up?". The answer that is that the textured eyeball is always covered by the refractive part, so Mental Ray never sees it, it only sees it refracting through the top sphere. And don't call me Jerk hole.

You can extrapolate that to the eyelashes. The transparency makes the diffuse black, and the texture is so small, you can't actually see it.

So to fix your problem for the eye, grab a refraction pass as well. composite it as using 'screen' over the top and you are done.

You could also try adjusting you framebuffer pass to output RGBA, i am not sure if that works, never tested it in this situation but i think that it should work as the image isn't really rendering black, it is rendering nothing, but nothing is very hard to display without an alpha channel.

I hope this helps, and please report back if it does so other people can learn from this. If you want to know anything else, don't hesitate to ask, i promise my subsequent answer will be less wordy. I am just bored at work.

Agent X

(ps For those who care, i believe that MR's physically correct way of dealling with colour is why the framebuffers you have to output for it are different from the frame buffers you out put for regular maya shaders. Keep in mind that i don't work for Mental Images, i am not a scientist, and i am barely sober so that is purely opinion not fact.)

12 December 2009, 08:33 AM
What shader you are using for passes?

12 December 2009, 12:36 PM
Hi Agent X,

Thanks for the post was very very helpful! All quite obvious once you have explained it. Wordy is fine with me :)

The problem I am having now is whilst this is suitable for the eyes, it doesn't work so well for the hair for a number of reasons. In comp it is very difficult to get rid of mattle lines where the refraction pass meets the others. The refraction is not always identical to the passes you are comping it on to(especially if you are tweaking them) if you know what I mean?

Also the AO is just white planes, doesn't seem to include transparency at all. So is there a way for the alpha I'm using to actually make the surface be entirely invisible(cut out opacity crashes Maya when rendering). If not is there a way for it to work for AO or do you know a suitable work around?

Also the spec is much much brighter when taken out as a pass than it appears as part of the SSS. I'm not hugely bothered by this as I can adjust it in comp but is there a reason for this?

Thanks again for your help is hugely appreciated!


I'm using mia_x_passes shaders and raytrace renderer :)

Agent X
12 December 2009, 03:34 AM
Hey Finbo,
I am glad that was coherant enough to be understood. Before you asked, i had never actually thought about why refractive/reflective object didn't show up, i just knew they didn't. When i start writting a response, i all kinda fell into place.

Before i start answer your new questions. 2 quick points:

I really hope i don't come off as condesending or patronizing. I realise that by explaining everything in such basic terms it can appear that i am treating you like you don't know anything. The truth is, i have no idea of what you know or who else would read it, so i try to be as thorough as possible.
Mental Ray is very powerful, but it is like your IT guy, it knows all the answers, but it makes it as hard for as possible to get them. I don't mind this, as you can tell from my posts, i like to feel smart, and know the mathmatics behind everything and i work for myself to deadlines aren't that much of an issue for me. So if you aren't in the mood to spend ages slamming your head against the wall, other renders do pass systems WAY easier, as i am sure that Jupiter Jazz and 3iege will tell you all about.
now onto your new queries:

You have asked quite a bit and i will try and be as thorough as possible. The way i fix most of the issues you are having are by tweak in the compositing software. There are a few quick things you can do in maya.

Everybody please correct me if i am wrong:

In the render settings, under 'quality => framebuffer' turn off 'premultiply'. this makes your main pass look crappy, but if you "post-multiply by alpha" when you import it into your compositing software, it should get rid of you matte lines.(It does for me) Try it out and let me know.

In regards to your 'hair issues', i would suggest learning how to use pass contribution maps(here is a thread that has a link about them ( This is also the answer to your AO question. If you render out a pass that is just the eyelash diffuse, making sure that you in the 'render pass options' for that pass you change 'number of channels' from '3' to '4'. I refered to this in the earlier post about making sure you get the alpha in the pass.

So if you have the eyelash diffuse (with an alpha channel) and use that as a luma matte for the ao pas, you then have the ao for the eye lash and you are done.

It sounds more complicated than it is. once you do it a few times you will get the hang of it.

The refraction SHOULD always be correct so i can't help you with this one and i don't really know what you mean by "The refraction is not always identical to the passes you are comping it on to(especially if you are tweaking them)" so i can't help. It could be a gamma thing. I am sure you have read a few hundred threads about that.

I am not sure about the specular. It could be many things, but check the gamma again. You would be suprised how many times it is something as simple as that.

What are you using to composite?

If you want to send me the file to mess around with i am happy to. I don't know how protective you are of your work, but you could post it up here, and i can tweak it , and post the fixed version, so people could have a file that they can use for reference for these things. If not, just send me a PM with your email i will get in contact with you.

What i found really helpful for this was Tim Jone's "Multipass Compositing" Gnomon tutorial (

It isn't in Maya or even mental ray, but the way he explains everything is great and also the way he talks about using 2 passes to get 4 passes worth of information in compositing. I found it really helpful in getting my head around it. It also helps that it isn't in Maya because you have to find out how to make it work in maya, which increases your knowledge (the whole "give a man a fish" scenario).

Anyway, let us know how you go, and have any more issues, some images would help in narrowing down you issues more.

I am not going to read over this. Any spelling or grammatical errors i apologise for!

12 December 2009, 01:26 PM
Hi Agent X thanks again for taking the time to reply. Don't worry about explaining everything in detail. I have much more experience comping(Digital Fusion) than rendering with Maya. The program we use at work is Houdini so I am still trying to adjust to Mayas very special needs :P

As for using different renders/plug ins for passes I am stuck using bog standard Maya 2009 as it will be rendered on a farm I have no control over. I really wouldn't mind getting into the nuts and bots of how things work as I am keen to understand how I would wrench the passes from Mental Rays clingy fingers if I could find a decent resource. I've done a bit of this, understand how to use frame buffers to get different results out of mia passes shaders.

I'll have a look at that post but are you suggesting I render out the eyelashes seperately from the rest of the head and have the diffuse cut out the correct shape? Otherwise if I do this method with the rest of the head visible will I not just get a cut out and not what is actually behind the eyelash?

As far as refraction goes - if I'm looking through the refraction at the head behind it, the image in the refraction is going to be the master beauty pass. If I have taken my 3 layers of SSS and comped them in fusion in any way different than this then it will appear different from the skin that I will see in my refraction if you know what I mean? I just really wanted to know if I can actually cut out the alpha so that it is invisible and not refracting/specular/diffuse without using cut out alpha which just crashes my renders in 2009.

I think the SSS shader must do something to the spec that the basic spec pass doesn't take into account. Gamma seems to be the same.

Not too fussed about my file will share it later if needs be!

Forgive me but I'm rushing atm need to leave in a second will come back to all of this with more time.

Thanks again for the reply will be back later with more questions no doubt.

Agent X
12 December 2009, 11:45 PM
Excellent. I don't have an issue with Cutout opacity in 2009, rendering it on a core 2 dou with 3 gig of RAM, so maybe it is something to do with your system

Pass contribution maps are what you want. It is very much like render layers, but they render at the same time as the master beauty. So if you put the eyelashes onto a pass map, that map would only contain the eyelash. Its great because you can really isolate what you want, but you end up making heaps of passes (eyelash diffuse, eyelash reflection, and so on and so forth).

I think i get what you are saying about the eye now. You are saying that if you, for example, changed the skin colour to bright blue, the reflection in the eye would still be human skin coloured. Yep that is true, but you have to think about how much people would notice it. In the example i used it would a little noticable, but if you were doing a slight hue or sat change people probably wouldn't notice. Test it out on someone who hasn't seen the shot.

i have never tried SSS using pass i have no idea about any of the pit falls.

I really don't think that i can help you more with out seeing the file, as your problems seem to be very specific. I am doing some arch vis todoy, but if i get the chance i will smack out a quick example with all the things i am talking about.

Maybe with a fusion file so you can chuckle at my pitiful attempts at comping!

CGTalk Moderation
12 December 2009, 11:45 PM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.