View Full Version : Object-Based AO

12 December 2010, 10:46 PM
In an idea that I had the other day, I thought of the possibility of creating ambient occlusion based on objects. In my mind, I shot down the idea because although you can apply an AO shader to one object and set all other objects to white, the shader won't catch the shadows that interact between two objects.

Still, I thought to at least try it. I've almost run out of ideas so I wanted to ask the forums, how could one create ambient occlusion for only one object AND the interaction of that object with other objects?

To give an example, you have a ground plane, a sphere, a box, and a torus. All 4 objects intersect but I only want to get the AO data on the cube. The cube however shares borders with (and thus AO data with) the sphere, the torus, and the ground. Without applying ambient occlusion to the rest of the objects of the scene (effectively keeping them pure white) is there a way to isolate the AO on the cube and the cube's interaction with the objects nearest it?

12 December 2010, 12:38 AM
DJX has a helpful blog that explains this pretty well:

DJX AO Labels (

12 December 2010, 01:53 AM
Marvelous! Thank you very much for the timely response, I always wondered what those attributes meant at the bottom of the scene (I guess I had forgotten them in my brainstorming) and great work by djx!

12 December 2010, 06:47 PM
Ah, one question though. It looks like setting the nonself to whatever label I want (so that it only takes that object) but I have a problem, Id_nonself doesn't work for me.

I downloaded djx's example file and it works in his but not when I create a file from scratch. Id_inclexcl works perfectly but nonself, no matter what I set it to, has no effect.

Walking through my process -

I create two spheres and a plane. I link up my amb_occ to a surface shader and apply it to all of them. I render as normal and, since I never adjusted the amb_occ's values, I get a low sampling (16) image.

I create miLabels for the two spheres and the plane. One sphere and the plane get label 0 and the second sphere gets 5.

I go back to the amb_occ and bump it up to 256. I then write in 5 for the Id_nonself.

As far as I can see, that mimics djx's example file perfectly. Again, I can change Id_inclexcl with no problems (and the desired effect is achieved) but my nonself won't work. Do you or anyone else have an idea of what's going wrong?

12 December 2010, 04:52 PM
No one on a solution for this? The example scene itself isn't on the main website that you linked to, david actually links to it in the comments section. Can someone please look at that file and my step by step process and tell me what's missing?

12 December 2010, 04:59 PM
Well, i dont know if its nice to say, but Vray for maya has this option when you set AO. It works very easily and its all built in..

to be honest, i dont know how i would do it in mental ray except for rendering out alphas for the objects and masking them in comp.

12 December 2010, 05:21 PM
Yeah, I was made aware of that feature but I'm a MR man so that option is out xD. I could do what you had said but that defeats the purpose of the topic, which is basically made to address efficiency with reliable results. Thank you for posting though ^_^.

12 December 2010, 07:48 PM
Heh, well if it means anything, I've been working with Mental Ray every since i stated working in this industry, even before then when i was in school. I never liked VRay, it was too clean and ive always felt that it was all too 'automated' for me. I liked how i could write my own shaders and plug nodes in to different places to achieve all types of things. But when i tried VRay, it really isnt automated, for me its like mental ray enhanced.

Anyways, i've figured a solution using MentalRay but im not sure if this is what you were trying to achieve since it really took only about a minute. Basically AO is scattered everywhere but you only have information of a selected object. is that correct? ive attached the scene through Maya 2011. let me know if this was what you were trying to achieve. you've also got holes in it for the alpha channel.

Attach txt_ss_AO to the objects where you want to see AO.
Attach txt_ss_noAlpha to the objects where you want it blank.

12 December 2010, 08:55 PM
I can't see the file probably because you have Maya 2011 and I don't xD. Could you please send the file as a .ma?

By the way you describe it, that appears to be the effect that I'm going for. Honestly the process is a 1 minute thing. DJX (from the link above) got my question answered already spot on but I couldn't seem to recreate it myself. The idea is to be as efficient with your AO as possible, confining the Ambient occlusion to be evaluated only on objects that deform AND their interaction with other objects. If you file has a standard AO but then is automated to mask out the AO pre-render then that's not what I'm talking about. It still has to calculate the entire scene's AO before it uses the alpha to rip it all out. I might as well render a whole AO pass and an alpha and comp it if that were the case.

12 December 2010, 09:58 PM
Ok, ive attached the .ma file.

as for your concern, it doesnt calculate the AO pass for the objects you dont wish to have AO'd. for example:

Render with 1 object as AO = 5 seconds
Render with whole scene as AO = 17 seconds

and yes the object still retains the same information as it would have when the whole scene is AO'd.

PS: when you open the file in maya, you should check the box "ignore version".

01 January 2011, 10:11 PM
Man, I'm sorry for not responding to this forum! In the end, it was just a simple problem of not creating the miLabel attribute right.

And that's great news on the part of the render time but, when I asked that question, I was referring to positive integer values in id_inclexcl. That's great on the note of single objects but what if you wanted to isolate every object's interaction? Is it still going to decrease render time? I ask now after a long time because I went through some tests and did see a slight difference but I don't have anything other than a test - style scene to use it in. I'd like to see how this works in a practical animation.

If you had a room or a city for example, there are a lot of static objects that are sitting around. Since they don't move, you could theoretically bake the object's AO to a texture or vertex color. But, to my knowledge, baking in either method only includes self AO, not interaction between objects. So instead you could bake an object's self occlusion and then multiply the interaction AO in post. Handling the AO in separate passes would decrease render time, assuming that it interaction was in face faster to render even on multiple objects. Can anyone confirm this?

01 January 2011, 07:00 AM
As far as I recall, you should be able to bake the AO from other objects at the same time no worries.

01 January 2011, 02:37 AM
Alright, thanks for the update Kev3D. I haven't done it in a while because the bake failed for me but perhaps I can get it to work again. I'll test once it becomes relevent (and if I remember to I'll post here :D )

CGTalk Moderation
01 January 2011, 02:37 AM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.