PDA

View Full Version : MAYA for Post / Compositing


VOX3D
06-24-2010, 08:36 PM
I'm doing a big animation and i'm having some problems in dividing my scene for post. I have some cameras animations but then i have 3D people and 3D cars moving.

So that i can avoid getting flickering animations i want to render them separately so i have my main scene where the cars and people are off and i batch render that.

After that i used a "Use Background" shader for the floor so that i can get the shadows for my cars/people and for all the rest i tried two things:

1 - Set primary visibility off - But this causes a problem when i have geometry in front of the cars/people that need to be cut off

2 - I assign everything else a surface shader - But with this i can't cut out on After effects for example because it cuts some of the other blacks i have on the cars/people, mostly the car tyres. And if i change the surface shader color to a green or blue color it changes my cars/people color to the one i've chosen.

For example if i have a green color on the surface shader everything will turn into a greenish color.

I've tried do some Keying in After effects but it's not as clean as i want. It always cuts something.


Isn't there a simple and efficient way to do this?


I even tried render layers but i keep stuck on the same problems.

And i even tried rendering out to Toxik but no luck again.


I just want to be able to have the folowing animation passes:
- Main (Everything without the cars/people)
- Cars and People (Diffuse and shadows)


Can someone please share their techniques and experience on render for post-production and compositing.

PS: For the sky and because i'm using Physical Sun and Sky with Alpha, when i pass it to After Effects/Toxik if i put there an image for the background i have to animate it or else when the camera

moves the image doesn't follow. Any tips for making this?

Tools: Maya, Mental Ray, After Effects, Toxik (Maya Composite)

Thanks,
VOX

JasonA
06-25-2010, 12:54 AM
If you want to keep it as basic as possible, I'd break it into four overall passes. 1) the background with no cars/people, 2) shadows only for the cars/people, and 3) the cars people. $) then make your RGB mattes for the objects you need

Ill summarize rather briefly so I hope you can troubleshoot along...

1. Unless you need the cars/people to show up in the reflections of objects in the environment, I'd just hide them alltogether and render out your background.
2. For the shadows, use the 'use background' shader and set the first three attributes on the shader to zero except for shadow mask. Assign it to the ground plane, and have your cars/people primary visibility set to off. render out this shadow pass.
3. Set your environments primary vis off, and render out the cars/people without matting (do that later).

I dont recommend using black surface shaders in your scene for matting right out of the render view because it will screw with the lighting solutions on your objects - best to render out seperate RGB mattes and use those in comp instead of trying to matte in your maya scene. However, if you really feel the need to do this, then make sure to set all the objects with the surface shader to "not visible in reflections" or else the colors of the surface shaders will show up in reflections on your hero objects. Also remember that if you use a surface shader for matting, set the alpha color to black so those items are stripped from the beauty and alpha.

Ok so in AE you can use Set Matte along with your RGB mattes to isolate colors on your mattes, and then use those for color correcting or occluding as needed.

By no means is this the only way to do it but I hope its simple enough to start with.

Jason

VOX3D
06-26-2010, 04:31 AM
Thanks for the answer.

I use to most of the things you've mentioned. One thing i don't use to do and that i think might help me out is to make RGB Mattes for the Objects. That would be the best way to select them in comp definitly.

Can you explain me the best way to do it pls?

And what can i do when i have objects to be cut off. Objects like buildings or other stuff that appears in the front of the cars during the animation?

The thing is that render everything togethers causes some flicker cause it's the camera movement + objects moving and Final Gather go nuts...It's one thing i hate in Mental Ray...Always a challenge (if not almost impossible to get to everything "flicker free"). Any tips about that?


Concerning the Sky what can i do? I mean, i'm using Physical Sun & Sky and when i batch my animation and open it on AE or Toxik it comes with alpha cut off and i want to put some clouds there...The problem is that when the camera moves i have to animate the image plate and it gets very difficult to track the plate precisely along the animation. I used to put a plane or a sphere in Maya and assign my image there but that gives me no room to control the image in comp.

By the way, for this kind of comp what do you think would be the best software to use? AE, Toxik?another compositing software?
All comments are welcome since i need to buy comp software for my company.


VOX

UniqueCharacter
06-26-2010, 09:22 AM
I'm not sure if you have display layers of your different parts, but that will also help to hide certain parts from rendering. I don't have much more to add because Jason pretty much covered it. Definitely use the shadow pass like he posted to solve one of your issues.

In after effects you can also use the alpha's from other renders. You also have that option to cut out things you might not want.

VOX3D
06-26-2010, 01:45 PM
Thanks for the answers.

Jason - I use to most of the things you've mentioned. One thing i don't use to do and that i think might help me out is to make RGB Mattes for the Objects. That would be the best way to select them in comp definitly.

Can you explain me the best way to do it pls?

And what can i do when i have objects to be cut off. Objects like buildings or other stuff that appears in the front of the cars during the animation?

The thing is that render everything togethers causes some flicker cause it's the camera movement + objects moving and Final Gather go nuts...It's one thing i hate in Mental Ray...Always a challenge (if not almost impossible to get to everything "flicker free"). Any tips about that?


Concerning the Sky what can i do? I mean, i'm using Physical Sun & Sky and when i batch my animation and open it on AE or Toxik it comes with alpha cut off and i want to put some clouds there...The problem is that when the camera moves i have to animate the image plate and it gets very difficult to track the plate precisely along the animation. I used to put a plane or a sphere in Maya and assign my image there but that gives me no room to control the image in comp.

By the way, for this kind of comp what do you think would be the best software to use? AE, Toxik?another compositing software?
All comments are welcome since i need to buy comp software for my company.


VOX

InfernalDarkness
06-26-2010, 08:25 PM
The problem is that when the camera moves i have to animate the image plate and it gets very difficult to track the plate precisely along the animation. I used to put a plane or a sphere in Maya and assign my image there but that gives me no room to control the image in comp.

Simply set the sky material/object itself to an alpha channel. Matte opacity = .25 or whatever you need it to be! You can also exclude it from FG/GI/whatever, as necessary.

VOX3D
06-26-2010, 10:03 PM
But i'm using Mr Physical Sun & Sky.
When i open my animation in AE i can cut out the Alpha but the problem is after that. Whenever i connect an image or a video it will be out of sync if my animation camera is moving.

Infernal, maybe i didn't understant what you meant.

thanks

RickToxik
06-27-2010, 01:35 AM
Hey Vox3D, this is a very interesting thread, I'm glad I can join in! My first language isn't english, and I'm not sure to understand everything you're dealing with, but here might be some tips that could help you on your way, even though I am not a top Maya sorcerer...

1- Try to find out how to use the "black hole" mode of the shaders, that might be the solution you're looking for when objects pass in front of the others. If you "layer render" the wanted elements with the others in the black hole mode (it renders black alpha on the chosen objects), your comp job might be easier, and chances are that you will be able to re-use some "alpha layers" for other layers of the same sequence..

2- I'm not sure what your camera (matching) issues are, but when I have a very complex scene that I must send in post, I have been working very often (on PC Windows) with 2 Mayas opened at the same time.. So if your scale is consistent from one part of the scene (i.e. one opened Maya window) to the other, you can easily copy/paste your camera (or other objects, then rename the hierarchy) containing an animation path to another Maya window: therefore, I don't see the need to sync with image planes... Everything matches the same camera, if your scales are okay, it's not a perfect method but I find it usefull in some cases. Find a way though to never move it by accident, because results will be disastrous haha!!! But I'm not sure I understood this issue very well in your post, so, sorry if I'm off-track.

3- Grrr... Flickering... I think you render your sequence with mental ray, I have red a LOT on this issue before rendering my first sequence with mr, because I knew I would get some flickering on the road, and I was unfortunatly right... From what I understand, you should:
-make sure you select the "jitter" option in the rendering panel,
-and somewhere else in that panel select the "optimize for animation" option when you have moving objects.
But sadly, I'm starting to believe that most of the flickering comes from too low values in mental ray rendering options, like fg and gi... Meaning that.... animations must have a very high setting for the fg rays, photons, and thingies for them to never be "missing" (like lack of rays/photons) in every part of the image, most of all when they are not fully lit or change in light consistency. For what I know, high quality mental ray animation requires weeks to render without renderfarms :((
- One thing that might help to improve the quality of the animation would be slower movements of the camera... I'm not sure about this, but I THINK that if the camera moves slower, the fg rays are calculated more evenly from one frame to another, so that could allow you to throw in reasonable numbers in fg and gi quality (and rendering time). BTW, Motion blur in Maya is very long, but is another component of the quality of the sequence....

If you are running Maya 2010 like me (don't use those settings with previous Maya, for they have changed the Mental Ray algorithms or so), I would not render an animation with lower settings than:
fg 150-200 gi 250-500
But hey! I too am still in the learning process of Maya, so don't write a Bible out of my words ; )

Ok man! Let's keep this thread alive! Toxik is very hard to learn but is very powerful. I still use my old Combustion 3 though for many things like masking and selections....

InfernalDarkness
06-27-2010, 04:31 AM
What I was getting at is that I understand your matching issues in post, but if in your scene you created a hemisphere or spherical sky object and then set its shader to a matte opacity different from the other elements in your scene, you'll be able to adjust this sky object separately in post using layers/channels/whatever.

So for example, if all the main elements of your scene have a matte opacity of 1 and the sky has a matte opacity of 0, then you can easily deal with this in comp because the negative of your alpha channel WILL be the sky itself. And meanwhile, it should react fine to animation in your scene because the sky itself will be in your scene.

I'm likely not making any sense at this point! I guess, to sum up: you can have your sky in the scene too. Just give it a different matte opacity at the shader level and then you can tweak the sky in post just like any other element.

JasonA
06-28-2010, 06:19 PM
... One thing i don't use to do and that i think might help me out is to make RGB Mattes for the Objects. That would be the best way to select them in comp definitly.

Can you explain me the best way to do it pls?

I get my beauty scene and save out a copy as a matte file. Then I create 4 surface shaders, and color them as red, green and blue respectively. The fourth one I set to black, and also zero out the opacity. Then I put the black surface shader on everything in my scenen and for each renderlayer that I need, I assign the RGB surface shaders on items as needed. In comp (AE in your case) you can use these as mattes by use the Set Matte function and isolate the color there, then use it as a matte for whatever you need/

And what can i do when i have objects to be cut off. Objects like buildings or other stuff that appears in the front of the cars during the animation?
if you made RGB mattes for those objects, then you can use them to stencil out of your other passes as needed.

The thing is that render everything togethers causes some flicker cause it's the camera movement + objects moving and Final Gather go nuts...It's one thing i hate in Mental Ray...Always a challenge (if not almost impossible to get to everything "flicker free"). Any tips about that?
Ah mental ray gotta love it. Well a brute force solution is to increase you point density to something higher than 1 and give it like 400-600fg rays. This will hurt you in render times, but usually eliminate flicker fairly well. but really you need to diagnose your scene to get the most optimal settings.

Also, make sure your flickering is really a FG problem though cause relfections can also have flicker issues. Its hard to know for sure unless your rendering the AOV' passes.


Concerning the Sky what can i do? I mean, i'm using Physical Sun & Sky and when i batch my animation and open it on AE or Toxik it comes with alpha cut off and i want to put some clouds there...The problem is that when the camera moves i have to animate the image plate and it gets very difficult to track the plate precisely along the animation. I used to put a plane or a sphere in Maya and assign my image there but that gives me no room to control the image in comp.
Get a spherical image of a nice sky that you like and assign it to a big sphere with a surface shader. Hide everyhting else in your scene and bring in your animated camera and render out the ky as a scondary background plate. You should be able to comp this behind your cg environment and have something that tracks with your camera movements nicely.

By the way, for this kind of comp what do you think would be the best software to use? AE, Toxik?another compositing software?

Id recommend any that are node based, and not stack based. We're using Nuke here and I love it.

CGTalk Moderation
06-28-2010, 06:19 PM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.