View Full Version : Linear compositing and alpha channels from 3D in AE = busted?
05 May 2010, 03:14 AM
I've been troubleshooting this problem all day and cannot for the life of me get any headway.
Basically I'm noticing that After Effects ( AE CS4 ) is not interpreting my alpha channels from 3DStudio Max properly when the project is Linearized. I'm exporting non-linear 32bit EXRs (but I have tried multiple formats w/ the same results ). The goal is to get it to look just like it does in the MAX frame buffer, in AE - in a 32bit, LINEAR project. It looks fine when the project is not linearized. You'll also notice that the 100% opaque portions of the image remain constant whether the project is linearized or not, the way the transparent portions look is the only part that changes. If the project is linear and the EXR is premultiplied ( and interpreted as such, interpret as linear is also off... this workflow works great save for alphas ) the transparent portion is EXTREMELY overexposed... this makes no sense to me!
Oddly enough Fusion 6 can load, and interpret this perfectly without any problems, so i'm convinced that AE CS4 is the problem.
Does anyone know whats happening and if there is any way around it? Any help would be tremendously appreciated! Thanks!
MINOR UPDATE: After playing around and trying some things, I've learned 2 things:
1) if I don't use exposure control when rendering out of max to tonemap, then the alpha comes reads/looks ok ( but of course the image needs to be tonemapped, and there aren't great tools for that in AE yet ), so thats not the best solution.
2) If I take the render that is premultiplied, and have it use its own alpha as a matte, then it looks like it should! I have seemingly, double apply the alpha! Even when I do this though, the bottom RGB values are still way hot even though they LOOK correct. What would cause this? Does this give any clue as to what I may be doing wrong to begin with?
click to enlarge a bit:
05 May 2010, 05:53 AM
Since you don't render linear Gamma in MAX, the issue may actually depend on your material and render settings. Since color modes are global per project, AE has no way of correcting one without affecting the other. It would also be a question, whether MAX produces a clean, unbiased float Alpha in the fist place. Many 3D programs only use 8bit or 16bit Alpha channels and those usually have Gamma bias since with 8bit/16bit, the RGB data would also be Gamma corrected. I really think it's more a matter of you combining certain things than AE messing up per se.... You could also provide the files for otehrs to have a look, maybe someone who uses MAX knows what to do.
05 May 2010, 05:13 PM
Download sample Max and AE files as .zip ( 9.6mb ) updated to include sample EXRs from MAX (http://www.colinjames.tv/temp/linear_alpha_problem_withEXRs.zip)
Thanks for the reply! The link above has some sample files as requested.
The reason the files are non-linear coming out of MAX is because I'm using the mr Photographic Exposure Control to bring the values into a more manageable range and Max handles the tone mapping. I've yet to find a quality method of tone-mapping (http://en.wikipedia.org/wiki/Tone_mapping) ( a tonemapper is like a smart normalizer, that compresses the image into a certain bit depth to be viewed. Like clamping the highlights and shadows i into a certain viewable state. ) completely linear, non-exposure controlled renders from AE. If I could do all of my lighting and everything w/ exposure control on, and then turn it off at render time, and then do the tone mapping in AE to match that would be great, I just don't know how do that so it looks like it did in Max w/ exposure control on.
I did a test and turned the exposure control off and the alpha seems to behave properly in a linear project, it's just that everything else looks "off" because there hasn't been any gamma / correction or tonemapping. So if I understand this, you're saying AE (unlike fusion apparently ) treats the alpha as linear in addition to the color space... is there anyway around this?
05 May 2010, 05:27 AM
Have you tried Stefan Minning's alternate Tonemapper plug-in? Maybe that solves your issue on the AE end. I can't really advise on the specifics of MAX, but yes, Alphas should be linear if the RGB data is linear. That would especially be critical with premultiplied, since obviously it would affect colors at the edges. I guess your problem relates to MAX probably producing using different ranges for Alpha than for RGB, resulting in uneven compensation and in turn the shifting of the transparency levels. Could you provide the rendered EXRs in addition to your other files?
05 May 2010, 09:00 AM
Funny thing i never saw this thread but basically answered in the next thread something that might help.
Blending with linear colours does mess up the alpha channels in AE it seems.
When colours are blending using 2,2 gamma, the alpha is blended okay, but when set to 1,0 gamma the alpha will get all ugly, like it changes from linear to 2,2.
Obviously the resulting alpha looks wrong so i assume something isn't right with this mode.
I found that you manually have to reduce the alpha levels on each layer to make them look right. A gamma adjustment of 0,454 does the trick, so i assume the alpha levels goes from linear to 2,2 when it shouldn't be affected by the linear option.
But it's possible to get exact the same result with 3ds max frame buffer if you have the right tools. Thing is that you should actually make any tone mapping or exposure controls after rendering in the composite software. So try and turn off any exposure controls in 3ds max, be sure that the output is 1,0 and render to OpenEXR or any high dynamic format that works. I think you have to be aware that you actually have to sRGB the final result (add 2,2 gamma to the linear material) to match the 3ds max frame buffer, since i think the buffer displays in sRGB in the end anyway.
But my guess would be that you have to reduce the alpha levels by 0,454 to get the correct result.
05 May 2010, 06:02 PM
Thanks for your reply, and confirming my suspicions about whats happening in AE. The problem with just adjusting the alpha levels by .454 to compensate is that it may LOOK ok, but AE maintains the funky exposure shifts it does to the values that are semi-transparent. Like in frame B from my image above... when clamping the alpha levels, it will now look like frame C, but those crazy over exposed RGB values remain, so if you do any sort of glows ( say a Diffusion filter from MBLooks ) the semi-transparent portions ( the reflection in my case) is going to glow way brighter than the solid object itself.
The only solution I've found is no longer to use any exposure control out of max, and keeping the render output EXR completely linear and like you say, doing any tonemapping/correction in AE.
So now my workflow is this: In max, I'll put a Utility Gamma & Gain shader on all of my color channels ( to linearize color maps used in the scene ) on my materials. I'll temporarily turn on mr Photographic Exposure control while I'm doing my lighting, and when I start getting something I like, I'll turn it off and render out to AE. In AE my project is 32bit Linear, so it detects the EXR as linear, and automatically puts it into the project working space (ie. it stays linear), and with AE's display color management on, I see something very similar to the way it was looking in max w/ exposure control on. Then I just have to adjust my exposure a little bit, some levels etc...maybe tweak lights in max, back and forth and so on. Then use AE's color management to export a final image in sRGB.
What I WAS doing, which doesn't work with AE, is using Max to handle the gamma correction and tonemapping via mr Photographic Exposure Control, but still exporting a 32bit EXR so I'd still have some overbrights to playwith. Then when AE brought it in, under footage interpretation > color management, I'd specify that Interpret As Linear Light was OFF. It was under these circumstances that fully opaque objects looked and composited exactly as expected, it was only until I started dealing w/ semi-transparent objects that I realized AE freaks out in this scenario.
The only TRUE fix I've found is to do all tone-mapping in AE, and export out of max completely linearly. Which pros like Stu Maschwitz have recommended anyway. I did it the other way until now because the tonemapping tools in AE weren't as robust as I'd like, I just need to be a little more careful in Max now instead of relying on mr Photographic Exposure control to reign in all of my different crazy light values.
Does any of this sound right to you guys or am I still doing it wrong?
05 May 2010, 10:28 AM
To get good control over the max framebuffer and the result in AE i usually only use Photographic Exposure Controls together with Mental Ray. I only use the Exposure Value (EV) to adjust the brightness of the scene. AE should have exposure controls that works just like that, which means an exposure value of 15 in Mental Ray matches the same brightness as an exposure value of 15 in AE.
How to match the gamma controls in MR's exposure controls are a bit more tricky though since AEs exposure effects doesn't contain individual controls for shadows, mids or highlights.
Perhaps if you have a custom colour corrector that works with float and have the necessary matching parameters you can get an exact matching look. Toxik have excellent tools for this though.
But as i mentioned earlier it's often necessary to apply an sRGB profile to the output (or apply 2,2 gamma) from AE to get the exact same match to the 3ds max framebuffer. But it depends on what your display LUT is set to in max preferences.
If the 3ds max LUT is set to 2,2 and the output to 1,0 from 3ds max, then you need to apply a 2,2 LUT in AE to get the same viewing result.
A 3dsmax LUT of 1,0 and output of 1,0 means that AE should have a 1,0 LUT as well ect.
This is tricky stuff, and AE doesn't help you much with this workflow unfortunately, but you can solve most things with some extra efforts. Most of them are problems you only need to solve once anyway.
05 May 2010, 04:58 PM
I never typically use the built-in MAX gamma prefs, I typically manually pre-correct my maps/color materials w/ the mrGamma&Gainer node, and then render to 2.2 using the mrPhotographic Exposure control. Those 2.2 EXRs still have overbright information. When I bring those into to my AE comp, I let AE know during interpretation of the footage file that this is ALREADY 2.2 and NOT linear. I use AE's built in Display Color Managment/LUT system, so basically it takes 2.2 files, and automatically converts them to linear space when it does all of its blending and calculations, and then the Display Color Managment/LUT displays the result back into 2.2/sRGB space.
It's worked for months, but the first time I've run into problems is now, as I've got semi-transparent pixels ( the reflection )... however, I am going to do some tests today to determine if its just the shader that I'm using in MAX to "catch" the reflections on the ground in my test scene ( mr Matte/Shadow/Reflection ) that is causing the funky values in AE. If that's the case, then I can continue to revert back to my old workflow except for whenever I use that shader, at which point I'll just have to render out of max linearly w/o exposure control.
EDIT: I did a test w/ just a transparent cube, and it has the same issues as the reflective floor, so it doesn't have anything to do w/ the mr Matte/Shadow/Reflection shader's output. My solution is to just bring in things linearly from now on if there is any sort of semi-transparency going on.
05 May 2010, 08:39 PM
If you want to work linear i see no reason to use 2,2 files in comp at all, other than applying a 2,2 gamma correction as a LUT for the final result only. Perhaps it works without fuzz but i think linear is the way all the way until it's displayed on the monitor, that's where LUT comes in.
Previously i think 3ds max never applied any exposure control to float material output, if you rendered to OpenEXR the output would totally ignore the exposure control, outputting only pure float material. In the recent versions of max i think it actually does include the exposure settings before output (perhaps it was a requested feature.) I'm not sure if it's the right way to go if you want to comp with 'realistic' material in AE though.
Think of the exposure controls and gamma correction as the post work, or grading. It's applied afterwards. The real world obviously uses linear light, so that's how your scene should work.
I'm not entirely sure if produceral maps actually should be linearized before input to the scene or not, but sRGB textures does need a 2,2 gamma correction to become linear before input to the scene. So i assume that any light that's being input to the scene should be linear if you are looking for realism. Perhaps you already know that stuff but it's good to know anyway ;)
With this is mind what you are doing, or should be able to do with the proper tools, is to take the same information a 3d program would calculate for you, and stuff it into a comp software and give you the exact same conditions to work with. Using 2,2 material, linearize it and then LUT it to 2,2 again might work okay, but i don't see any reason for it as long as the tools you are viewing in can use LUT.
06 June 2010, 09:39 PM
if you have this sucker set in 16bits your are not getting the 32bits file you are expecting.
06 June 2010, 08:06 AM
if you have this sucker set in 16bits your are not getting the 32bits file you are expecting.
AE will have the same alpha problem with low dynamic images as well. I haven't tested CS5 version to see if this issue has been adressed in some way.
06 June 2010, 08:06 AM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.