PDA

View Full Version : Framebuffer Gamma correction VS mia_exposure correction


royter
05-18-2009, 03:26 AM
First i have a simple question regarding the current results in the 2 methods: why is the method (B) with the gamma correction in the frame buffer brighter thean method (A) with the gamma correction in the mia_exposure_simple:
http://sor.typepad.com/.a/6a00d8350600cb53ef01156f9a2eb4970c-pi

(check the PSD (http://sor.typepad.com/files/exp2.psd) to see the difference)
shoudn't they be the same?


Second i just want to know if these are ALL the pros and cons of the two methods while rendering in 8 bit:

FrameBuffer Gamma 1
mia_exposure Gamma 2.2
Gamma correct nodes for file textures (0.45):

-inserting a bunch of gamma correct nodes
-inacurate color bleed in pixel sampling
+accurate results in gradiants



FrameBuffer Gamma 0.45
mia_exposure Gamma 1
No Gamma correct nodes for file :

+no gamma correct node required
+accurate color bleed in pixel sampling
-inacurate results in gradiants


are there anymajor pro/cons missing?

H2
05-18-2009, 04:07 PM
Not sure if this is the reason but do you have the compression and knee set to 0 in your mia_exposure_simple?

spiralof5
05-18-2009, 05:13 PM
maybe I'm wrong but I thought the framebuffer gamma corrects everything, which means all your black and white maps are now wrong. Seems easier to go with your color nodes that you have less of than your black and white maps.

OR

Just do each texture correctly from the beginning. You can gamma correct your textures straight in photoshop before they ever hit your 3D application. This is the best way to do things (especially in a work environment with a few people working on one project) because you don't need to explain to other people how to do this. Just set up two folders in your source folder: One that says gamma corrected; the other that says new. Then everytime a person creates a new texture, you wait until that fills up and then batch gamma correction out of photoshop into your gamma corrected folder.

royter
05-18-2009, 05:19 PM
Not sure if this is the reason but do you have the compression and knee set to 0 in your mia_exposure_simple?

yes they are se to 0 and gain to 1 (in both exemples).
that's why i am surprised with the difference.



Just do each texture correctly from the beginning. You can gamma correct your textures straight in photoshop before they ever hit your 3D application. This is the best way to do things (especially in a work environment with a few people working on one project) because you don't need to explain to other people how to do this. Just set up two folders in your source folder: One that says gamma corrected; the other that says new. Then everytime a person creates a new texture, you wait until that fills up and then batch gamma correction out of photoshop into your gamma corrected folder.

that could work for image textures.
but you still have to ungamma all your procedural textures on way or another.

Ironhalo
05-18-2009, 05:50 PM
im still wrapping my head around gamma and linear worflows, so im im wrong please ignore me.

arent the swatches in maya linear by default? if you're correcting srgb images in photoshop to linear gamma you dont need any gamma correct nodes. if you leave the framebuffer at 1, and leave the tonemapper at 1 you're rendering in complete linear space.

all you'd need to do is temporarily set the lens shader to 2.2 to act as a LUT so you know what you're getting in comp. this is of course assuming you're going to be rendering a 32bit .exr and applying gamma in comp.

royter
05-18-2009, 06:24 PM
i
arent the swatches in maya linear by default?
No they are not: It is true that Maya/MR renders linear images by default. But your color swatches are gamma corrected so they display like they should on your monitor (just like photos taken with cameras are automaticly encoded with a gamma of 2.2 so they display right on monitors). So the renderer is rendering in a linear way is BUT with color swatches that are not linear.

Anyways, We are getting off topic here.
this thread is about 2 things :
1-the reason for the subtle visual difference (see illustration) that exists between the 2 methods.
2-a list of pros and cons of both methods.

So plz guys let's try to stick with these 2 points.

royg
05-18-2009, 07:50 PM
Here's my theory regarding the subtle difference (it appears to only be in the bounced light): That the FG is calculated before the gamma correction takes place on the file texture (in the 0.454545 framebuffer). When you use a framebuffer of 1 and directly connect a gamma node, the FG is using that de-gamma'd file texture to calculate. Thoughts?

Galakgorr
05-18-2009, 07:58 PM
djx has a good post about this, as he always does

http://www.djx.com.au/blog/2008/10/06/linear-workflow-and-gamma-update/#more-101

in most cases it seems like you can get away with tonemapping at gamma 2.2 or 1.8 or whatever and then setting your framebuffer to the inverse. this is usually what i do, and i rarely notice any difference between this method and the correct-every-damn-texture method.

for you guys who like gamma correcting all of your file textures and swatches i wrote a script to do it automatically: http://www.highend3d.com/maya/downloads/mel_scripts/rendering/mental_ray/hfAutoGamma-mel-5676.html

Hamburger
05-25-2009, 12:31 PM
I've noticed that textures become "washed out" when using elliptical filtering and the framebuffer .455 method.

Is this a bug or is it how things are supposed to work?

Normally with .455 framebuffer method, textures are automatically adjusted, however it seems when you turn on elliptical filtering in the file node, this "automatic" translation is broken?

Obviously it's fixed using GC node, but it's odd.

djx
05-25-2009, 02:35 PM
I also had some odd problems with elliptical filtering and framebuffer gamma=0.455, and as a result I've switched back to frameBuffer gamma=1 and exposure gamma=2.2 for most of my work.

The problem was incorrect shading when using final gather and bump mapped surfaces close to camera. Shading became very jagged in the region where the surface curved away from the light. It looked kind of like a low resolution proxy was being used for part of the lighting calculations.

-- David

royter
05-25-2009, 02:57 PM
Here's my theory regarding the subtle difference (it appears to only be in the bounced light): That the FG is calculated before the gamma correction takes place on the file texture (in the 0.454545 framebuffer). When you use a framebuffer of 1 and directly connect a gamma node, the FG is using that de-gamma'd file texture to calculate. Thoughts?

you could be right.





I also had some odd problems with elliptical filtering and framebuffer gamma=0.455, and as a result I've switched back to frameBuffer gamma=1 and exposure gamma=2.2 for most of my work.

The problem was incorrect shading when using final gather and bump mapped surfaces close to camera. Shading became very jagged in the region where the surface curved away from the light. It looked kind of like a low resolution proxy was being used for part of the lighting calculations.

-- David

hey david,
for me the BIG advantage of using a framebuffer 0.455 render is the color bleed issue that you illustrated on your blog. And this was "enough" for me to switch to that method.
But i discovered yesterday that you could use a framebuffer 1 (and mia_exp 2.2 + ungamma nodes...) and obtain accurate color bleed.You just have to apply your lense shader to your output shader and not to the lense shader slot(so the sampling happens before the lense correction and the lense shader happens on a pixel level and not on a sample level).
what do you think about this method?

Ash-Man
05-25-2009, 03:54 PM
@ Dave
Have you tried turning of the filter on the texture node itself, then do the comparison ?

djx
05-25-2009, 11:37 PM
royterr: I've never tried that. I'll give it a go soon. Have you noticed any differences with the aliassing? If the exposure is happening on pixels and not samples I would expect the aliassing to be worse, yes?

Oh, I hope my blog was not the only thing to make you (or anybody else) switch. I write about my experiences at the time, but later I may change my methods when I learn more. The gamma/linear thing is something I still learning about. Sometimes what sounds good in theory has problems in reality. Good to see you are exploring all the possibilities though!

Ash-man: Which comparison? I think for animation you must use some kind of filtering on textures to avoid pixel crawl artifacts. Elliptical filtering is, in most cases, the best choice when trying to preserve the sharpness of the original texture.

-- David

fabergambis
05-26-2009, 01:02 AM
I'll try to tell my eperience with 32 bits mode hoping not to be too OT:
I always used to work with Gamma Correction nodes into all textures set to 0.4545 (except for whites, blacks and bump/displacement maps and I really don't know why :curious:), Framebuffer Gamma set to 1 and Photographic Exposure Gamma set to 2.2 and I always thought it was ok.
With this method, to preserve color swatches, I used to assign solid colors via gamma correction node value directly into diffuse channel.
During my last archiviz project I started to test the other way, using no more gamma correction nodes, framebuffer gamma set to 0.4545 and photographic exposure always at 2.2 NOT at 1 as stated by djx in his very useful blog (that's the only 'wrong' thing I read in it :D ).
Why I use exposure gamma at 2.2 and not at 1?
Simply because I compared the images made with the two different methods and I noticed that the second one was absolutely darker that the first one with the same settings.
I preceidingly made some tests with this second method (frambuffer gamma to 0.4545) and get the same results with exposure photographic set to 2.2 not at 1, but the more I read about it, the more I'm getting confused with this matter.
Was I wrong before, am I wrong now or both?
So I'm asking if the problems pointed by royterr are concerning only 8 bits mode or not, because I also noticed a different bounced lighting result comparing my images with the two methods as supposed by royg in the 7th post.

royter
05-26-2009, 03:40 AM
royterr: I've never tried that. I'll give it a go soon. Have you noticed any differences with the aliassing? If the exposure is happening on pixels and not samples I would expect the aliassing to be worse, yes?
-- David

you are right David. Aliasing is worse (just like the aliasing of a float render).
you can see the comparison between the 2 methods here PSD (http://sor.typepad.com/files/comparison-1.psd) .
(this is anoying , tought i almost got it.....
If only the lense shaders gave correct color bleed)

djx
05-26-2009, 06:37 AM
royterr, I looked at your PSD (in post #15). The difference in the bounced light is more obvious than in your first post. I think royg got it right in post #7 with his theory about fg using the ungamma'd texture.

The join between the red and blue swatches shows a good example of why getting this done correctly is desireable. I think I agree that the aliassing looks worse when you use an output shader, but I'd need to see some more tests to decide if it was too bad to use in production.

Did you also get the "brighter FG" if you rendered to 8-bit with fb-gamma=0.455?

I suppose we could use mip_rayswitch to gamma the fileTexture just for FG, but who wants to do that for a complex scene? :hmm:

In much of my work, these differences would be too subtle to worry about. It would still be nice to have an accurate solution though.

-- David

Hamburger
05-26-2009, 12:49 PM
I think it's interesting and I'd love to here of a "final" solution.

btw, here's a post from Duncan regarding the issue:
http://forums.cgsociety.org/showpost.php?p=5374296&postcount=6

royter
05-26-2009, 02:57 PM
Did you also get the "brighter FG" if you rendered to 8-bit with fb-gamma=0.455?
-- David
didn't try that one.
but i guess the accurate FG solution is the first one (FB gamma 1/mia_exp gamma 2.2).


In much of my work, these differences would be too subtle to worry about.
-- David
you'r talking about the color bleed right?


actually the color bleed issue is not always true.
in this exemple PSD (http://sor.typepad.com/files/comparison-2.psd) you can clearly see that the color bleed of the exposure_simple apllied as a lense shader is more acuurate then if it was applied as an output shader

Hamburger
05-26-2009, 11:04 PM
didn't try that one.
but i guess the accurate FG solution is the first one (FB gamma 1/mia_exp gamma 2.2).


No, read the post by Duncan I posted a link to.

royter
05-27-2009, 12:53 AM
No, read the post by Duncan I posted a link to.
if this means that using gamma correct node for textures with a gamma of 2.2 gives inacurate FG solutions than i am totaly confused.

mr Bob
05-27-2009, 02:08 AM
I'm not sure if it makes sense to gamma correct the textures for lighting

No it does not

If you want to work in a linear pipe you should strip the gamma off every OBJECT texture map you use. Doing it via the framebuffer is not the way to go as it will de gamma everything and set you off into a world of untold pain.

Not sure what you do in MR to achieve this but I would simply use a power function with a constant value plugged into it with a value of 1/0.454545


b

Hamburger
05-27-2009, 02:46 AM
Doing it via the framebuffer is not the way to go as it will de gamma everything and set you off into a world of untold pain.

b

Framebuffer not the way to go? Now I'm confused too. Especially since Autodesk on their Mayastation blog only a few days ago said both methods are fine.

http://mayastation.typepad.com/maya-station/2009/05/colors-are-washed-out-with-mental-ray-sunsky.html

Truly it seems best way to go is strip the gamma off and apply it to the whole image later I agree with Duncan and yourself. But why does mental ray have two lens shaders?

What's the point of even having a mia_photographic lens shader if the framebuffer set to .455 is wrong and gamma correcting textures is wrong? Surely it would've been better for Maya to have a 32bit Render View instead??


Not sure what you do in MR to achieve this but I would simply use a power function with a constant value plugged into it with a value of 1/0.454545
b

What do you mean by "it"?

After reading a lot of things I can only assume it all comes down to personal opinion as there are so many ways to go about this.

Personally, I've never had a problem with framebuffer method so what are the disadvanges? This thread mentioned bad dark/shadow texture problems, but I've never come across it.

royg
05-27-2009, 03:29 AM
Doing it via the framebuffer is not the way to go as it will de gamma everything and set you off into a world of untold pain.What do you mean "everything"? It certainly does not de-gamma colour swatches - only file textures. So I don't quite see where the world of untold pain would come from. Sure, you get more control applying a gamma node on to every file texture in your scene, but that is also a really ugly solution in a complex scene. And a scripting solution makes it just at "brute-forcish" as the framebuffer. Care to elaborate?

djx
05-27-2009, 03:41 AM
royterr: When I say the differences are not enough to worry about, I mean when it is an animated sequence done for tv broadcast you often don't notice small artifacts like the color bleed for example. Maybe now we are moving to HD things will change a bit.
In your comparison-2.psd I think part of the difference may be due to the sampling method (pixel vs sub-pixel). These are really interesting findings though.

HamburgerTrain: I read Duncan's post a while ago and I think I agree with his logic. I think this gamma issue often gets complicated because it is possible to apply corrections at almost every stage of the pipeline. Then add in the way the renderer treats the data internally depending on a whole bunch of different factors and you get a bit of a mess. (And thats not even mentioning bugs with elliptical filtering.) But I still think its possible, with a bit of testing, to come up with a workable solution, but its hard for me to tell someone else "this is how you shoudl do it" unless they do every step exactly like me.

One thing to remember with the exposure nodes is that they do more than just gamma. So you may wish to use frame-buffer gamma=0.455 and lensShader gamma=1, but you might still want to reduce the dynamic range of the image to avoid high contrast aliassing problems for example.

-- David

ArianTibbs
05-27-2009, 11:42 AM
So the way things have all progressed up until this point, what ARE the correct settings to use? If there are two equally acceptable solutions, what are those settings? Including for textures, framebuffer, and the exposure?

EDIT - Found the answer on the link posted a couple replies up. Sorry.

1. Per file texture
Add a Gamma correct node between the texture and the Shader
Change the Gamma value to (1 / 2.2 )= 0.455

2. Per Scene
In the render settings, under the “Quality” tab > framebuffer section change the value to .455
Under the mia_exposure node, change the Gamma value to 1.0

wizlon
05-27-2009, 02:50 PM
if this means that using gamma correct node for textures with a gamma of 2.2 gives inaccurate FG solutions than i am totaly confused.

From my tests it appears that if you use gamma correct nodes for textures and colour swatches and set lens shader to 2.0 you should also set secondary bounce scale in your render settings to 2. This then compensates and gives accurate colour bounce results.

Please can someone else confirm this.

djx
05-27-2009, 02:56 PM
Its been bothering me that Duncan suggested (http://forums.cgsociety.org/showpost.php?p=5374296&postcount=6) that method 2 was better (framebuffer gamma 0.4545, no degamma on fileTextures). In particular this quote:

For example if the color value was 0.5 it would scatter half the light hitting it, however with the gamma applied to the texture it might only scatter a small portion of light hitting it. This would not be accurate if the color was suppose to represent a 50% grey material.

It bothered me because the test renders seem to contradict this - and Duncan knows his stuff. A degamma'd texture with a lens-gamma=2.2 (method 1) looks correct, while an un-degamma'd texture with a framebuffer-gamma=0.4545 (method 2) has incorrect fg bounce light which is too bright.

Then it occured to me that Duncan's comments are completely correct if we use linear fileTextures (32 bit float EXR for example).

Remember that a typical 8-bit JPG or IFF looks correct on our monitors because it is in sRGB color (or something close). So what looks like a color of 0.5 is actually stored in the file as a brighter value to compensate for the monitor gamma (typically 2.2).

But if we convert that same image to 32-bit float the data will be stored without that compensation and may therefore look darker depending on the application you view it with.

I think the problem is that when maya renders using an 8-bit image, the indirect lighting calculations uses the actual values in the file, ignoring compensation gamma, which leads to overbright finalgather unless we degamma the image first. But if we do that with method 2 we get a double degamma and the result is too dark.

But if we use the 32-bit float image we get the correct result with either method 1 or method 2 and neither method requires us to degamma the texture.

So if we use 32-bit float EXR fileTextures with method 2 we get correct color bleed and correct final gather and we dont need the gamma node on our color fileTextures.

wow... that was longer than I planned :hmm:

-- David

royter
05-27-2009, 03:00 PM
From my tests it appears that if you use gamma correct nodes for textures and colour swatches and set lens shader to 2.0 you should also set secondary bounce scale in your render settings to 2. This then compensates and gives accurate colour bounce results.

Please can someone else confirm this.

of course you can do that.
but that's not a solution.
you must have the same FG solution with exactly the same FG settings.
and if the 2 results are different then one of them is "wrong" and that the image rendered with method B.

ArianTibbs
05-27-2009, 05:18 PM
(snip)

Then it occured to me that Duncan's comments are completely correct if we use linear fileTextures (32 bit float EXR for example).

Remember that a typical 8-bit JPG or IFF looks correct on our monitors because it is in sRGB color (or something close). So what looks like a color of 0.5 is actually stored in the file as a brighter value to compensate for the monitor gamma (typically 2.2).

But if we convert that same image to 32-bit float the data will be stored without that compensation and may therefore look darker depending on the application you view it with.

I think the problem is that when maya renders using an 8-bit image, the indirect lighting calculations uses the actual values in the file, ignoring compensation gamma, which leads to overbright finalgather unless we degamma the image first. But if we do that with method 2 we get a double degamma and the result is too dark.

But if we use the 32-bit float image we get the correct result with either method 1 or method 2 and neither method requires us to degamma the texture.

So if we use 32-bit float EXR fileTextures with method 2 we get correct color bleed and correct final gather and we dont need the gamma node on our color fileTextures.
-- David

So using method one, if we place a gamma correct node in front of an 8-bit texture, it will still be incorrect? The only true method to work is to have 32bit/float openEXR files as the textures?

djx
05-27-2009, 10:38 PM
Originally posted by Arian Tibbs
So using method one, if we place a gamma correct node in front of an 8-bit texture, it will still be incorrect?

No, I think this would also be correct.

But with method 1 the blending of colors in sub-pixel sampling is handled differently, resulting in a slightly unnatural result. For example this is most noticeable at the intersection between a pure red and a pure blue surface and is much less noticeable in general, so in my opinion method 1 would still be a valid solution.

-- David

leif3d
05-27-2009, 11:20 PM
Man...this thread confused me again...but that's good I guess :)

When you guys say "convert the image to a float EXR," is there a specific process you guys go through to convert this image? or is changing it's color space and saving as EXR enough?

Chaging the color space in photoshop and saving as HDRI or EXR seems to work fine. It seems to give me the same results as adding a gamma node to the 8-bit file. Apparently the render assumes the image doesn't have gamma applied to it because it is Float.

Or do I need to take out the gamma embedded into it somehow and then save the file to Float?

djx
05-27-2009, 11:46 PM
Chaging the color space in photoshop and saving as HDRI or EXR seems to work fine.

Yes, that's all I do too.

-- David

djx
05-28-2009, 12:14 AM
But if we convert that same image to 32-bit float the data will be stored without that compensation and may therefore look darker depending on the application you view it with.

I thought I would clarify what I mean here.
In photoshop, when you swicth the mode from 8-bit to 32-bit the image may or may not change to display darker depending on how you have your preferences set.

Here's a snapshot
http://www.djx.com.au/cgtalk/gamma/photoshopPrefs_GLSettingsX.jpg

If "Color Matching" is checked as shown, then your display will show linear 32-bit images with compensation for the display gamma, so the image will still look like the 8-bit version.

If "Color Matching" is not checked the image will look darker.

You can use the info window to compare rgb values and see that they are the same in both cases. So for the 0.5 grey example, 128 in 8-bit sRGB becomes 55 in 32-bit linear.

-- David

leif3d
05-28-2009, 01:15 AM
Thanks for the clarification David.

wizlon
05-28-2009, 07:53 AM
Getting more confused now.

How does this affect colour swatches?

If I have a deep red colour (R - 0.5 G - 0 B - 0) which I then add a gamma node too.
Which gives me (R - 0.25 G - 0 B - 0) given a lens shader gamma value of 2, does this then mean I have incorrect FG contributions?

wizlon
05-28-2009, 12:00 PM
See attached images, FG bounce colour fades away. Shouldn't they all be identical as the colour remains consistent but the FG does not.

wizlon
05-28-2009, 12:01 PM
one more at 2.2.

Hamburger
05-28-2009, 12:07 PM
Is there anyone in this thread that isn't confused now? :argh:

wizlon
05-28-2009, 12:29 PM
Just to be clear. All the above images are the same bar the gamma values.

Mia exposure simple was used as a lens shader.

Maya gammaCorrect nodes were used to correct the colour swatches.
(Also tried mip_gamma_gain but this gives same result.)

The diffuse colour of the ball and floor remain consistant through all images, as it should. But FG practically disappears as we increase gammas influence.

Hamburger
05-28-2009, 12:39 PM
What happens if you use Irradiance Particles?

lazzhar
05-28-2009, 01:43 PM
While this linear thing helped a lot to improve some people, it confused a good part.
I feel like it's turning into circles if autodesk doesn't say something about it in an official documentation.
For god sake, put it in the manual even if some will always say it's wrong.

djx
05-28-2009, 02:15 PM
Shouldn't they all be identical as the colour remains consistent but the FG does not.
No, they should not look identical. The color value used in the final gather calculations depends on the degamma correction in each case. We want all our lighting/shading calculations (including finalgather) to be done in a linear space and only at the end do we apply our 2.2 gamma so that it looks correct on our display device. We need to do the reverse for swatches and 8-bit fileTextures. They already have the 2.2 gamma, and they look fine on our monitor, so we need to remove that by degamma of 0.455, so that they will be correct in the linear space.

In short: color swatches always need to be degamma'd if you want fg calculated correctly. So whether you use framebufferGamma=0.455 or lensShaderGamma=2.2 you still need to apply a degamma of 0.455 to the color swatches.

(You can use mip_gamma_gain with gamma=0.455 if you need to input a specific color, but I find that after a while I can just guess and enter a darker color)

In my opinion your test images show how fg is incorrect if you leave swatches uncorrected. The first image shows an unnatural irradience. Your last image shows no noticeable color bounce, but I think if you increase your finalgather secondary diffuse bounces to 2 then you will start to see something more like you would expect.

-- David

leif3d
05-28-2009, 03:25 PM
I've been getting a little confused after reading a couple of threads on what needs to be gamma corrected.
We all know that 8-bit textures need to be, but when you guys say gamma correct swatches, you mean any color input that has an RBG input right? Because I read somewhere that inputs that have a single luminance value (Black or white) don't need gamma correction correct?

Where I get confused, is in transparency or specular color, because according to this, I would have to gamma correct them even when choosing a gray value, because it's an RGB input right?
Well...according to what I've read, this would not be the case, and I would have to leave the input alone unless I'm choosing a "color" not a black and white value. Can anyone confirm this?

What really strikes me as odd, is how the renderer understands if the input needs to considered when adding gamma to the final image. Apparently it would not add gamma to a specular input with a <<.5,.5,.5>> value, but it would add gamma to the same input if it finds any color <<.5,.2,.7>> ???

Is this correct? or should I be gamma correcting my transparency and specular even though they are black, white or gray?

wizlon
05-28-2009, 03:46 PM
In my opinion your test images show how fg is incorrect if you leave swatches uncorrected. The first image shows an unnatural irradience. Your last image shows no noticeable color bounce, but I think if you increase your finalgather secondary diffuse bounces to 2 then you will start to see something more like you would expect.

-- David

djx, The scene above has 8 diffuse bounces, 8 reflection bounces (as otherwise this caps diffuse bounces) and 0 refraction bounces, with max trace depth set to 16.

I think the first image is correct, and as gamma increases FG incorrectly gets calculated on the gamma corrected colour (whos HSV 'value' drops further towards black as the gamma vaule increases i.e. 2.2 / 0.454.)

If i render this scene with no gamma correction no lens shaders (32bit exr) and then correct in PS or shake the image looks like the first one above, which leads me to believe it is the correct one.

wizlon
05-28-2009, 04:52 PM
The first image shows an unnatural irradience. Your last image shows no noticeable color bounce, but I think if you increase your finalgather secondary diffuse bounces to 2 then you will start to see something more like you would expect.

-- David


djx, based on your comment that the first image shows unnatural irradience, I checked my scene and found the Secondary Diffuse Bounce Scale to be on 1.8 from a previous test.

However I fixed this and re-ran the tests and the results still hold. This time with 16 diffuse bounces.

The first image is the correct one and the second shows almost no colour bleed.

This is due to mentalray/Maya FG implementation looking at the wrong values from the gamma corrected colour swatch (wether using gammacorrect node or eyeballing it). The red sphere is RGB 0.5,0,0 but FG looks at it and sees RGB 0.217,0,0 because of the gammacorrect node applied to it, which is set to 0.454 (i.e. 1 divided by 2.2.) FG calculation should see the red sphere as 0.5,0,0 as this is the correct reflectance value regardless of how the final image is displayed.

The only solution to this would be a application wide gamma solution as they have in Max.
Where colour swatces can be set to linear space with a single click. The colours are now linearized and your eye perceives them as so, but values remain the same i.e. 0.5,0,0 has remained unchanged unlike Maya where this has become 0.217,0,0.

Ironhalo
05-28-2009, 05:06 PM
man, reading through all the gamma stuff lately really messes with your head!

leif, i thought that any input that was a single float didnt need gamma correction. i think marius said i was totally wrong, and someone later tested it and agreed with marius. i think the problem is if you view greyscale images with an srgb color space. if they're converted to linear then no gamma correction is needed.

im guessing mental ray ignores any kind of color profile and reads it as linear information. that means (assuming a srgb profile) that the render engine will see a washed out image. if anyone can expand on this i'd appreciate it.

all of this would be so much easier to deal with if the maya swatches and render view defaulted to linear space!

bart from lamrug talked about a fully linear pipeline: every texture was degamma'd in photoshop and converted to a 32bit exr. (i do 16bit, havent seen a difference) any color swatch or procedural color in maya should be degamma'd, since they're all a 2.2 gamma. render to 32bit exr, and gamma/gamut in post. i know this isnt one of the original posters 2 options, but i think keeping everything linear in render keeps things less confusing.

djx, this quote confused me:

You can use the info window to compare rgb values and see that they are the same in both cases. So for the 0.5 grey example, 128 in 8-bit sRGB becomes 55 in 32-bit linear.


you're not saying 50% grey changes with bit depth right? if it doesn im completely confused again :banghead:

leif3d
05-28-2009, 05:28 PM
Hi Paul,

If what you're saying is correct, then I would need to de-gamma my transparency right? After all, it is a color swatch and it has an RGB input right? The thing is, that it just doesn't make any sense to me, because it's driving a value (0-1), not a color, but it needs a color input.


I hope I'm getting through with what I'm saying...

transparency > (.4,.4,.4) - does it need gamma correction? it shouldn't...
transparency > (.5,0,0) - does it need gamma correction? it should

they are both rgb values going into an rgb input, but apparently the gamma correction is only needed in non-luminance (non 0-1) values.

Ironhalo
05-28-2009, 05:45 PM
my understanding is that if the input value is anything except a 0 or a 1 it will need correcting. assuming:

-you're using a maya swatch
-you're using a file texture thats in srgb space

-if its a file texure has the gamma removed (ie linear) its ok.

like you said, gamma wont change the 0 point or the 1 point.

leif3d
05-28-2009, 05:53 PM
Great, thanks for confirming Paul. It makes a lot more sense now. :)

treedee
05-28-2009, 06:17 PM
Man this is confusing... :curious:

So file textures that are 8 bit need to be converted to 32bit exr first?

Since the internal renderer values are floating point, all colors swatches within Maya are fine and do not need to be set to .454 individually? I think this is the part thats most confusing to me...

And Lastly Does this workflow only apply to Maya when using Mental Ray? What about Maya software, Renderman, others etc?

leif3d
05-28-2009, 06:27 PM
So file textures that are 8 bit need to be converted to 32bit exr first?

You can either gamma correct them in Maya or do it in photoshop by converting it to a Floating point EXR. Either will give a slight color blend difference, but it is negligible. Therefore both are recomended.


Since the internal renderer values are floating point, all colors swatches within Maya are fine and do not need to be set to .454 individually? I think this is the part thats most confusing to me...


Quite the opposite, you need to gamma correct every single color swatch that is not 0 or 1. The issue of final gather not giving correct color bleed seems to still be an issue though...


And Lastly Does this workflow only apply to Maya when using Mental Ray? What about Maya software, Renderman, others etc?

This applies for any renderer. I do this for renderman actually and it works great.

Please step in, if I'm wrong in any way :) I too get very confused.

treedee
05-28-2009, 07:05 PM
Thank you Leif, just when I thought I had a grasp on this workflow...I like the idea of Method (2) as just setting the frame buffer in Maya to .454 is easier for me. What confuses me now is if Im not using Mental ray lets say renderman I wont need to apply any gamma correction to my cameras only the frame buffer settings in the render settings?


So my workflow would be:

1) if using Mental Ray Features/Quality/Framebuffer Gamma .454 RGBA 4x32 Bit

2) ( Maya Software ) Render Options/Color Compositing/ Gamma correction .454

3) Then make sure file textures are 32bit or within Maya and Maybe a Mel script that converts your color swatches to .454 without you having to do all of this manually?

4) I am evaluating RFM 3, and are curious also what its equivalent?

royter
05-28-2009, 07:11 PM
Just to make things clearer.
The FG solution in method A is accurate.
So when we are talking about the FG issue, we are talking about about case B (FB gamma 0.45).

As for the the ungamma node, it should be hooked to every color swatch in a shader: only 2 exceptions: displacement and bump.

Ironhalo
05-28-2009, 07:17 PM
just to throw another wrench into the mix... i read somewhere that zbrush will create linear files for you automatically. obviously this matters a lot for normal maps. can anyone speak to the accuracy of this?

royter
05-28-2009, 07:25 PM
just to throw another wrench into the mix... i read somewhere that zbrush will create linear files for you automatically. obviously this matters a lot for normal maps. can anyone speak to the accuracy of this?

it doesn't matter because bump maps/normal maps/displacement maps don't need degamma anyways.

leif3d
05-28-2009, 09:31 PM
So my workflow would be:

1) if using Mental Ray Features/Quality/Framebuffer Gamma .454 RGBA 4x32 Bit

2) ( Maya Software ) Render Options/Color Compositing/ Gamma correction .454


If you go through the thread a couple of pages back, DJX and others have been discussing the drawbacks in setting a global framebuffer to .454. In the end, it's not a good idea.
The better way is to gamma correct all swatches which are not 0 or 1 and leave the framebuffer at 1.


3) Then make sure file textures are 32bit or within Maya and Maybe a Mel script that converts your color swatches to .454 without you having to do all of this manually?


Well, it's either or. If you change the framebuffer gamma to .454, then you leave all file textures alone. If the framebuffer is 1 then you change the gamma of all swatches and textures to .454 manually. This last, will give you more predictable results.


4) I am evaluating RFM 3, and are curious also what its equivalent?

I'm also figuring that out now. I've already asked this in Pixar forums. BTW, a new feature was added to the RM texture controls which applies gamma correction to textures. To use it do the following:

in a FileTexture node go to > Attributes > Renderman > Add Renderman Texture Controls > in the extra attributes tab in the file texture you should now see "Apply sRGB" This will Gamma correct your 8-bit texture.

treedee
05-28-2009, 09:39 PM
Thanks again for the info leif. I've always been curious as to why this workflow was never implemented in the very beginning. I do understand we were or are conditioned to viewing things incorrectly from the start just seems like it should be as easy as one switch to let you choose the workflow you would like to work in. Almost like how Nuke lets you choose between your color space on a read-in node.

Hamburger
05-28-2009, 09:41 PM
By the way, this was posted on the Maya blog....

Also, Maya swatches need to be adjusted with gammaCorrect nodes using the frambuffer .455 method and also file textures with Elliptical filtering used.

The reply from Ash Aiad (Maya support) was:

As for the other note, its under the developments consideration. Cheers Ash

So hopefully something happens like what wizlon was talking about in Max.

SlipAway
05-28-2009, 10:00 PM
The more people figure this out, the more confused I get. I've said in other posts I wish the Maya/mental ray people would figure this out for us and set one standard all of us can understand and learn to work with and include it in the documentation.

So to unconfuse myself, how does one gamma correct Maya swatches? Does the same need to be done for mental ray swatches? This applies only to colored shaders I'm assuming and has nothing to do with file textures right? :banghead:

djx
05-29-2009, 06:59 AM
I feel like it's turning into circles
I like to think of them as spirals, and the truth is at one end, but I haven't figured out which end yet :curious:

Just to make things clearer.
The FG solution in method A is accurate.
So when we are talking about the FG issue, we are talking about about case B (FB gamma 0.45).
That's what I thought too. But then wizlon made some new tests which caused me to rethink some of this. I've been swayed back to method B (FB gamma 0.45) as being the accurate FG.

Here's my modified view:
When we render in a linear space with the intention of adding gamma 2.2 at the end we need to degamma our 8-bit sRGB fileTextures, bacause they already have a 2.2 gamma and we dont want to double up. If we use method A (lensShader gamma=2.2) then we must insert gamma nodes, but if we use method B (FB gamma 0.45) then we dont need the extra gamma nodes and the renderer degamma's the whole frame buffer. Both methods give us the same overall color.

The difference between the two methods changes the way FG is working.

If we use method A and we degamma the fileTexture then FG is using a darker version of the fileTexture and that is not good. FG is not really a color, it is the amount of diffuse light being bounced back. If our original fileTexture was 50% grey, then FG should bounce back half the light that hits it. With method A, where we degamma the fileTexture the 50% is reduced to approx 21% so the FG is incorrectly reduced.

Compare this to method B where we do not degamma the fileTexture. The FG sees 50% grey and bounces half the light. The resulting frameBuffer contents are then degamma'd so that we end up with our sRGB image. I now think this is the correct one :blush:

I realize now that this is really what Duncan was talkng about in the earlier quote.

Unfortunately this creates a problem when you use 32-bit fileTextures. These are linear so they dont need degamma as far as color is concerned. But for FG they are wrong. In fact to get the correct result for FG you need to add a gammaNode=2.2 - but just for FG, which you can do with mip_rayswitch. I dont really thing I'll be going that route very
often though.

I'm sorry if I've been one of the contributors making this thread confusing, but for me it has been quite an educational process, and I welcome your thoughts and contradictions. Maybe I'm getting to the tight end of the spiral. :curious:

-- David

wizlon
05-29-2009, 08:57 AM
Just to make things clearer.
The FG solution in method A is accurate.
So when we are talking about the FG issue, we are talking about about case B (FB gamma 0.45).

Both solutions are wrong for FG.

Here's a solution for colour swatches. (Will also work for Textures.) See attached.

djx
05-29-2009, 09:22 AM
Hey wizlon, thanks for all your input.

That just looks way too complicated to implement in practice.
You said "will also work for textures" but I just tried a 50%red swatch rendered with your setup and it looked the same as a 50%red fileTexture connected directly to the shader (no extra nodes). I'm using method B (FB gamma 0.45).

Can I suggest a simpler swatch set up too.
I used a mip_rayswitch.outValue connected to mia_material_x.diffuse.
I connected a mip_gamma_gain.outValue to the mip_rayswitch.eye with gamma=0.455
I connected the same mip_gamma_gain.input to mip_rayswitch.finalgather.
So I only have to enter one swatch value, and still only have one material.

It seems to work. What do you think?
I'm making an assumption that the secondary rays are derrived from the primary FG.

-- David

leif3d
05-29-2009, 02:58 PM
Ok, so Framebuffer at .454 fixes the color bleed issue right?
So how do you deal with all the random texture artifacts, color blending issues and other problems that where pointed out by other users?

It seems to me like the safest way to work is with method A...and the then I just tweak my color bleed until I get what I'm looking for. Can't I just tweak the Primary and Secondary diffuse scales in the render globals to compensate for missing color bleed?

royter
05-29-2009, 04:47 PM
With method A, where we degamma the fileTexture the 50% is reduced to approx 21% so the FG is incorrectly reduced.
-- David

i don't think so, i think the fg calculation is right in method A.
If MR works in a linear way, it does it in all it's calculations.
So when a texture is gamma corrected it becomes darker in the viewport but not in MR eyes.
So when it calculates FG, it sees the right texture (texture with linear gamma curve) and after that the lense shader is applied.Here's what happens in method A:
http://sor.typepad.com/.a/6a00d8350600cb53ef011570afd76e970b-800wi

leif3d
05-29-2009, 05:04 PM
What you're saying makes sense Roy, but I think David came to this conclusion after the tests Lee did with the red ball.
If what you are saying is correct then the color bounce would stay consistent independent of the gamma node, but it doesn't. So there is obviously a problem somewhere.

royter
05-29-2009, 06:03 PM
If what you are saying is correct then the color bounce would stay consistent independent of the gamma node, but it doesn't. So there is obviously a problem somewhere.

i did alot of tests and i didn't find any difference in the FG multibounce solution between method A and B.
and i personally think that the FG solution in B is blown out. Maby because the FG solution in B happens before gamma correction, hence calculating FG with wahed out textures. Now i am not sure what's happening in B, it's just a guess. But i am definitivly sure about what hapens in A.

djx
05-30-2009, 12:49 AM
What you're saying makes sense Roy, but I think David came to this conclusion after the tests Lee did with the red ball.

Frustratingly, I agree with both parts of this sentence.

The red ball tests seemed to indicate that with method A, as we increase the exposure gamma and decrease the fileTexture gamma, then we see a reduction in the FG intensity, but it should remain constant.

Roy, your diagram is how I thought it worked in my earlier posts on this thread. After seeing the red ball tests I started to rethink it. (I should add that in the first set of tests, especially the first image, the FG was way too strong, but when he reposted them it looked more reasonable. I've done the same tests myself, with pretty much the same result.)

What if MR is doing what photoshop does?

If I look at my 8-bit fileTexture in photoshop and sample the middle grey color I see a value of 50% in the info window. I know that the data stored in the file is actually different due to the sRGB profile, but if photoshop tells me the number is 50% then maybe MR does the same.

Maybe MR also assumes a sRGB profile for an 8-bit image and reworks the numbers accordingly when it reads the file. If so then the dark line your 1st graph would be linear, and in your 2nd graph would be lower in the middle. This would be consistent with the results shown for FG in those red ball tests. In the 3rd graph we reapply 2.2 gamma and get back to linear, and then MR writes out an 8-bit file and applys an sRGB profile again.

-- David

redforty
05-30-2009, 01:17 AM
I'm just thinking out loud here.

Is there a way to basically 'gamma correct' FG calculation without using the rayswitch method?
For example: Increasing the "Primary Diffuse Scale" to 2.2?

I'm not in at my work comp, so I can't test it. :-(

djx
05-30-2009, 01:38 AM
I tried it, but it is still not correct. This is a gain control and affects the whole solution. Remember the problem is caused by gamma, so only the middle values need correcting.

-- David

redforty
05-30-2009, 10:31 AM
Damn. :-(

The problem I've been running into lately is that I do almost all of my shading in Maya without textures. And because of this, I've been working with a viewport that is always too dark and it's pretty much impossible to work with playblasts.

One workaround to that dilemma I've found is to always use mentalray shaders whilst plugging in lamberts or blinns into the standard material slot of the shader group. But this just doubles the amount of work when I'm already adding a gamma correct node to every single color swatch in every single shader network. And good god, this method gets tedious when working with complex shader networks.

It would be cool to gamma-correct the viewport when working in linear space.

wizlon
05-30-2009, 02:48 PM
That just looks way too complicated to implement in practice.
You said "will also work for textures" but I just tried a 50%red swatch rendered with your setup and it looked the same as a 50%red fileTexture connected directly to the shader (no extra nodes). I'm using method B (FB gamma 0.45).

Can I suggest a simpler swatch set up too.
I used a mip_rayswitch.outValue connected to mia_material_x.diffuse.
I connected a mip_gamma_gain.outValue to the mip_rayswitch.eye with gamma=0.455
I connected the same mip_gamma_gain.input to mip_rayswitch.finalgather.
So I only have to enter one swatch value, and still only have one material.

It seems to work. What do you think?
I'm making an assumption that the secondary rays are derrived from the primary FG.

-- David


Hi David.

I'm only using method A, FrameBuffer Gamma 1, mia_exposure Gamma 2.2 , and this appears to work fine using the above set-up for both file textures and colour swatches.

Method B may be fine if all you use is file textures, but otherwise there's no avoiding a complex gamma node workflow, as framebuffer gamma doesn't work for colour swatches. (Here we use 99.9% mia and carpaint.)

So unfortunately the only way I see to get everything working the we want is to use this complicated set-up, until Autodesk address the gamma workflow issues.

FrameBuffer Gamma 1
mia_exposure Gamma 2.2
Gamma correct nodes for file textures and colour swatches (0.454) for all materials
Non gamma corrected file textures and colour swatches for FG and any secondary rays.

It may look complicated but it's quick and easy to set-up, just duplicate your material break the gamma connection drop the first one into mip_rayswitch_advanced default and the other into finalgather and any secondary.

-complicated gamma non gamma set-up.
+accurate results in gradients
+accurate color bleed in pixel sampling

David,I've tried your set-up but couldn't get it to work right. could you post me a scene?

Cheers.

Lee.

wizlon
05-30-2009, 02:51 PM
Ok, so Framebuffer at .454 fixes the color bleed issue right?

Only for file textures (will test this Monday to make sure)

Not so for colour swatches, as framebuffer gamma does not affect these.

royter
05-31-2009, 12:01 AM
Frustratingly, I agree with both parts of this sentence.

The red ball tests seemed to indicate that with method A, as we increase the exposure gamma and decrease the fileTexture gamma, then we see a reduction in the FG intensity, but it should remain constant.

Roy, your diagram is how I thought it worked in my earlier posts on this thread. After seeing the red ball tests I started to rethink it. (I should add that in the first set of tests, especially the first image, the FG was way too strong, but when he reposted them it looked more reasonable. I've done the same tests myself, with pretty much the same result.)

-- David

with a mia_exposure gamma of 2.2, i am getting normal FG bounce.
i don't know why it's not showing in your test.
here's a scene (http://sor.typepad.com/files/sceme4.mb) with a simple directional light.As you can see with method A, FG works just fine:
http://sor.typepad.com/.a/6a00d8350600cb53ef011570b29b98970b-pi

in the second image the FG solution is brighter because you are telling FG to use the texture (or color swatch) with a gamma of 2.2 (wilzon's recomandation), and as expected it's brighter because it is computing FG with non linear textures.
i don't know what makes u think that MR is working with a dark texture because it's not.
FG is being computed correctly with the linear textures or color swatches in method A.
hope my scene will proove what i am saying.

djx
05-31-2009, 03:18 AM
with a mia_exposure gamma of 2.2, i am getting normal FG bounce.
i don't know why it's not showing in your test.
Hey Roy, I think this is because your red has a value of 1 and is therefore not affected by any gamma change.

I'll post some pictures and a scene soon.

-- David

djx
05-31-2009, 03:02 PM
Ok, here's my 50% red ball test
http://www.djx.com.au/cgtalk/gamma/redBallTest1.jpg
I've attached two scene files (maya 2008 .ma) showing my gamma set up for 8-bit fileTextures and swatches for both methods. Both renders look the same. The color for the left side of the ball comes from a swatch and the right side comes from a fileTexture (included in the zip)

In both methodA and B the swatch set up is the same. mia_material_x and diffuse color is set using mip_rayswitch with inputs from a mip_gamma_gain with color swatch=(0.5,0,0), gamma=0.455. The connections are:

mip_gamma_gain.input to mip_rayswitch.finalgather
mip_gamma_gain.outValue to mip_rayswitch.eye
mip_gamma_gain.outValue to mip_rayswitch.default

FileTextures are handled differently in each method.
In method A:

file.outColor to mip_rayswitch.finalgather
file.outColor to mip_gamma_gain.input
mip_gamma_gain.outValue to mip_rayswitch.eye
mip_gamma_gain.outValue to mip_rayswitch.default

In methodB the fileTexture connects straight to the diffuse color of the mia with no extra nodes required.

The only extra thing to be aware of is that I exagerated the FG in the render by increasing the finalGather scale to 2 in renderSettings. This was done so I could more easily compare the results and has nothing to do with the actual setup.

-- David

royter
05-31-2009, 05:28 PM
Ok, here's my 50% red ball test
http://www.djx.com.au/cgtalk/gamma/redBallTest1.jpg
I've attached two scene files (maya 2008 .ma) showing my gamma set up for 8-bit fileTextures and swatches for both methods. Both renders look the same. The color for the left side of the ball comes from a swatch and the right side comes from a fileTexture (included in the zip)

In both methodA and B the swatch set up is the same. mia_material_x and diffuse color is set using mip_rayswitch with inputs from a mip_gamma_gain with color swatch=(0.5,0,0), gamma=0.455. The connections are:

mip_gamma_gain.input to mip_rayswitch.finalgather
mip_gamma_gain.outValue to mip_rayswitch.eye
mip_gamma_gain.outValue to mip_rayswitch.default

FileTextures are handled differently in each method.
In method A:

file.outColor to mip_rayswitch.finalgather
file.outColor to mip_gamma_gain.input
mip_gamma_gain.outValue to mip_rayswitch.eye
mip_gamma_gain.outValue to mip_rayswitch.default

In methodB the fileTexture connects straight to the diffuse color of the mia with no extra nodes required.

The only extra thing to be aware of is that I exagerated the FG in the render by increasing the finalGather scale to 2 in renderSettings. This was done so I could more easily compare the results and has nothing to do with the actual setup.

-- David

Ok i see the problem now:

the FG solution (specially secondary bounce) is weakened bu the ungamma node when using method A. I can see what you are talking about now. Youe tests prouve that the problem goes for color swatches and textures.
And you are utilizing the rayswitch node to tell MR : use the gamma corrected texture/color swatch for eye/default rays, and use the non gamma corrected color swatch/texture in the same time for thr FG computation.right?

Now in theory MR should use the gamma corrected (or linear) color/texture for eye/default/fg and all other rays (just like my illustration a fiew posts earlier).
But You are suggesting that FG is using the same values of the gamma corrected texture that we see in our viewport (dark texture) in the FG computation, right?

If that's the case, then the error comes from nvidia/autodesk and not from our side.
Wich means that MR should use the same gamma corrected color/texture for all of it's calculation, even FG. Do you agree?

Because otherwise it would be totaly insane to use this workaround each time you nedd tp gamma corrected a slot. (think of complexe scenes with fluids/tons of materils/paint effects/particles, that would result in thousannds of nodes...)

djx
06-01-2009, 03:57 AM
royterr: Yes I agree with your summary. I'm not sure exactly who is at fault but it seems to be a problem in the way mentalray for maya interprets input for FG calculations, so does that mean it's an autodesk bug?

I dont think I will change my work flow to use the extra rayswitch nodes unless I am doing something where the difference is really obvious. I've been doing rendering with fg for many projects and if it was not for you and wizlon I would have still been blissfully ignorant about this problem. :hmm:
-- David

ArianTibbs
06-01-2009, 03:59 AM
I dont think I will change my work flow to use the extra rayswitch nodes unless I am doing something where the difference is really obvious. I've been doing rendering with fg for many projects and if it was not for you and wizlon I would have still been blissfully ignorant about this problem. :hmm:
-- David

Just out of sheer curiosity, what is the workflow/settings you're planning on sticking to for the time being?

royter
06-01-2009, 04:41 AM
I dont think I will change my work flow to use the extra rayswitch nodes unless I am doing something where the difference is really obvious. I've been doing rendering with fg for many projects and if it was not for you and wizlon I would have still been blissfully ignorant about this problem. :hmm:
-- David


it could be a bug.
Or it could be that the right FG multibounce solution is the one without rayswitch node, afterall there's a subtle red color bleed on the ground and maby that's the right solution and not the one with the rayswitch node. maby this is how it's supposed to be, don't forget that FG color bleed is subtle compare to color bleed obtained with photons.
(i wonder if it's the same problem with GI).
i would really like to hear what Masterzap has to say about that.

djx
06-01-2009, 04:43 AM
what is the workflow/settings you're planning on sticking to for the time being?
Ummm... method B (frame buffer gamma correction) unless I have issues with elliptical filtering in which case method A (mia_exposure gamma correction). Actualy, I'm comfortable with either method and dont mind adding the gamma correct nodes to my fileTextures. And I usually just eyeball the swatches. But I'm not comfortable with having to add rayswitch nodes all over the place. That kind of FG accuracy is beyond the needs of the projects I usually work on.

-- David

anevsky
06-01-2009, 06:55 AM
This whole thread just makes me want to become a gardener instead. Just work in the sun.

no?

seriously, someone please write a script that fixes everything. You just choose method A or B and the rest is under the hood.

a "blissfully ignorant" pill sounds good too.

royter
06-05-2009, 03:23 PM
here's a mathematical explenation of what's hapening.
i still haven't found a direct link bteween these numbers and the usage of gamma nodes:
http://www.highend3d.com/boards/index.php?showtopic=258879&pid=308699&mode=threaded&start=#entry308699

redforty
06-05-2009, 04:16 PM
It would be cool to gamma-correct the viewport when working in linear space.

Looks like Max has it handled:
http://mentalraytips.blogspot.com/2009/05/3ds-max-2010-metasl-and-mental-mill.html

Now, when is big "Industry Standard" going to catch up? :-P

Hamburger
06-06-2009, 01:45 AM
here's a mathematical explenation of what's hapening.
i still haven't found a direct link bteween these numbers and the usage of gamma nodes:
http://www.highend3d.com/boards/index.php?showtopic=258879&pid=308699&mode=threaded&start=#entry308699

Is there anything Joojaa doesn't know about? :bowdown:

Looks like Max has it handled:
http://mentalraytips.blogspot.com/2009/05/3ds-max-2010-metasl-and-mental-mill.html

Now, when is big "Industry Standard" going to catch up? :-P

Hopefully 2010, Autodesk are aware of the gamma problem. Last I heard it was under "development consideration".

Hamburger
06-07-2009, 07:36 AM
http://www.youtube.com/watch?v=QIFaAmZGAGY&feature=rec-HM-fresh+div
New Features in 3ds Max Design 2010: Gamma Improvements

Wouldn't it be nice if it all just worked like that.

djx
06-07-2009, 12:40 PM
YES
-- David

royter
06-07-2009, 03:59 PM
Let's hope they also have the budget to integrate this into Maya!

royter
08-16-2009, 07:02 AM
they didn't have the budget this year :(

djx
08-16-2009, 02:07 PM
Yes, after watching Zap's siggraph master class vids, I was hopeful that we would get something like that - until I found out that 2010 was not really an upgrade. Reading between the lines, I'm optimistic that 2011 will come soon with a well integrated linear workflow implementation.

-- David

royter
08-20-2009, 07:31 PM
was the Fg multibounce + gamma correct node bug fixed in Maya 2010?

Bitter
08-24-2009, 08:04 PM
This appears to have become complicated for a process that shouldn't be. Part of this is due to the fact that everyone seems to be over compensating for something I don't even see as needing compensation.

Ignoring swatches. . .the workflow is amazingly simple.

DIFFUSE textures you are using need to be in linear space. Re-read that: linear space (0-1 range). For most of us with a compositing package that's easy enough to do. Open the DIFFUSE texture and write it out as linear, NOT sRGB.

Render as usual with a framebuffer set to Gamma 1.0 and an image set to 16-bit half float.

Open again in something that will apply the correct LUT you need based on your destination, be it film or web/TV. Web/TV is where the gamma 2.2 comes in. So this is where we will change it to sRGB colorspace (gamma 2.2 basically) Now you are viewing it as it will be seen/used for most of you. Write those images into that colorspace keeping in mind some other post package might change it again. You want that change to be near the end of the pipeline, work in linear float as long as you can while using a LUT to preview what it will look like.

sRGB -> linear -> sRGB

More detailed:

Diffuse texture -> linear -> render -> post correction/compositing -> destination colorspace

This doesn't need to happen to other textures that simply provide data, like transparency or bumps. Maya has a weird RGB transparency, in real life it's just a scalar. You don't need to convert it, or bumps, etc. Those provide data to the shader for an effect, not a color response.

If you follow this method and use correct physical shaders and lighting (quadratic falloff, correct scene scale, etc) then your scene is physically correct and renders as it should. Regardless of your particular satisfaction with the image, that is physically correct, anything else is your artistic license which honestly, is huge. The obsession with "correct" is just an obsession sometimes, but I digress.

This ignores swatches. To be honest, a swatch is a cheap preview and I can tell without correcting the gamma that it's what I want or not. Part of the problem is that after everyone messes with all the information. . .something is wrong.

Simplify.

The reason this seems to be strange is because renderers are not aware of your destination. If I render to film, I do NOT want sRGB. I have a different colorspace. Not having a built-in switch means I can do whatever I want with my materials. If I render to linear files with enough depth, I can repurpose my images to whatever their destination may be. TV, Film, etc.

Also, keep in mind that Maya's render preview does lack a preview LUT, hence the lens shader. Maya's preview is linear, not sRGB. You're better off rendering preview and using that float file in your images/tmp for your destination package. Yeah it's an extra step, but it will tell you what it really looks like.

royter
08-24-2009, 09:41 PM
This appears to have become complicated for a process that shouldn't be. Part of this is due to the fact that everyone seems to be over compensating for something I don't even see as needing compensation.

Ignoring swatches. . .the workflow is amazingly simple.

DIFFUSE textures you are using need to be in linear space. Re-read that: linear space (0-1 range). For most of us with a compositing package that's easy enough to do. Open the DIFFUSE texture and write it out as linear, NOT sRGB.

Render as usual with a framebuffer set to Gamma 1.0 and an image set to 16-bit half float.

Open again in something that will apply the correct LUT you need based on your destination, be it film or web/TV. Web/TV is where the gamma 2.2 comes in. So this is where we will change it to sRGB colorspace (gamma 2.2 basically) Now you are viewing it as it will be seen/used for most of you. Write those images into that colorspace keeping in mind some other post package might change it again. You want that change to be near the end of the pipeline, work in linear float as long as you can while using a LUT to preview what it will look like.

sRGB -> linear -> sRGB

More detailed:

Diffuse texture -> linear -> render -> post correction/compositing -> destination colorspace

This doesn't need to happen to other textures that simply provide data, like transparency or bumps. Maya has a weird RGB transparency, in real life it's just a scalar. You don't need to convert it, or bumps, etc. Those provide data to the shader for an effect, not a color response.

If you follow this method and use correct physical shaders and lighting (quadratic falloff, correct scene scale, etc) then your scene is physically correct and renders as it should. Regardless of your particular satisfaction with the image, that is physically correct, anything else is your artistic license which honestly, is huge. The obsession with "correct" is just an obsession sometimes, but I digress.

This ignores swatches. To be honest, a swatch is a cheap preview and I can tell without correcting the gamma that it's what I want or not. Part of the problem is that after everyone messes with all the information. . .something is wrong.

Simplify.

The reason this seems to be strange is because renderers are not aware of your destination. If I render to film, I do NOT want sRGB. I have a different colorspace. Not having a built-in switch means I can do whatever I want with my materials. If I render to linear files with enough depth, I can repurpose my images to whatever their destination may be. TV, Film, etc.

Also, keep in mind that Maya's render preview does lack a preview LUT, hence the lens shader. Maya's preview is linear, not sRGB. You're better off rendering preview and using that float file in your images/tmp for your destination package. Yeah it's an extra step, but it will tell you what it really looks like.

Your method works fine but what if i want to render an animation with a MR lense shader apllied to the renders and i don't want to tonemap in post?

It's clear for every one that Diffuse should always be gamma corrected.
bump and displacement shouldn't.
but what about those other values, do you know what should be gamma corrected
and what shouldn't?
reflec color
refrac color
color max distance
translucensy color
Diffuse weight
Diffuse roughness
reflectivity
glosiness
transparency
glosiness
translucensy weight
ambient shadow color
ambient light color
specular balance
cutout opacity
additional color

Bitter
08-24-2009, 11:46 PM
If you can achieve the desired look by using the lens shader, then your method doesn't matter. The method I have described is basically the simplest approach to creating a linear workflow.

The answer for what should be linear space is anything where there is a color perception from a texture for viewing. So if you are to view, say, your environment from the render and it is a texture, then it should be in linear space.

If you are mapping something to a scalar value like roughness etc, then that is not being perceived as a color for viewing, it simply alters the diffuse channel which should be in linear space already. Same with mattes, transparency, etc. Scalar values are usually used to alter something else that should be in linear.

Additional color is not considered a physical attribute. It breaks the physical correctness of the material for mia_x. Since it operates as an emitter I would say use whatever looks best there.

But honestly, whatever looks best is the approach you should take for anything.

Physical correctness and a linear workflow aren't the Holy Grail of CG.

Doing good work that gets you paid well is the Holy Grail. :-)

Villijs
09-01-2009, 09:15 AM
We recieved our maya 2010 (hoping for lot of new add-ons, fixes, proper gamma workflow like max 2010, corrected render passes system and everything else :D) this morning, and i did a few tests to see if there is any changes about this topic:

Texture - gamma uncorected; framebuffer 0.454; exposure 1.0;
http://img215.imageshack.us/img215/7213/text1framebuffer0454ton.jpg (http://img215.imageshack.us/i/text1framebuffer0454ton.jpg/) http://img215.imageshack.us/img215/text1framebuffer0454ton.jpg/1/w720.png (http://g.imageshack.us/img215/text1framebuffer0454ton.jpg/1/)

Texture gamma corrected (in photoshop in levels gamma 0.454); framebuffer 1.0; exposure 1.0
http://img215.imageshack.us/img215/3964/text0454framebuffer1ton.jpg (http://img215.imageshack.us/i/text0454framebuffer1ton.jpg/) http://img215.imageshack.us/img215/text0454framebuffer1ton.jpg/1/w720.png (http://g.imageshack.us/img215/text0454framebuffer1ton.jpg/1/)

With framebuffer set to 0.454, i noticed that it calculates FG with uncorrected images, and adds correction to it after... as well as it affects bump textures, seems it doesnt affect color swatches:
http://img27.imageshack.us/img27/7213/text1framebuffer0454ton.jpg


So to me it seems that framebuffer solution isn`t the best idea... it affects bump textures (so i assume that displacement, relfectivity etc. will be affected as well), and calculates FG from non-linear textures...
Correct me if i`m wrong or i`ve missed something! :)

jimjunky
09-07-2009, 10:14 AM
Hi all,

I thought this might be the best place to post my queries after finding no similar topics.

On the subject of linear workflow. Do any maya users know a way of tone mapping/filtering the viewport like max does?

Obviously once you have inversley gamma corrected your sRGB textures they appear incorrect in the viewport until you render them with the mia_exposure simple set to 2.2 gamma.

Also, I have a scene where I have camera mapped my backplate on to some geometry (projected an image with a projection node set to perspective) with an sRGB texture. This all lines up perfectly until I inversley gamma correct my image in the hypershade, with a gamma node.

Then the image no longer lines up correctly, as if the fit method has changed?? I literally have done nothing else apart from put a gamma node in betweeen the file node and the projection node.

Can anyone think why this might be?

Thank you for your time.

James


EDIT:

So If I put the gamma correction after the projection node it doesn't encounter this problem. Although this doesn't explain why this happens...?

mercuito
09-08-2009, 01:55 AM
Both these methods have their pros and cons, however they both require some setting up or gamma correction nodes... doesn't that just seem like a whole lot of hassle just for something that should be built into maya?

If you answered yes... you may wanna check this out :cool:
http://forums.cgsociety.org/showthread.php?f=7&t=803178

valler
10-19-2009, 02:04 AM
Hello everybody,

Now hopefully this won't bring in even more confusion, but I'm afraid it could.
I've been playing around a little bit, to find out which workflow would suit best my personal needs. So here's what I THINK i found out so far:

Fist of all maya/MR does not seem to assume 8-bit textures have a sRGB-profile embedded (because that's what i have often read on blogs and in forums).
Here's a test i did today:
http://img10.imageshack.us/img10/6462/uncorrected.jpg

http://img10.imageshack.us/img10/5122/linearlook.jpg

This is a rendering of 4 spheres with constant shaders applied to them. The second image was gamma-corrected.
From left to right:
0.5 rgb defined via color swatch
0.5 rgb sRGB texture
0.5 rgb ProPhoto RGB texture
0.5 rgb 32bit floating point texture
To me it looks like, it doesn't matter which bit-depth or profile your pictures have been saved out. All that counts is the actual RGB-values in those files.
As we know, they will be handled and displayed linearly from 0-1-space inside the renderer and the preview.

A handy little trick to find out your monitor's gamma inside PS is under...
Edit -> Color Settings...
choose your monitor-profile as your RGB-working space
click the RGB-working space again and choose "Custom RGB..." right at the top
the options will list your gamma (for mine it's 2.23)

I'm using the same method for degamming my textures. For example choose ProPhoto RGB as your working space and follow the steps above.
Now change the gamma from 1,8 to 1,0 and name the profile "linear ProPhoto RGB"
click the RGB-working space again and choose "save RGB..."
save it wherever your color profiles are being stored in your OS
Now i can convert all my images to linear space inside Photoshop and keep them as a 8-bit format (since maya/MR WON'T expect them to be sRGB) and adjust the rendered preview for a display-gamma of 2.23 - in my case. No need to convert to 32-bit or messing around with 1/2.2, or 1/1.8 maths and stuff...

And now here comes the important part. Let's say you want, for simplification, to render a neutral grey tone in linear space. The renderer shall do the math behind the scenes with what you usually see as neutral and also display it right afterwards (FG calculation and diffuse color outlined in this topic).
So heres my solution:
What means neutral grey? From photography i know it's defined as an 18% bright grey, isn't it?
And well, nature "renders in linear space" but our eyes see it in a non-linear way. That's my starting point ...actually this sentence was the finish line already.
The renderer does just the same thing. Rendering a neutral grey means calculating 50/100 luminance, which is 18% birghtness (HSB/HSV) or 0.18rgb in linear color space.
Create a blank image in PS and asign any standard profile to it
Fill it with Lab 50,0,0 (not 0.5rgb!!!)
Convert it to the corresponding linear profile and what you get is roghly about 0.18rgb (you can of course do the job with a gamma node and test the result with "getAttr.." and the shaders outColor).
So after degammaing the texture, we get just the same values, we would have had in nature. When rendering's done, for the preview we compensate for the monitor's gamma with the exposure node, or view the image in a color managed software. What we see will be a neutral grey tone and the math will have been done right under the hood. No need to apply any rayswitch nodes or anything in your pipeline.
In PS you would asign your monitor profile to the image and then convert to which ever profile you prefer (described here (http://www.artstorm.net/journal/2009/07/color-management-wide-gamut-dell-2408/)).

I hope everybody got my point?!

For my understanding, swatches like "diffuse weight" shouldn't be degammaed, since their RGB values act as multipliers to the individual channels of the corresponding attribute(s). Anyway I'm not always sure if a swatch should be considered as a actual color or a color's multiplier..

See attached images, FG bounce colour fades away. Shouldn't they all be identical as the colour remains consistent but the FG does not.
The only right solution is to map the colors back to the value they would have had in linear space. Don't do things by halves here.
Degamma the embedded gamma; render; compensate for monitor gamma if your watching the preview in maya.

I'm no expert at all, so pleeeease correct me if I'm wrong!

Best regards,


Valerij

djx
10-19-2009, 12:48 PM
Your basic logic seems good, but I tend to disagree with this part.
Now i can convert all my images to linear space inside Photoshop and keep them as a 8-bit format
At first it seems to make life easier, but you need to consider the limitations of the 8-bit file format and how they relate to the data being stored. One of the reasons sRGB is a good choice for 8-bit image storage is that it skews the way the data is stored so we get more steps in shadow detail and less in the brightest parts, which is similar to how we perceive changes in luminance. If you linearize your 8-bit data you risk introducing noticeable banding in the shadow areas.

In many cases your fileTextures will be used in a way that you will probably get away with it and never notice, and if it saves some messing around, well I guess its ok. Still worth keping in mind though.

-- David

coccosoids
10-19-2009, 12:50 PM
Hello everybody,

Now hopefully this won't bring in even more confusion, but I'm afraid it could...


Holy bejesus, :argh:


You lost me there...






So, can you please put all that into something a bit more schematic...



Let's say I have an interior render... and a bunch of textures taken straight out of google.

Where do I take it from there... do I gamma, degamma, ungamma, pregamma, postgamma them... and if I do any of those things, how do I setup the renderer - mental ray, in maya, afterwards? We use a the framebuffer gamma 'tehnique', we do it via a lens shader? Hoh...

valler
10-19-2009, 01:52 PM
Google your textures
degamma them (most likely sRGB ~2.2) in PS or use a gamma node in maya
eyeball colorswatches - or degamma them if you want to match a textures color accurately for example
choose openEXR; 4x32-bit rgba; framebuffer gamma 1.0 for the rendering
for preview use the simple exposure node with your monitors gamma (everything else zeroed out, gain = 1)
For the final render, set the exposure gamma to 1.0 and don't remove any gamma nodes!

I like to degamma the textures in PS, because i can batch process them and won't need gamma nodes in maya. Eyeballing colorswatches is fairly enough for my work.

So far I've never used the gamma option in the framebuffer. I don't even know when it will apply, that's one of the reasons I avoid going this way.

Your basic logic seems good, but I tend to disagree with this part.

At first it seems to make life easier, but you need to consider the limitations of the 8-bit file format and how they relate to the data being stored. One of the reasons sRGB is a good choice for 8-bit image storage is that it skews the way the data is stored so we get more steps in shadow detail and less in the brightest parts, which is similar to how we perceive changes in luminance. If you linearize your 8-bit data you risk introducing noticeable banding in the shadow areas.

In many cases your fileTextures will be used in a way that you will probably get away with it and never notice, and if it saves some messing around, well I guess its ok. Still worth keping in mind though.

-- David

Thanks for the hint David! But what's the difference if I convert to 32-bit? I still have 8-bit of color information only, in both cases evenly spaced on a linear curve. Banding could occur in the 32-bit file aswell then? The texture is only used for sampling, isn't it?. Since no new data will be stored in the file itself, you won't need the extra "room" between two values you gain compared to the 255 steps/channel(?)
Or could rounding cause the problem? Like degamming a 176 value would give you 54,7 and therefore 54 or 55 in 8-bit (using random numbers here..)? One could try to make two copies of a gradient. Converting one to linear 8-bit, and the other to 32-bit float, then converting both back to the original profile and compare all three of them.
Perhaps i should simply stick with openEXR instead of worrying :D


Valerij

FrizzleFry0
10-19-2009, 03:23 PM
If you can achieve the desired look by using the lens shader, then your method doesn't matter.......

But honestly, whatever looks best is the approach you should take for anything.

Physical correctness and a linear workflow aren't the Holy Grail of CG.

Doing good work that gets you paid well is the Holy Grail. :-)

This is worth quoting. Ultimately this is the main thing that matters. There are so many variables and ways to convert between sRGB and linear. When you render your final image out of Maya, Nuke, Shake, or whatever package you're using just make sure it's exactly what you and/or the client want.

The 2 main things to watch for are the black and white values. Using sRGB space give you a much smoother gradient of lower values, which can help out in shadow areas of your scene. Controlling your gamma of sRGB textures being applied to color values (diffuse color, reflection color, etc.) when used in conjunction with a lens shader will help minimize areas blowing out to 1 or over-saturating.

I rarely ever keep things at what is technically correct, like using a 2.2 gamma on a lens shader and then gamma correcting all applicable textures to .455. I adjust the gamma on the lens shader first to find a good spread of black to white values (which could be 1.6), then adjust the gamma on my textures to match it - in the case of 1.6 I would be using a .625. Just remember to use your own artistic judgement and not default to what people think should technically be happening.

djx
10-20-2009, 01:48 AM
Or could rounding cause the problem? Like degamming a 176 value would give you 54,7 and therefore 54 or 55 in 8-bit (using random numbers here..)?
Yes, that is the problem, and more than just the fractional rounding of one number. In the original image you had 176 steps in your luminance between black and this color, now in the linearized 8-bit data you only have 55 steps (or whatever it really is). Less steps = more banding, which could become quite noticeable depending on the nature of the image. A float format like exr would not have this problem.

-- David

valler
10-20-2009, 07:54 PM
Sure, if there's a faster, satisfying solution, I stick with that. But up to a certain level I like to be aware of the technical side just as well. In my experience it makes trouble-shooting much easier.

And ouch... this subject sometimes is so complicated that I start to overlook the simple things. 176 is more than 55 - Who would have thought :argh: Thanks David!

Thank you everybody, you helped a lot!

Skoczylas
10-26-2009, 03:04 AM
Sorry to bump this thread up again.
After reading hours about the linear workflow again I'm totally confused now.

So there are some advantages and disadvantages about Framebuffer Gamma correction and mia_exposure correction.
Anyway, if I choose the Framebuffer Gamma correction:
- Textures/Swatches: no correction
- Framebuffer: 0.454
- Mia_photographic: 1

Then please tell me what I need to do when batchrenderig. Do I just need to delete the Mia_photographic? :argh:

djx
10-26-2009, 07:05 AM
Anyway, if I choose the Framebuffer Gamma correction:
- Textures/Swatches: no correction
No, your color swatches will still need gamma correction.

Do I just need to delete the Mia_photographic?
The reason this thread is so long is that there is no simple answer for every situation. The answer for you maybe "yes", but it could just as likely be "no" with a few if's and maybe's.

Once you understand the underlying concepts (and you maybe do already) then you can easily set up a test for your pipeline to check that you got all the gammas in the right places, including where the fileTextures originate and what you do with the images after you render them and how you view your previews and final renders.

-- David

Skoczylas
10-27-2009, 07:10 PM
Ok I totally missunderstood the colorswatches thing...
I read your blogentry about Gamma workflow and the update youve written again.
I think my workflow when rendering is right if I keep my framebuffer at .454, degamma swatches and delete my Mia_photographic because I apply a gamma of 2.2 in my comp application. Please correct me if I'm wrong :)

djx
10-27-2009, 09:39 PM
That sounds good.

My only caution: With fb-gamma=0.455, you may notice some artifact issues if you use eliptical filtering on your bump map textures.

-- David

Skoczylas
10-27-2009, 11:36 PM
Thank you David. I know the artifacts issue, since its posted at your blog anywhere. But I realy never got any benefit when using eliptical filtering - it took more time and it didnt look any better than mipmap filtering :shrug:

leif3d
10-30-2009, 03:14 AM
I think my workflow when rendering is right if I keep my framebuffer at .454, degamma swatches and delete my Mia_photographic because I apply a gamma of 2.2 in my comp application. Please correct me if I'm wrong :)

Why would you want to degamma swatches if the framebuffer is already doing that for you?

I personally leave the framebuffer at 1 and degamma all color swatches and textures with a gamma correct node at .45, it's a lot more predictable and you can setup a script to do this for you.

djx
10-30-2009, 06:36 AM
Why would you want to degamma swatches if the framebuffer is already doing that for you
Framebuffer gamma does not change color swatches. Colors from the swatches will render different if you do not compensate for a change in framebuffer gamma.

-- David

CGTalk Moderation
10-30-2009, 06:36 AM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.