32bit and AA problems in Mental Ray

#4

i really want to do this job in post… but i can’t if i’ve aliasing problems
i’m making some test with 4x16 half

there’s no solution for the aliasing problem?

#5

this is the render with 16x4 half in framebuffer
i don’t see any difference with 32x4 :frowning:

#6

it’s quite late now (for me) but where are those aliasing probs?

p

ps I suggested to use EXR because the tonal range is wider aswell as space efficient.

#7

it’s late for me too :wink:
i’m using .exr in my renders (exept for the 8x4 bit render)

this is a 2x zoom

all the settings are the same for both images:

AA 0 -1
Mitchell 4 4
rgba threshold 0,05 0,05 0,05 0,05

if i use lanczos filter the problem is much more visible, the only filter that haven’t this problems is gauss

#8

a test with high AA values
exr without tonemap framebuffer 16x4 half

AA 1 3
Mitchell 4 4
rgba threshold 0,02 0,02 0,02 0,02

the aliasing doesn’t go away!
:cry:


and this is the 8x4 bit rendering with same settings
:shrug:


#9

What are the pixel values in those very bright areas? It looks like they’re probably very high and the filter isn’t able to cope.

That would explain why the 8-bit is fine, because the 8-bit pixel values are probably clamped before filtering.

You could try using different filters, perhaps a softening filter like gaussian with a wider filter radius. Of course that’ll end up over-softening your image which you probably don’t want either.

Of course the real problem is more subtle. You’ve created a scene that would be impossible to photograph with a camera or view with the human eye. In order to stop the bright areas over-exposing the whole image, a photographer would have to close the aperture on his camera so much that the room inside would become very dark.

Tone mapping is the obvious solution as this mirrors what you do in real life, but if mental ray maps the tones after it does the filtering, you’re still stuck with the same problem. In that case, you’ll just have to go back and relight the image to balance it out a bit better.

#10

you are right, using the gaussian filter is a solution
but i dont want to use it in my render

i agree, this is a hard scene for a photo… but, what i want is only use the 32bit (or 16bit) range for the post work…
scaling the gamma make the same work, but i dont like this solution and i’m just trying to take advantage of exr format, this is a great possibility in mental ray…
but, if i can’t control the alising in overexposed area i’ve a serious problem for my work

#11

I see aliasing in this picture.
But, how do you know you see aliasing in the original file?
What are you using to translate a 32x4 or 16x4 into 8x4 for us to view?

And what are you using to view it in post?

Are you using the clip option in the mitchell filter?

#12

the strange is that increasing the AA quality doesn’t make any effect in the alising problems :frowning:
sorry, what are the clip option?

for saving the images i’m using photoshop (and the exr plugin), same for colorcorrect (+ other plugins), but in the last ones i haven’t used any colorcorrection
in the exr that i see in photoshop i see this problem, but if i use hdr or tiff is the same… i dont think that’s an exr (or photoshop plugin) bug :frowning:

#13

the strange is that increasing the AA quality doesn’t make any effect in the alising problems :frowning:
sorry, what are the clip option?

the clip options control how color is clipped into valid range: you can give precedence to RGB or to alpha. mayatomr default is ‘raw’ (the both of the previouses are clipped independently).

for saving the images i’m using photoshop (and the exr plugin), same for colorcorrect (+ other plugins), but in the last ones i haven’t used any colorcorrection
in the exr that i see in photoshop i see this problem, but if i use hdr or tiff is the same… i dont think that’s an exr (or photoshop plugin) bug :frowning:

The point is that the .exr file on disk is not tonemapped (while what you see in the renderview IS!) hence we need to figure out if the problem is in how we visualize the floating point data in the range 0–>1 in our viewers (renderview, photoshop, imf_disp…) or if the aliasing is present in the file itself. More later.

p

#14

ok, now i making some test with batchrender / clip option in exr/tiff/hdr
i will post the render soon
if u want i can send you the source files (exr/tiff/hdr)

thanx :wink:

mat

#15

ok, nothing to do… hdr/tiff/exr it’s the same
i’ve used clip option and batchrender, i can’t solve this problem…
i can attach my scene or the source files if u want
(btw bart, u have already this scene if u want to test it :wink: )

#16

Ok Marcians, stop to talk in alien language and speak in a clear english way :smiley:

#17

OK. There’s a couple of things to try to clarify.

  1. When rendering to exr, the bit depth for any R,G, or B component is increased.
    When choosing a 4x8 rgba buffer, it means that 4 channels of R, G, B, and A are each 8 bits.
    When choosing exr, it typically means that 16bit floats are used for each component, rather than 8 bits. The OpenEXR format also supports 32bit float, and 32-bit integer formats. See www.openexr.com.

  2. Anway, a display device such as a computer monitor cannot typically display more than 8 bits per component, so a mapping must be done from the higher to the lower. Although it may have nothing to do with the aliasing, we are wondering if this translation is having an adverse effect.

  3. Notice that there is a separate framebuffer control in mental ray Render Globals. In the Framebuffer tab, look for the DataType under Primary Framebuffer. It appears you have to set this type higher besides setting the output to EXR to have an effect. To match typical OpenEXR output set it to RGBA (Half) 4x16 bit.

4)This is also where you set gamma. During interactive rendering, I’m having a hard time seeing its effect when the Framebuffer is set at higher than 4x8. When I export and render stanalone it seems to be fine.

#18

so: avoid relying on maya do the tonemapping and do it yourself in post. in mental ray renderglobals> preview:

  • turn off ‘preview convert tiles’
  • turn off ‘preview tonemap tiles’

(ah: note that if you turn off only the second you will have hot pixels for the areas over the range 0–>1, sort of a diagnostic mode if you want)

don’t care about the image in the renderview (which is not fp) and get the *_tmp.exr file that maya generates in your project images’ folder (or render with batchrender and get the *_tmp-less .exr file which is written). If you open it with imf_disp you’ll see it’s an 4x32bit (half) .exr

If you’ll open it in - say - photoshop you can tonemappit yourself and you have no aliasing.

Same if you render in standalone.
I can’t attch an archive bigger than 97 kb here… so get it from here: http://www.jupiter-jazz.com/temp/room.zip (it contains the .exr and a posttonemapped .jpg with no aliasing :slight_smile:

have fun!

p

ps when tonemapping in maya the aliasing though shouldn’t happen, you might log a bug through the proper channel: Alias.

#19

yeaaaaaaaaah!! now it’s working fine!! :applause:
thanx paolo, thanx bart :wink:

ciau :scream:

mat

#20

By the way, I’m getting decent results inside Maya in the Render View with the first Preview option turned off and the second one on, ie

  • turn off ‘preview convert tiles’
  • turn on ‘preview tonemap tiles’

Then, I can use gamma, tonemapping and RGBA (half) 4x16 bit.

But, it might just be the scene I’m using.
And I haven’t checked how the preview image compares to the actual file written out.

#21

i’m getting good results in the renderview too, with both the option turned off… don’t know… but i’m so happy now!! :slight_smile:

#22

very well!
BTW, I thought about starting a blog that allows me to post something very quickly, hence, guess what, I did it: http://blog.jupiter-jazz.com (in a rage of excitement I just added somethig about this issue… will be probably extended).
I will post here some TRIXR4KIDZ + random lollies&follies.
Of course bounded by the two magic words: “time allowing…”

p

#23

yes, ofcourse, BUT this is NOT a fp buffer, hence your data is already in 0—>1 range. you cannot RMB and save an EXR from the renderview: only regular formats here.

if you need to tweak your data in post rely on the rendered file on disk.

p