View Full Version : really bad quality rendering problems!

08 August 2010, 05:59 PM
so I think the time of trying to get it running by myself is over. I spent weeks, but I have no luck.
Something seems to be totally wrong with my scene, or my settings or whatever. And I cant figure out what it is. I even tried so much, I cant imagine anything left to do..

I have a pretty little scene . Indoor. A physical sky with a directional sun. One portalLight (just one window) and one point Light (at the other end of the room to lighten it up a bit).
I used Importons and Irradiance Particles and saved it to a file.
So far so good.
My problem starts here.
When I render the image, it takes A LOT of time to render. I think mostly because of the portalLight.
But even with a crazy amount of samples (AA 0/2 protalLight 24/24) its noisy and in an animation it jiggles and wiggles everywhere.
I've never seen something like this. I never had to use more than AA 0/2 and portalLight 24/24 seems to be REALLY A LOT samples.
the jiggling doesnt even really stop with AA 2/2 but I cant afford rendering at those settings.
So I split up my rendering into layers.
I used the rayswitch shader to make a Reflection/Refraction Layer and Illumination Layer (basically just the Light information) and the DiffuseColor (without any Light) and I realized that I have to render the DiffuseColor Layer (just the Textures assigned to a surfaceShader) at AA 2/2 everything below that Jiggles like hell.. I tried everything.. I tried blurring all the textures (File -> Effects -> Filter) [the preFilter doesnt seem to do anything in MR ?? ] but no success.
when rendering the DiffusColorLayer with AA 0/2 and 0.07 contrast the textures are jiggling really bad.
Then I tried the Illumination Layer.. Just the Light Setup with a basic grey lambert, to catch the Light info.
To get a decent speed/quality I tried AA 0/2 contrast 0.07 and portalLight 24/24.. the portal Light is STILL noisy. but it doesnt flicker in the animation! I could work with that.
But the worst part is the Reflection Layer.
EVERYTHING jiggles.. I first I tried to reduce the portal Light samples to get more speed. but it is a huge mess. the quality is real bad and even with AA 0/2 contrast 0.05 it looks more like a preview rendering.
I tried AA 2/2 and still some white reflections spots appear and disappear in the next frame.. everything seems beeing totally noise, even when I turn of the shadows of the lights.
And its not the blurry reflection samples - I tried 128 samples and its still noisy and jiggly.
I cant think of anything anymore, that could cause such long rendering time and such bad quality..

For Full HD it takes my QuadCore 2.4Ghz 6Gb Ram machine 4hours on a frame with AA 0/2 contrast 0.05 Mitchel 4/4 portal Light 24/24 and its still jiggles f***ing to much.

I would be really happy if someone could help me think! I am trying now for weeks and I am getting sick and tired of this..
I am trying to make an animation and I cant render more than 4hours per frame.. :(

08 August 2010, 06:00 PM
thanks in advance!

08 August 2010, 10:40 PM
Seems like the Irridiance Particles and Importons are the culprits . I havent used them much in animation but I can tell you that the settings are sensitive and can cause long rendertimes even in a simple scene setup .
Did u try using FinalGather ? .
Another thing would be that just using a default Maya lights and rendering out a couple of frames to confirm it might be doing with the GI settings .

If u can share some images , It would be easier for artists here to understand the problem . Goodluck

08 August 2010, 09:24 AM
hej Hasnz, thanks for your reply.
so yeah. I also tried the GI of course and turned it off. anyhow after baking the Irradiance Particle Map the actual rendering isnt much slower.
To clear things up I am rendering just a camera fly through and not a object animated animation.
So I saved the IP map to a file. this shouldnt jiggle at all, which is kinda cool.

Somehow Ive got the feeling that the textures are the problem. whenever there is a texture in the reflection, it is jiggling a lot!
I even started some test to see how MR is sampling the textures and it seems that, when laying a texture with a lot of fine detail on a sphere. the edge of the sphere gets jiggly when rotating witch a camera around.
even with preFiltering nothing changes.
Does preFiltering do anything in MR?
quadratic, box, mipmap .. all the same result.. this seems odd? doenst it?

I try making some images and movies to see the problem..

08 August 2010, 10:11 AM
I haven't experience with importons but what you have to do is breakdown your scene and find out what is causing the flickering.

Apply a standard material to everything, render, does it flicker.

If it does you know to start looking at your lighting, if it doesn't, you look at your textures/reflections/sampling/texture filtering etc.

I also noticed you mentioned there is flickering in reflections.

When using area lights you set a high samples, high samples limit and low samples.

If your high sample limit is 1, that means only eye rays, as in directly seeing your shadow, will use the high samples of 24,

after that ie, when you see that shadow through a reflection, it will then use the low sample settings. Maybe this may be what is causing your flickering in reflections.

You also have a similar setting for things like FG and such as to how many reflections/refractions they are seen through at the higher samples.

But I would troubleshoot your scene and figure out what is flickering, then people can help you a lot easier as to how to solve the problem.

08 August 2010, 10:23 AM
thanks irwit.
but this is everything I already tried..
so I made a test scene.
and this is the result I was talking about..
this happens to every texture. even when it is something that couldnt cause basic moiré..
the settings for this test rendering were:
AA 0/2 contrast 0.025
Triangle 2 2
hard raytraced shadows without smoothing.. (

and here is the scene

test.mb (


08 August 2010, 10:27 AM
Are you using 2011 ?

Had a look at the movie now and I see what you mean.

08 August 2010, 10:28 AM
no, good old 2009.. this problem is new to me!

08 August 2010, 10:31 AM
Just had a quick look, now give me a slap if Im teaching you to suck eggs but theres a couple things I noticed straight away, your contrast threshold was set quite high in sampling, I would also recommend turning off sample lock and turning on jitter for all renders. I have time to test render but maybe try this and see if it helps ?

08 August 2010, 10:36 AM
hej. thank you for your suggestions. but I even tried this already..

in the manual it says that when turning on "sample lock" it reduces flickering with a sample pattern !! ?
yeh I usually turn on "jitter", but forgot it for this test..
anyhow tried it. same result
for the contrast threshold mine is at 0.025. the lowest I can go is 0.01.. so why is this too high?

08 August 2010, 10:43 AM
Well setting it to 0.025 means that if a neighbouring pixel is within 2.5% value of the next it will stop sampling and treat it as correct.

I'll do a couple of tests if I get chance. Maybe its that the texture isn't being filtered through reflections and that could be something to look into?

08 August 2010, 10:54 AM
thank you for your help irwit.

i simply cant get the contrast below 0.01 and even when I set AA to 2/2 (so no contrast threshold is needed) the flickering stays..

the reflection texture flitering sounds promising. but where can I change the settings for this?
I used the File -> Effect -> Filter, but even when the texture is really blurred , a little bit of this flickering remains.

08 August 2010, 10:57 AM
You are right about the sampling at 2/2 so I guess sampling may not be the cause. Just for kicks could you try 3/3.

When I render headlights and stuff, basically very high contrast areas of an image, I would normally use a slightly higher aa setting to avoid slight flickers and jaggies. I know its a brute force solution that you shouldnt need to do but would be interesting to see if it helped.

08 August 2010, 11:05 AM
ok AA 3/3 is making it better, but its still there..
I mean that is 64 samples a pixel! and it is still there.. that cant be right.. ??!!

I wrote a quick script for rendering little animation in the viewport.. just in case you havent one of those already..

int $startInt = 0;
int $endInt = 10;
for($x = $startInt; $x <= $endInt ;$x++){
currentTime $x;
renderWindowRenderRegion renderView;
renderWindowMenuCommand keepImageInRenderView renderView;

08 August 2010, 11:15 AM
Yeah thats crazy,

I really dont think that the texture is being filtered through your objects reflections. Maybe as an performance optimization.

Could you try making your checker map in photoshop and rendering that on your object instead of the procedural, as you get more texture filtering options for the texture node.

Also, on the procedural checker, maybe trying setting the Filter offset to something low like 0.03. This may just take that harshness out of what is reflecting back.

Ive had a search through about texture filtering through reflections but haven't found much. Maybe its worth asking in the mental ray forums too. Hopefully Master Zapp may pop in and le us know what the score is.

08 August 2010, 11:26 AM
I tried what you said, but when I try the Filter Offset the whole texture is going to be really blurred..
so when you are rendering it, the problem is the same?
so its not my machine?

08 August 2010, 02:50 PM
Sorry Philip, I didn't get chance to test render due to work. If I get chance this weekend I'll set off a quick test render to see but Ive only got 2011 at home so not sure if it will be a fair test. Worth a try though.

08 August 2010, 08:06 PM
have u tried increased the aa sampling?
i tried it with 2011 AA @1/3 and 0.01contrast (0.02 might works too)
ive attached the .mb file and a short clips.
did it look better on flickering?

08 August 2010, 09:12 PM
looking at the example ; It would be a good idea to take the reflection as a seperate pass ; and blur it a little in post .

As irwit suggested ; a texture might also help over a procedural texture with the filtering support .

Another option that comes in my mind would be trying the Sampling override per object ; Like applying a 2-3 sampling limit on the reflective spheres . That way you dont need to oversample the entire image .

08 August 2010, 10:25 PM
this is interesting. it seems that this is a maya 2009 issue.. after trying the same scene in maya 2011, with exactly the same settings, the problem is gone.

thanks for all your great help!

08 August 2010, 03:57 PM
ok. the problem is still there.. it is a lot better, but the reflections are still flickering on the edge.
the thing is, when tracing only one reflection ray on an area in which is a high contrast sometimes the ray gets a bright and sometimes a dark value, so you basically have to oversample a lot!
but com'on its not 1990 anymore. why cant there be a edge driven reflection oversampling?
this problem cant be new..

I rendered my scene with only one material, no textures and no GI.
so it has to be a contrast reflection problem.

I summon master Zap, what shall we do!

08 August 2010, 04:39 PM
Have you tried the elliptical filtering?

08 August 2010, 04:45 PM
hej zerogee, thanks for your help, but I dont use any textures right now and its still flickering. as I mentioned. I think it is the high contrast reflections which need oversampling. but I never thought that much!

08 August 2010, 04:59 PM
Heres a quick 25 frame render I did . It looks pretty smooth imo . I think the setting can be lowered a bit . Maya 2009 .
0-2 antialising
.05 contrast
Lanczos filter
Default raytracing .
2-3 antialiasing override on the reflective spheres .
FG on with physical sun .

08 August 2010, 06:09 PM
hej Hasnz
its a nice test. and the two sphere at the bottom are pretty nice. only the upper one ist still a bit flickery.
anyhow. you are oversampling with 2/3 on the spheres, which is to much for my scene.
with 2/3 on every object on which the flickering appears I would render over 8h per frame.

we've got all this importance sampling and when it gets to the basic stuff, I have to oversample even the eyerays with over 64samples per pixel to get a "pretty ok" result? I think there has to be a better solution?!!

thanks anyhow..

08 August 2010, 06:17 PM
as now its just the reflection is flickering.
u might render seperate reflection pass. it will save u alot of time on rendering this kind of image.

08 August 2010, 07:25 PM
This high settings have been a rare case in my experience . And Breaking down the scene has always helped to solve certain problems . Either separating by objects or render passes. That will be the practical approach .
About the flickering on the top sphere , I think that would be being too picky :) ; Hardly noticeable to a common eye .But i would still expect such flickering there as there is a hi frequency bump map which is reflective a very fine tiled checker pattern .
If you can pm me a still of the actual problem frame ; maybe Iwould be able to give a better suggestion . Still render passes would be the way to go . Good luck .

CGTalk Moderation
08 August 2010, 07:25 PM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.