View Full Version : Alpha Shadow Matte is flickering!?

07 July 2009, 01:32 PM
I'm trying to render out a shadow matte of all the rocks i have on the floor of my scene.
I have setup my shadow matte material as follows, and I have GI on with an FG Map calculating the FG.

mib_color (factor at 0) connected to background of a mip_matteshadow, which then connects to the MR material and shadow shader slots of a Lambert.

The flickering only seems to happen on the smaller rocks in the scene..

Here's a preview of the scene
Link to image (

Anyone got any ideas how to solve this?

The problem still persists if I turn off FG.

07 July 2009, 03:52 PM
Are you using depth map shadows?

07 July 2009, 03:57 PM
Nope, I have 4 lights in the scene, only one of them is producing shadows and they are ray traced with the following settings:

Light Radius: 0 (sharp)
Rays: 32
Depth Limit: 5

It only seems to affect objects that are small in size (the small pebbles) - when I scale one up and render again, the shadow does not flicker. I really don't get it.

One thing I should also mention, I am using a physical sky too. However the directional light attached to that does not use any of the shadow attrib so not sure what the shadow contribution from the physical sky is doing.

Do I need to perhaps increase the Shadow Softness or Samples (currently at 12) on the Physical Sun?

07 July 2009, 04:07 PM
Hmm, sub-sampling settings? I'm trying to think what could cause the small rocks to 'slip through' detection. The vimeo video is very low res and there are loads of small rocks, many of which are not appearing at all, have you tried a single frame at a much higher res? It could be that many are just too sub-pixel.

07 July 2009, 04:16 PM
It's an odd one.. even at high res 720p its doing it, literally frame to frame the shadow on a pebble will appear and then completely vanish only to appear again.

Sampling is set to 0,2 with 0.030 anti-aliasing using the Lanczos filter.
Jitter and Sample Lock are on.

07 July 2009, 05:42 PM
Ok I have narrowed it down to the physical sky.. when I remove it, the shadows are fine and in fact some pebbles that previously were not even displaying their shadows are now. So next thing is finding out what setting/attribute in the physical sky is causing this issue.

07 July 2009, 05:56 PM
Ahh, must be the sample number in the physicalSun light, never noticed that effect before. Strange how removing the FG didn't affect it though...

07 July 2009, 01:13 AM
well, I'm still getting issues with the flicker I mentioned before. I have included an example scene, if you render frame 1, look at the alpha you will see every sphere has a shadow. Then run forward to frame 5 or 6 and render again - the shadows of the rear spheres disappear.. then scroll on to frame 10 and you'll see all shadows are back. This therefore is causing shadow flicker in the animation.

I have a feeling this has something to do with the angle of view (which i cant change), but that still doesnt explain how the shadows are there in one frame and not the next.

Any help would be much appreciated.

File is for maya 2009.
Scene (

07 July 2009, 12:02 PM
I've got to admit mate, I'm totally stumped. I checked everything I could think of, systematically turned up the relevant settings, and tried different techniques - without changing your existing setup too much. All to no difference, in fact that was quite weird, I doubled the sampling rate, shadow rays and output res - no difference.

I've used the mip_matteshadow shader a lot and had no problem - although it's always been with larger objects and usually HDRI lit.

07 July 2009, 07:26 PM
Yup...been driving me mad Cam.. Anyone else care to take a look? Perhaps I should send this to Autodesk? Is DJX around? :P

I'm wondering if this is still a MR filtering thing.. I decided to try the same scene with maya software render and if i put the edge anti-aliasing to high and put the production quality to high, there is no flicker. This is very odd.

07 July 2009, 12:50 PM
vfx: I had a look at your scene. I think that the problem is due to sub-pixel sampling. I don't know exactly how it works, but using Adaptive Sampling with min=0, max=2 means only 1 sample per pixel in low contrast pixels, which may mean the whole image since the rgb is all black or white. So the small shadows might get missed depending on where they are in relation to a pixel. If you increase the Min Sample Level to 1 or 2 you will see those small shadows again.

Its also worth mentioning (because I probably influenced you here) that setting the Ambient Color to 10000 in mip_matteshadows is working against you and lightening the shadows. Set it back to 0 for daker shadows. That extreme value is something I use with physical sky when I am catching shadows and ambient occlusion. I have found that the Amb Color needs to approximate the brightness of the scene so the shadow color works as expected if set to anything lighter than 0,0,0

-- David

07 July 2009, 01:19 PM
Hey DJX - I knew you'd be able to answer this one :P Really appreciate your help in the shader/render scene.. and as usual, you have an answer! I've just tested your results and can confirm that the min sample value was indeed the issue. Thanks for breaking that down for me, wasn't something I would have though of.

I understand that the AO is better set to 0 when rendering just shadows, I just put that scene together real quick to test and left it at 1000 as I was using the preset from my actual scene which does have ao too :)

I'll render out a test and play it back.
Thanks mate! Really appreciate it, was driving me nuts.

Test completed, seems to be working perfectly! Thanks DJX!

07 July 2009, 04:14 PM
*Post deleted

CGTalk Moderation
07 July 2009, 04:14 PM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.