premult vs non-premul and fringe

#1

I’m having issues related to compositing premultiplied CG images in Combustion–I get a black fringe. I’ve tried to be systematic in identifying where the problem is, and I’ve included the results of my attempts below. Any thoughts on precisely how to make this problem go away would be greatly appreciated.

I’m rendering my images in Maya 6.5, using mental ray. There are four spotlights in the scene, equally spaced around a cone.

I rendered images using just one lighe at a time. The images are all rendered using mental ray’s production settings, and as 16-bit+alpha uncompressed tiffs.

I rendered a set of four images premultiplied (below top) and not premultiplied (below bottom).

I brought the images into Combustion 4.0.2 in two separate workspace files. In each file, the bottom layer has a transfer mode of Normal, while layers 2, 3, and 4 have Screen transfer mode. The footage of each of the layers in the premultiplied version has the “Premultiplied with” flag checked, and the associated color is black. The non-premultiplied version has the flag unchecked.

When I add a background image in the same composite node as the cone images, there is no fringe. When I instead add the same background image as a layer below a nested group of the other images, the fringe appears (as below).

All of the associated files are here:
renders_and_compositing_files

I need to render various nested layers over each other. Any thoughts on how to defeat the fringe would be much appreciated. Thanks very much.
-Paul

#2

The problem is being caused by the Screen Transfer mode. It is actually expanding the alpha information some how. There is a partial fix I found, its not 100% but it is closer to what you want. In the Straight (un-premultiplied) composite add a Set Matte Operator on top of the “Composite - Nested Layers” node. Set this to the alpha channel of any of the original Test images. I would also file a bug report with Autodesk on this so it can be corrected in a future version, which may have to be done through your reseller since I don’t see a link on the web page.

To verify this add an Discreet Keyer to each test layer image. Set the output to Matte (this will replace the data with the alpha data). Now Duplicate any layer, set its transfer mode Difference. The white edge indicates a change in the alpha data. Now set the top screen layer to normal and the white edge goes away.

Further testing reveals that Add, Multiply, Screen, Overlay, Soft Light, Hard Light, Negative, Color Dodge, and Color Burn all display some sort of indication of differences (tested using the bottom layer set to Normal, middle 2 layers off, top layer set to the various transfer modes, and duplicated layer set to Difference).

Hope this helps,
-Eric

#3

Eric,
Thanks very much for looking at this. Tomorrow I’ll take a shot at implementing your approach.

From your post, it sounds like if I were to do the same thing in, say, Shake, I wouldn’t be seeing the fringe. Is that right?

Again, thanks very, very much–I really appreciate it.
-Paul

#4

Combustion is the only compositor I have so I can’t really confirm if the results would be any different in another app. However, if they are correct in Shake or AE then be sure to report that to Autodesk as well.

Edit: Some further testing this morning I found that you can totally remove the problem by adding an Alpha Arithmetic Operator (Operator:set, Operand:100%, Blend:100%) to each fooatge layer each , this will remove all Alpha data. Now add a Set Matte on top of the Nested Layers Node set to the alpha of any of the original footage layer. This will force it to work properly, because there is no alpha data for the transfer modes to alter.

-Eric

#5

Eric,
Thanks again for your help.

All,
By way of closure to my original question, I’ve decided to just skip the nesting and everything works fine. My reason for using the nesting was to try to organize/simplify a complex set of layers/passes. The work-arounds are good to know about (and thanks very much for them!), but in this case they would just re-introduce complexity I was hoping to avoid.

I’m really surprised that I would encounter this issue in Combustion. I don’t think that my use of nesting was unusual or unorthodox, so I’m a bit baffled why such a basic tool would seem not to work. Eric’s discoveries notwithstanding, maybe there’s still some sort of “operator malfunction” on my part.

If anyone’s able to comment on my attempted usage (misusage?) of the nesting feature I would welcome the opportunity to learn something new from this.

Thanks again,
Paul

#6

Actually you really should be using the work around if you are using nested pass comps in any application.

The proper process would be for all layers to be unmultiplied. Or use the:

Composite/MergeSet Matte (FootageA Alpha)

Composite/merge[indent]LayerA:

[Operators]
Alpha Arithmetic (Set: 100%) - [Alternatively this could be an Alpha Divide Operator if you find one]
FootageA (Color + Alpha and Premultiplied)

[/indent]Otherwise you will always get fringes, and it wouldn’t be a bug, the math would be working out correctly.Oh btw… I’ve also had some problems with TGAs and Combustion in the past. Another avenue to investigate…

#7

This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.