baking textures and antialiasing problems...

[steev]

still banging my head against this one...

in the maxscript help it says antiAliasing: <boolean> is available if the standard scanline renderer is being used.

if i change the render line in my function to...

render rendertype:#bakeSelected antiAliasing:false etc. etc...

i would assume this would solve the problem. but it doesn't. does anyone know if you specify #bakeselected as the rendertype it will not use the scanline renderer? if not, what renderer does it use?

[PiXeL_MoNKeY]

is that AA or lack of padding that you are seeing? By default Padding is enabled and set to 2.

-Eric

[steev]

i'm assuming it's AA. if i setup RTT padding to be higher than 2, there's more overlap past the UV bounds, but it's still aliased.

if in the script i set the line...

obj.INodeBakeProperties.nDilations = # (which i'm assuming correlates with padding)

...to be anything, i get the same antialiased results.

[PiXeL_MoNKeY]

Unpremultiply the rgb by the alpha. Then process the alpha and make any value not equal to black and make it white.

What render engine to you have enabled? Have you disabled AA in the renderer? That is all I can think of.

-Eric

[steev]

the rendere is default scanline. that antiAliasing:false line turns off the anitAliasing. if i call a standard render with that setting the edges come out aliased like RTT, so it works there, just not with rendertype:#bakeSelected

thanks for the input, tho.

[PiXeL_MoNKeY]

You could always try calling:scanlineRender.antiAliasing = falseTo force the scanline settings to be disabled. Make sure the render scene dialog is closed. According to the Macro_BakeTectures.mcr, which is what is used to RTT, it just uses #bakeselected. So really have no idea what is going on.

-Eric

[steev]

yep, should've put that in the first post. that line was one of my many shots in the dark. i just retried it in case i had the render dialog open previously, but no dice.

thanks for the suggestions. i do appreciate it.

[steev]

ok, so this is a longshot, but i'm running out of hair to pull out...

in RTT you specify the color of the "element background" & it renders the UV chunks against this color. this is how i'm getting the halo at the uv seams - the antialiased edges are picking up that color.

can anyone think of a way to force it to render against the original object texture?

btw - a friend suggested i try rendering out as a .png since .tga can be "buggy" (?) - i still get the antialised edges when run from the script. even with all the antialias switches set to false.

*grasping at straws*

[PiXeL_MoNKeY]

I have never had issues with TGA, as long as premultiply alpha is disabled. While PNG always saves with premultiplied alphas from 3ds max.

I'm really not sure what is going on with the script, have you tried reading through the Macro_BakeTextures.mcr file?

-Eric

[steev]

yep - read through it twice! it's BIG.

anyways, props go to my friend chris c. in NY for figuring this out for me. he's becoming quite the RTT guru since i last worked with him. anyways...

the heart of the problem lies in the fact i was rendering the bakeMap directly to a bitmap...

bitm = bitmap Rsize Rsize
render rendertype:#bakeSelected ... to:bitm

what this was doing was when RTT generates it's internal bitmap & then saves it as the outputfile (.tga) it's blending the alpha. however if i create a placeholder bitmap & copy the contents of the internal bitmap into it i retain the exact output of the bakeMap...


render rendertype:#bakeSelected vfb:off progressBar:true outputSize:[Rsize,Rsize]
rendPath = (maxfilepath + outputFile + ".tga")
tempBitmap = bitmap rsize rsize filename:rendPath
copy bakemap.bitmap tempBitmap
save tempBitmap


this created a new problem. if i changed anything in the script, like dilations or even tried to run the script on a different object, i would continue to get the same rendered result everytime. the bakemap.bitmap remains static during further script evaluations. there's no command to unload the bakemap.bitmip or delete it. after poking around a bit, i discovered good 'ol gc light:true works fine.

so, for posterity sake, here's the full script...


fn RTTit obj Rsize outputFile=
(
bakeMap = diffusemap()
bakeMap.outputSzX = Rsize
bakeMap.outputSzY = Rsize
bakeMap.fileType = ".tga"
bakeMap.filterOn = false
bakeMap.shadowsOn =off
bakeMap.lightingon = off
bakeMap.targetMapSlotName = "Diffuse"
bakeMap.enabled = true

obj.INodeBakeProperties.addBakeElement bakeMap
obj.INodeBakeProperties.bakeEnabled = on
obj.INodeBakeProperties.flags = 1
obj.INodeBakeProperties.bakeChannel = 1
obj.INodeBakeProperties.nDilations = 1

render rendertype:#bakeSelected vfb:off progressBar:true outputSize:[Rsize,Rsize]
rendPath = (maxfilepath + outputFile + ".tga")
tempBitmap = bitmap rsize rsize filename:rendPath
copy bakemap.bitmap tempBitmap
save tempBitmap
obj.iNodeBakeProperties.removeAllBakeElements()
gc light:true -- must call this to clear the bakeMap.bitmap otherwise the first render will always get pasted in the tempBitmap
)


RTTit $ 1024 "test"


again thanks chris & pixelMonkey for your help!

[martinB]

Sorry, can't help you with the antialiasing question, but I have a question about your script:
Does that actually bake out the diffuse element? Here, it seems to bake out the complete map, i.e. including all shading, shadows etc.

-- MartinB

[steev]

hi martin,

interesting that you're seeing a complete map. here & in the studio it's rendering just the diffuse as expected. are you using an earlier max version than 2009? i seem to recall something while i was trolling through the docs & the .mcr that something was 2008-09 only, but i'm not sure what & i don't think it's in the script.

if the vfb property was set to true, you would definitely see the complete map, but the rendered .tga in photoshop is showing me just the diffuse.

could you create a simple sphere in an empty scene with one light and let me know what you get in photoshop?

thx

[martinB]

I created a simple sphere, and an omni light source, switched from mr to scanline.

Selected the sphere and ran your original code in 3ds Max 2009 64bit. I get a shaded result of the sphere's color, like what I would expect from a completemap output.

The second version of the function gave me the expected result, a solid color. :)

-- MartinB

[CGTalk Moderation]

This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.