View Full Version : How to create this effect in Maya?

07 July 2009, 09:57 PM
I've just checked Gnomon DVDs and see this new DVD is really interesting. This is the sample video of it.

Please watch the sample video. And this is the result of that.

For the first look, I guess he used AO + Fractal to get effect. I tried and this is what I got.

If you know how to archive this effect, please share or may give me some clues. Cause from the look of the the result (Gnomon DVD). I can use this effect for Water Color shader or I can use it to apply to the corners of hard surface model like rusted metal.

07 July 2009, 12:10 AM
in Maya that would be called contour rendering
There is a built in shader to handle that effect. You can do any combination of shape outline, intersecting lines or angle and lighting thresholds as well.

07 July 2009, 03:25 AM
Thank you. But when the camera changes the shape will change too. I want it to stay solid, no changing. I like the shader above because I think I could you it as a mask so I can create rusted iron.

07 July 2009, 09:00 AM
That's not a contour, it is an occlussion solution based on surface curvature. I think I saw something similar in long time ago and remember it was pretty slow then.

But it would be cool to make it in MAYA (it can be done for sure).

07 July 2009, 10:15 AM
I'm pretty sure that you can do that with Maya Toonshader. Check the examples in Visor and see if you can tweak your shader from there....

Otherwise you could look at mental rays round corners (or edges) shader.

Maybe you get mr Minning to share his minBevel shader.... minShaders (

07 July 2009, 04:39 PM
@DJ_: thank you for remind me the mia_roundcorners. I will try to utilize that node to see what I can come up with.

07 July 2009, 06:06 PM
i really doubt round corners will work as it only perturbs the normal, and what you want is to create a texture to use as a mask for a layered shader.

Toonshader is a post effect, which means you wont be able to bake it to a texture.

07 July 2009, 06:52 PM
i'd like to know this trick too! :)

but you can always use max to generate those textures or you can use ambient occlusion texture after you bake it from objects... it's not the same but it will give you something similar without noise... maybe you can make some nose or overlay on that texture..
i know it's not the same but im sure it will work!

let me know if you find the way! ;)


07 July 2009, 06:57 PM
looks like something called dirtmap.
dont' know how to do it in maya, but i don't think it's too hard if you really get your hands on it.

07 July 2009, 07:54 PM
before I read that it was AO with noise, I was thinking: it's AO with a lot of contrast and it's pushing even the light greys to black. I'm in the middle of a project right now so I don't have time to try myself though.

07 July 2009, 08:55 PM
1. hard edge selection script
2. vertex color on the selection
3. combine the vertex map with some noise
4. clamp the threshold for some hard falloff

but why not paint the map - you have much more control

07 July 2009, 11:38 PM
he says in the video he is using vertex colors

07 July 2009, 02:55 PM
shadingwise could be achieved with the tc_curvature and ambient occlusion node and some fractal.

also there's one shader which already does this from Binary Alchemy, but that's hard to control and has massive rendertimes.

i think handwork or swags workflow should work best.

07 July 2009, 08:53 PM
i am still curious if anyone can get this effect in maya.

Because it uses Vertex colors, it is hand painted, whereas in max it seems to be fairly simple.

07 July 2009, 04:08 PM
@swag: why not paint the map? Thinking about laying the UV of couple thousand pieces of polygons drives me nuts. In some case, 3Dsmax users don't have to layout the UV. That's why I wonder if I could archive something like that in Maya. By the way, I wonder 4 steps you mentioned above work with Maya? One of my friends are working on a script (Maya) so that I can select corner vertexes just one click and control the angle with a slider.

@Sorath: I don't think AO node will do the trick. I tested a scene that there is a cube and a sphere close together. When the surface of the cube close to the sphere, there will be a circle appear on that surface. I only want my fractual noise only appear on hard-corners of the object.

@ovspianist: you don't think it's hard if I get my hands on it? Well, I did try but I failed. I hope you will try and give me a good solution. There is something look too simple to me but it's not really that simple.

07 July 2009, 07:07 AM
Vertex Colors are the easy part, getting an attractive texture blend using vertex colors is slightly more challenging.

I was able to get something by using a layered texture with a noise map and the cpvTextureNod

07 July 2009, 04:13 PM
check out this post from the texturing forum:

i havent had time to test it out myself yet, but it looks very promising. its not vertex colors, but it may be a better alternative.

08 August 2009, 08:25 AM
Yeah, i knew of that method...Occlusion maps behave just like any texture really, so you can use it to blend between 2 different shaders.

What we need is something similar to minShaders that have a reverse AO or similar. I am just waiting for him to start selling those.

Where occlusion based shaders fail is when objects are animated.

08 August 2009, 10:08 PM
Where occlusion based shaders fail is when objects are animated.

True. But for something like this I would probably render to texture and go from there.

08 August 2009, 09:15 AM
As far as I understand he applies vertex colors to the hard edges, plugs the vertex colors into a warp node an distorts it with a noise. The result can be used to blend different shaders or textures.

So how would you distort the vertex colors in Maya? Is there something similar to the 3DSMax warp node?

08 August 2009, 11:38 AM

How to:

create cpv node and add a noise texture to the color gain attribute.

cpvNode behaves just like a texture node, so you can use it to blend other textures however you want.

08 August 2009, 12:35 PM
I worked with it a bit more:

I create 2 cpvNodes. One is flat black, and the other the one i paint white on the verts. I use a layered texture and "add" blend the white cpv node on top of the black one.

From there, I use noise textures on the color gain.

The HSV node is used as a contrast/clamp to sharpen up the entire network.

08 August 2009, 03:28 PM
Ah, thanks Mason, that's working pretty well. I added a couple of contrast nodes to adjust the noises and a reverse node at the end to switch the colors.

08 August 2009, 03:29 PM
@Mason: Thank you so much, Mason. I am on vacation right now so I don't have Maya to play with it. But It is exactly what I have been looking for so far :D your method will save us lots of time of texturing.

08 August 2009, 07:21 PM
Looks awesome! I'll have to keep this one in my pocket.

08 August 2009, 08:15 PM
that's nice. Now can that be baked to texture for a PS overlay? (not at home, couldn't help but ask)

Great job :beer:

08 August 2009, 06:51 AM
So what exactly do you guys use this for? To overlay textures such as rust or weathering?

08 August 2009, 08:32 AM
You can bake it of course, OR connect the result to a layered shader in order to blend between a Rust and Chrome metal.

For application, you really need to see the gnomon vid posted.

08 August 2009, 10:31 PM
How do you apply the vertexcolor Node to the polygon shape node??

08 August 2009, 10:26 AM
check the help files, its all there. You need the connection editor, and it is from shape node--> cpvNode

08 August 2009, 05:28 PM
so I finally got it to work just like the example above.. but I think this method is not any faster or better than making your own masks in PS because you still have to paint in the vertices, and if your geometry is too complex (lots of edges and vertices) you don't have enough control or accuracy. is good for boxes but I don't see any other application.. or maybe am wrong?

08 August 2009, 10:43 PM
It provides a good starting point to continue in Photoshop. Also, this method works acrossed UV seams or without UVs if you use 3d textures. Painting vertex colors can be pretty fast if you want to save the guess work you go through in photoshop.

This method also works well if you want to blend procedural shaders without painting texture maps.

Consider you may have an object that requires 20-30 4k texture maps. Optimizing your workflow with such methods can speed things up greatly.

12 December 2009, 10:01 PM
so I finally got it to work just like the example above.. but I think this method is not any faster or better than making your own masks in PS because you still have to paint in the vertices, and if your geometry is too complex (lots of edges and vertices) you don't have enough control or accuracy. is good for boxes but I don't see any other application.. or maybe am wrong?That's exactly when you'd write a script that selects the edges (based on the normal direction of the related polygons) and paints them for you. Then this is a HUGE time saver.

12 December 2009, 03:09 AM
I'm sorry if I'm gonna repeat what someone already said, didn't read all of it.
Lately I've been working on a project that was inspired by the same tutorial. I'm trying to do similar thing in Maya. Basically I want to texture bunch of highpoly objects without unwrapping them. There is many things happening on convex corners of objects (scratches, wiped dust etc) so I had to figure out how to get that effect as well.
AO seemed perfect to begin with but it only gives you mask of convex corners. Someone recommended me to flip polygons, generate maps and then use the maps as a mask but that would require 2 steps work that I wasn't happy with. Eventually what I am doing is:
1) setting up soft/hard edges the way that the corners I want to be masked out are hard.
2) Selecting all the hard edges
3) converting selection to verts, filing it white. inverting selection, filing the verts black.
4) running really quick to the object and paint black the edges I would like to skip, paint white the ones that has been missed.
5) in hypershader I read vertex color information with a mental ray node (it's in texture tab, forgot its name)
6) later I just use it as a mask, combine with noise, bring up contrast etc, whatever suits the needs.

On top of that I'm using blend box texturing method as in the mentioned tutorial (about 10 hypershade nodes in Maya) that doesn't require unwrapping and gives you decent general textures (can't have too many details though as blending would be too visible).
Add dust run by snow node, some dirt masked by AO, perhaps some nice reflection and you got yourself really nicely textured as complicated model as you can think of in less then a day. I'm not done with the project but I'm really happy with results I've been getting so far.
Please let me know if you need details, I'd be happy to help if I can.

good luck!

12 December 2009, 07:07 AM
Caro, it would be great if you could post an example of how your object looks when textured! :D

01 January 2010, 12:59 PM
Is possible tu use, inverse ao, usign the binary Alchemy shaders like the Raylenght. I'm use in combine with blend color utility node and in setting of raylenght i'm set "Ray lenght inverted (normal edge)".
See the images
Hope help you

P.S. No problem with the uv mapping, it's arbitrary of this.

01 January 2010, 02:06 PM
this is for example, with cloud mapped into FarPlane of raylenght, with never other parameters eccept for the scale of the clouds node ( (

01 January 2010, 02:10 PM
could you post the settings you used? I trying to do the same but the settings are quite weird . Far Plane? Fare output? thanks


01 January 2010, 02:18 PM
I'm remember to use BinaryAlchemy_Shaders

01 January 2010, 02:37 PM
cool thanks!

I simple connected the cloud into the BA_color_raylength node. It looks promising :) ( (

01 January 2010, 02:43 PM
cool thanks!

I simple connected the cloud into the BA_color_raylength node. It looks promising :) ( (

;) yep joke with parameters or gain of clouds to see it up more

02 February 2010, 10:40 AM
Hi guys, I found a better solution than the binary alchemy shader, the "crtl_occlusion". Here you can even influence the the direction of the dirt like the vray dirt :) crtl_occlusion is amazing !! It is the one I was always looking for

02 February 2010, 02:58 PM
hi guys great methods all, but I can't make that damn mentalrayVertexColor working under maya 2009? can someone share a simple scene that works with the mrvc connected to see if it comes from my maya version or if I'm doing something wrong?
thanks a lot.
Also where to find the Ctrl_occlusion shaders? and how fast is it?
many thanks

02 February 2010, 03:18 PM
There is a bug in Maya 2009. If I remember correctly you need Maya 2009 Service Pack 1a.


02 February 2010, 04:39 PM
probably there is another link were we can find crtl_shaders

crtl_occlusion (

02 February 2010, 09:34 AM
Jan: yes I confirm it's a bug on maya 2009, you need serive pack 1a for it to work...damn too bad I didn't feel like upgrading at the time, will have too now.

Stuccy: thanks o!


07 July 2010, 08:59 PM


I see that u have to paint the vertex in the object by painting them or assigning a apply a color to create a color set, I was trying to follow the method but something I am doing wrong.... I am not getting that effect...

I understand that if u have many object in the scene and each object are formed by others polygons
is convenient this method because you do not have to UV layout and you can apply a layered shader texture method, at the end what you get with this method is a kind of procedural MASK to apply between the the two shaders using a layered shader node

but the difference with 3dsMAX is that seems he uses a script to do the vertex painting in a automatic way and that is extremely handfull. however in MAYA you need to do that vertex painting manually and then apply the shading network that you built

anyway, if you need to do a "painting vertex manually" another way to do is using BODYPAINT 3D you can :
-select all your object in maya
-you can COMBINE the poly objects
-the create UVs Automatically
-in texture editor select polygons layout to keep shells non overlaping
-export as OBJ your object

-importing in BODYPAINT
and make an alpha channel there

this way you can paint the alpha mask in the exact way you want (dont worry about UVS) u are using just for paint the MASK in the areas you want

the advantages of this against your method is that you can control the shape of your mask by using brush types even using presure in your tablet

the you can go back to MAYA and apply that MASK as a 2d node texture on the layered shader node

07 July 2010, 04:59 AM
Hey guys nice bump on an interesting problem. I was wondering as well with automatically laying out UVs with no overlap (no longer working in Maya so, just guessing again) could you then use selection mask to select all edges with X amount of angle to get just the edges as in the first post's example and then in the UV editor export a uv snapshot with the highlighted edges and use that as a starting point for some PS work?

CGTalk Moderation
07 July 2010, 04:59 AM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.