View Full Version : Average UV Vertex Colours ?

10 October 2010, 03:03 PM

I would like to script the ability to average vertex colours based on whatever vert/s are currently selected, I have tried to 'echo' all the commands for the 'paint vertex color' tools, and then tried flood with smooth switched ( which is exactly what I want ), but I can't repeat the same functionality within Mel, I want to do two variations ( like the paint vertex color tool ) which work on channels RGB and RGBA. Any help or pointers in the right direction would be appreciated.

Thanks. :beer:

Robert Bateman
10 October 2010, 05:34 PM
1. use ls to get the selected verts
2. use polyListComponentConversion to convert that selection to the colour components used by each vertex.
3. use polyColorPerVertex to get the colours, and again to set the newly averaged ones.
4. ....
5. profit!

10 October 2010, 10:35 PM
Hi, thanks for the help.

I'm with you all the way up until 'and again to set the newly averaged ones.'

I don't understand how I can do that with the mel command polyColorPerVertex, there's certainly nothing in the MEL command reference relating to averaging / smoothing ?


Robert Bateman
10 October 2010, 01:14 AM
I don't understand how I can do that with the mel command polyColorPerVertex, there's certainly nothing in the MEL command reference relating to averaging / smoothing ?

averageColour = (colour1 + colour2 + colour3)/3;

It's only math.... Once you've determined the colour for the vertex, just use the -rgb flag to set the vtx-face colour.

10 October 2010, 11:23 AM
Thanks, again, but I'm still having trouble coming up with anything usable despite all your help.

Can you help me by breaking this down a little more ? :blush:

10 October 2010, 12:02 PM
Steven, if you open your Script Editor and turn on: History > Echo All Commands you can usually see a lot of the commands Maya runs "behind the scenes". The paint vertex context tool MEL command is: artAttrPaintVertexCtx and it is this command that is used to do the stuff when you use the tool. If you check out the MEL command docs for this command you should be able to figure out how to replicate it's effects via code. However, this will probably involve invoking the context, which may or may not be an issue in your case? If you just want to put smooth flood on a shelf button you should be fine.

Rob's "math method" is the "correct" way to program it and would be more appropriate if this was part of a more complex function or bigger script, but as you want the colours smoothed, not averaged, I think the whole process would be a little more complicated than simply adding, averaging and applying.


10 October 2010, 12:13 PM
Hi, Nathan


I do use 'Echo All Commands', but when I try to capture the command for 'smoothing' the vertex colours, it appears as : -

artAttrPaintVertexCtx -e -clear `currentCtx`;

which works within the context of selection, but not when replicating in a script. Like you said, I just want to add the 'smooth flood' to a shelf button ( or in my case within a little ui window I'm putting together ), but atm, I've not been able to replicate it's 'workings'.

I dont' seem able to pick out from the command reference where it does the smooth and flood parts, but I'll dig a little deeper and see what I can find.

The code I was putting together for Rob's technique was / is getting complicated really quickly, and I feel it may be a little out of my depth, when all I really want is a 1 off command line thing that I can call in my script. Like I said, I will look into the artAttrPaintVertexCtx command ref again and see what I can figure out.

Thanks for your help. :)

10 October 2010, 12:29 PM
here's some working code to get you going. I worked all this out just by doing echo all and looking at the command docs (although obviously I've done this kind of thing before too ;) )

string $pVtxTool = `artAttrColorPerVertexToolScript 4`; // initialize the paintVertex tool
artAttrPaintVertexCtx -e -sao "smooth" $pVtxTool; // set it to smooth
artAttrPaintVertexCtx -e -clear $pVtxTool; // perform a flood
to make you shelf button super helpful, you can always add the following lines to the start and end, so that when the user presses the button he doesn't lose the tool he had previously selected.
string $prevTool = `currentCtx`; // store the CURRENT tool, so we can go back to it
... insert above code here...
setToolTo $prevTool; // return to the previous tool

10 October 2010, 12:45 PM
Ah, excellent, thanks for that.

I had managed to come up with : -

artAttrPaintVertexCtx -e -sao "smooth" -clr;

but this took me into actually painting the mesh (but only if I had the tool open at the time )

your code ( this line in particular - I much appreciate :bowdown: ) : -

string $pVtxTool = `artAttrColorPerVertexToolScript 4`; // initialize the paintVertex tool

artAttrPaintVertexCtx -e -sao "smooth" $pVtxTool; // set it to smooth
artAttrPaintVertexCtx -e -clear $pVtxTool; // perform a flood

in fact, typing the command artAttrColorPerVertexToolScript into the command reference returns nothing, and echo all commands comes back with this line,

artAttrColorPerVertexToolScript 3;

what does the 4 do, and more to the point, how did you figure that out ? I really wish the command reference went a bit deeper some times. :/

Noticed you were at Sony Liverpool, I used to work there many years ago, good studio...

10 October 2010, 01:21 PM
Maya does not create the context tools by default, only when you first use them, so if you tried to use the artAttrPaintVertexCtx command on a specific context without actually creating it it would fail. Maya obviously creates the tool the first time you pick it from the Color menu, so I simply did echo all and chose "Color > Paint Vertex Color Tool", which gave me:

artAttrColorPerVertexToolScript 4;
... (loads of other crap)

now, I've coded in Maya long enough to know that any command that starts with a CAPITAL letter is probably a runTimeCommand or alias, which, checking with my openProc function turned out to be true, and the next line is always the "real" command stored in the runTimeCommand. so I again used openProc on artAttrColorPerVertexToolScript and checked what it did. turns out it initialises the tool, creating it if it doesn't exist. (4 enters the tool, 3 opens the properties dialog, and it describes this in the script's comment header)

you worked at Studio Liverpool too? small world! :)


10 October 2010, 01:30 PM
ah, thanks for the extra help, Nathan. Much appreciated.

The openProc command is super useful too, that'll be a big help in the future! :thumbsup:


CGTalk Moderation
10 October 2010, 01:30 PM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.