PDA

View Full Version : Facial Rig. Joint Based + Pose Space Deformation


andyLP
06-30-2011, 11:52 AM
Hi everyone,

Thought I'd share my latest work.

http://www.vimeo.com/23310624

Its a facial rig that uses a network of joints that follow the basic contours of the face.
There are a number of pose space deformations that trigger when certain poses are hit.
There is also a breakdown that I created on my vimeo page, if you'd like to see some behind the scenes type stuff.

Questions, comments and critique welcome. :)

theflash
06-30-2011, 02:07 PM
Good technique and good details in your rig. I have always thought that joint based rig + pose based correction is a good way to go to get more control. But it would be even nicer to have a pose read at vertex level or at a topology level like what daniel pook-colb's BCS system does in a way. So you don't create poses against higher level movements, but against low level movements.

Good behind the scene video as well. One question, I cold not get bSpiritCorrective script to work after trying many times. Did you get it working on the first go?

eek
06-30-2011, 03:26 PM
Hi everyone,

Thought I'd share my latest work.

http://www.vimeo.com/23310624

Its a facial rig that uses a network of joints that follow the basic contours of the face.
There are a number of pose space deformations that trigger when certain poses are hit.
There is also a breakdown that I created on my vimeo page, if you'd like to see some behind the scenes type stuff.

Questions, comments and critique welcome. :)


With the combination sculpting - do you store the 'order' of the combinations to allow for rebuilding?

For example, i build shape 'a', shape 'b' and a combination/fix shape 'c' of both. Now if i go and re-tweak 'a' - does 'c' get mashed as its an additive or do you store the absolute value of it so you can rebuild it from the new base shapes?

I built a framework for this several years back, you make your bases that get flagged as being order 0, you make a combination of 2 base which gets an order of 1 - a combination of a combination and a base gets level 2 etc.. Essentially its the level of combination - and basically teaches you to build the bases first.

The rule being a combination of two or more poses of the same or mixed level gets a level 1 higher than the highest level of the mix.

The second rule is with every combination, you store the 'absolute' value of that shape whether its a combination or not. This allows you to re-build that shape following the 'order' of levels - 1.. n

Ontop of this is pushing the poses into a weight space to allow for any combination at any value. With that its, just a simple product of weights. e.g.

a shape with weight 1.0
b shape with weight 1.0
c shape with weight (a * b)
d shape with weight of (c * a * b)

The values will always go to 1.0 and crucially be non-linear i.e 0.1 * 0.1 as apose to 0.5 * 0.5. e.g. 0.1111 > 0.25 > 1.0. Additional combination can be at a 'position' of two weights.

c shape at combination of a @ 0.5 and b @ 0.5. The rule being your still going from 0 > 1 ) 0 but in the space of 0 > 0.5 > 1.0.

cheers,

Xeash
06-30-2011, 05:08 PM
Nice rig!

Good technique and good details in your rig. I have always thought that joint based rig + pose based correction is a good way to go to get more control. But it would be even nicer to have a pose read at vertex level or at a topology level like what daniel pook-colb's BCS system does in a way. So you don't create poses against higher level movements, but against low level movements.
So e.g. edge compression drive the poses?

For example, i build shape 'a', shape 'b' and a combination/fix shape 'c' of both. Now if i go and re-tweak 'a' - does 'c' get mashed as its an additive or do you store the absolute value of it so you can rebuild it from the new base shapes?

I built a framework for this several years back, you make your bases that get flagged as being order 0, you make a combination of 2 base which gets an order of 1 - a combination of a combination and a base gets level 2 etc.. Essentially its the level of combination - and basically teaches you to build the bases first.

The rule being a combination of two or more poses of the same or mixed level gets a level 1 higher than the highest level of the mix.

The second rule is with every combination, you store the 'absolute' value of that shape whether its a combination or not. This allows you to re-build that shape following the 'order' of levels - 1.. n

Ontop of this is pushing the poses into a weight space to allow for any combination at any value. With that its, just a simple product of weights. e.g.

a shape with weight 1.0
b shape with weight 1.0
c shape with weight (a * b)
d shape with weight of (c * a * b)

The values will always go to 1.0 and crucially be non-linear i.e 0.1 * 0.1 as apose to 0.5 * 0.5. e.g. 0.1111 > 0.25 > 1.0. Additional combination can be at a 'position' of two weights.

c shape at combination of a @ 0.5 and b @ 0.5. The rule being your still going from 0 > 1 ) 0 but in the space of 0 > 0.5 > 1.0.

Bassically answer goes from what PSD he have drives: vertex position or blendshape weights.
Both of them are useful anyway.

theflash
06-30-2011, 05:39 PM
@eek
I am guessing he is using blendshapes in which case it will be additive and hence correction in lower order will affect all higher orders

@xeash
Yes, edge distance or vertex positions or any kind of representation at lower level will allow much more freedom. However, it will be computationally expensive.

andyLP
07-01-2011, 12:10 AM
@theFlash
Thanks for your comment - I got bSpiritCorrectiveShape going, though there's a couple of things that tripped me up. It isn't fussy about which tweakNode it uses to build the shapes.
So if there's more than one tweakNode in the history of your object then chances are it won't get the right one. I edit a quick hack to the script which seems to work in most instances, I can send to you if you like.
It won't work well if you have any blendshapes turned on at the time your run the script. In fact, the only deformation you want to have active is the skinCluster.
Its not particularly good if you have a smoothMesh node. And you should make sure to turn iterations to 0 on any polyAverageVertex nodes you might have. (I had a few!)

A pose read at component level would be handy. My 'system' (if you can call it that) really isn't all that fancy. The intermediate mesh is the result of the sculpt mesh minus the existing blendshape information. It is the intermediate mesh that is used to build the PSD.

@eek
Thanks for your feedback. In your example, shape c breaks if you tweak shape a (or b), yes. Though I can see how I might alter things to keep track of adjustments like that.

I have my definition of orders slightly wrong. In a purely blendshape driven system, order 0 would be the base shapes. In the PSD set up, I kind of thought of order 0 as the skinCluster deformation - which isn't correct at all, but helped me when thinking things through, as I always had to calculate the sculpted shape against the pose.

Building weight space shapes like you mentioned, makes sense. I never thought of making it a product of shape values like that. All my shapes are driven by positions of controllers (or combinations of controllers)

Have you got any links to your system in action?
And thanks again for your feedback - I'm sure I've been reading and learning from your posts on cgtalk for the last 4 years!

eek
07-01-2011, 12:24 AM
@theFlash
Thanks for your comment - I got bSpiritCorrectiveShape going, though there's a couple of things that tripped me up. It isn't fussy about which tweakNode it uses to build the shapes.
So if there's more than one tweakNode in the history of your object then chances are it won't get the right one. I edit a quick hack to the script which seems to work in most instances, I can send to you if you like.
It won't work well if you have any blendshapes turned on at the time your run the script. In fact, the only deformation you want to have active is the skinCluster.
Its not particularly good if you have a smoothMesh node. And you should make sure to turn iterations to 0 on any polyAverageVertex nodes you might have. (I had a few!)

A pose read at component level would be handy. My 'system' (if you can call it that) really isn't all that fancy. The intermediate mesh is the result of the sculpt mesh minus the existing blendshape information. It is the intermediate mesh that is used to build the PSD.

@eek
Thanks for your feedback. In your example, shape c breaks if you tweak shape a (or b), yes. Though I can see how I might alter things to keep track of adjustments like that.

I have my definition of orders slightly wrong. In a purely blendshape driven system, order 0 would be the base shapes. In the PSD set up, I kind of thought of order 0 as the skinCluster deformation - which isn't correct at all, but helped me when thinking things through, as I always had to calculate the sculpted shape against the pose.

Building weight space shapes like you mentioned, makes sense. I never thought of making it a product of shape values like that. All my shapes are driven by positions of controllers (or combinations of controllers)

Have you got any links to your system in action?
And thanks again for your feedback - I'm sure I've been reading and learning from your posts on cgtalk for the last 4 years!

Daniel's work pretty much uses the exact system i described - but he's created the actually blendshape plugin i think too.

http://dpk.stargrav.com/pafiledb/pafiledb.php?action=file&id=31

And the math:
http://www.stargrav.com/bcs/docs/data/ap1-dps.html#dps

in max its pretty simple as you just store the data on the channel - im guess you could use an enum in maya. One day i will write mine in python :)

theflash
07-01-2011, 07:07 AM
@Andy
It would be helpful if you can share the modified script. I think I only had one tweak node but I will have a look at my set up again.

I created a plugin last year called input mixer, which takes multiple 1d inputs and outputs between 0 and 1. Plugin calculates when all inputs reach a given goal then it will output 1. I wanted to make a demo of it but haven't managed to get time to do so.

Xeash
07-01-2011, 08:42 AM
Well i what i do usually is like this:
-i wrote RBF numeric interpolator node which interpolate on blendshape weights or activations weights(which drive deformers position). It works as any RBF on numerics should do: it can be have any number of dimension(any number of attributes can drive the weights), so you just fill all datapoints correct, compute weights, and thats do the job. It hits all datapoints exactly and also good with extrapolation if needed.
-So you have setup : hi-level controls with -1 1 clamp which drives PSD(RBF nodes) and thats drive weights => which drive blendshapes/deformers directly or with some sort of remapping(which is better).
Yep pretty much like in disney doc.

theflash
07-01-2011, 02:14 PM
My input mixer node does something similar, using Radial Basis function based on Gaussian function. I take all the inputs as start and end pair and process all inputs into unit space. And the unit output is mapped using different function similar to RBF to get different kinds of interpolations. I am planning to have some more functionalities to allow features like subsets within the same vector, but as of now it would still work by creating different nodes that define the subsets and a final super set combining the outputs of individual sets.

Xeash
07-01-2011, 02:49 PM
My input mixer node does something similar, using Radial Basis function based on Gaussian function. I take all the inputs as start and end pair and process all inputs into unit space. And the unit output is mapped using different function similar to RBF to get different kinds of interpolations. I am planning to have some more functionalities to allow features like subsets within the same vector, but as of now it would still work by creating different nodes that define the subsets and a final super set combining the outputs of individual sets.
Cool!

And the unit output is mapped using different function similar to RBF to get different kinds of interpolations.

So you filling RBF matrix/computing weights via Gaussian and then alter output weight by different handy functions?

theflash
07-01-2011, 04:17 PM
It's actually simple, the vector output is from 0 to 1 and it's mapped into output number range using Gaussian and some other functions based on what kind of shape you need, spline, cubic, linear etc..

andyLP
07-01-2011, 10:12 PM
@Xeash.
Thanks for sharing. Its good to get some ideas on where to head with my next rig. I'm always looking to bring out a better reel.

@theFlash, eek
I thought I would be able to get by, to some degree, without being an overly advanced programmer. Do you guys think, that at the top end studios, if you want to be setting up characters, you have to be a superstar programmer?

eek
07-02-2011, 01:20 AM
@Xeash.
Thanks for sharing. Its good to get some ideas on where to head with my next rig. I'm always looking to bring out a better reel.

@theFlash, eek
I thought I would be able to get by, to some degree, without being an overly advanced programmer. Do you guys think, that at the top end studios, if you want to be setting up characters, you have to be a superstar programmer?

I don't think so - in the end its being able to turn ideas into reality thats key. I've never done much programming just scripting, and read lots of math. (and a specific area - matrices, quaternions and trigonometry) I was'nt very good at math in school, only last 5 or so years got really into it.

What i would say is to find a mentor/peer who can help you understand core ideas. It's what i did and it helps to give you a push to learn on your own. Most tools really are grounded in the same way - just there approach is different. Crucially if you can learn something with a goal of a practical example (that is hopefully cool too :)) you'll learn really quickly.

tristratos
07-02-2011, 06:24 AM
Dear, all,

Where can I download bSpiritCorrectiveShape for maya 2012?

I've searched at http://www.b-ling.com/ but there isn't one available?

Does someone else provides a compiled copy of this to test it out ?

Tri

theflash
07-02-2011, 09:17 PM
@Xeash.
Thanks for sharing. Its good to get some ideas on where to head with my next rig. I'm always looking to bring out a better reel.

@theFlash, eek
I thought I would be able to get by, to some degree, without being an overly advanced programmer. Do you guys think, that at the top end studios, if you want to be setting up characters, you have to be a superstar programmer?
I do not work in the animation industry yet. But I can tell you from what I have gathered. Some technical artists use more math and some use less. However, all artists are able to solve the problems in their own way. That ability is more important. And as Charles mentioned that you also grow working with talented people.
Most of what I have picked up is from forums and many talented artists' work online.

andyLP
07-03-2011, 04:50 AM
I don't think so - in the end its being able to turn ideas into reality thats key. I've never done much programming just scripting, and read lots of math. (and a specific area - matrices, quaternions and trigonometry) I was'nt very good at math in school, only last 5 or so years got really into it.

What i would say is to find a mentor/peer who can help you understand core ideas. It's what i did and it helps to give you a push to learn on your own. Most tools really are grounded in the same way - just there approach is different. Crucially if you can learn something with a goal of a practical example (that is hopefully cool too :)) you'll learn really quickly.

That's great advice, thanks. I had been studying up on matrix and vector math in recent times, after having not touched it since university. Good to know quaternions and trigonometry are also ones to study up on. Makes sense! .. just gotta find a mentor!

andyLP
07-03-2011, 04:52 AM
I do not work in the animation industry yet. But I can tell you from what I have gathered. Some technical artists use more math and some use less. However, all artists are able to solve the problems in their own way. That ability is more important. And as Charles mentioned that you also grow working with talented people.
Most of what I have picked up is from forums and many talented artists' work online.

Thanks for your response mate, great to get your perspective.

andyLP
07-03-2011, 07:47 AM
Dear, all,

Where can I download bSpiritCorrectiveShape for maya 2012?

I've searched at http://www.b-ling.com/ but there isn't one available?

Does someone else provides a compiled copy of this to test it out ?

Tri

The plugin hasn't been updated for later versions of Maya. You'll have to make do with the mel script.

chovasie1
07-03-2011, 09:17 AM
Thanx for sharing Andy. Looks really nice and just what i was looking for!

I think i have compiled version of CBS for maya 2009. Christian was really kind and gave me source so i got it compiled for maya 2009. I can find it and share with you if you want... But i think i have only 32bit version so far...
As i did, you can ask Chris for source files and i'm sure he will give you so you can compile for any version of maya you need...

Only problem is that Bling CBS is not 100% accurate when you use rotations on joints in your expression... Returned blendShape will not match 100% with sculpt. It works perfectly with translations dough... More rotation = less accurate result...

There are some other solutions that works 100% accurate no matter how much translations and rotations you use in your expression.

Chovasie

andyLP
07-03-2011, 09:56 AM
Heh, no problem, glad to share!



Only problem is that Bling CBS is not 100% accurate when you use rotations on joints in your expression... Returned blendShape will not match 100% with sculpt. It works perfectly with translations dough... More rotation = less accurate result...


Forgive me, but I'm not sure I know exactly mean by 'in your expression.'
If you mean simply rotating the joints starts to cause issues, then that's interesting. I wonder if this is this the case for the mel script as well as plugin?


There are some other solutions that works 100% accurate no matter how much translations and rotations you use in your expression.


Cool, what are they!

I've never actually used the compiled version. I would be very interested to know how much faster it is than the script as the script version slows me down a bit as I'm using a lot of topology.
Thanks for your comment

tonytouch
07-03-2011, 09:57 AM
hi kids ... try this one :
http://www.chadvernon.com/blog/maya/free-cvshapeinverter-tool/ :beer:

andyLP
07-03-2011, 10:09 AM
hi kids ... try this one :
http://www.chadvernon.com/blog/maya/free-cvshapeinverter-tool/ :beer:


woooooooahhh that plugin is fast. thanks for the link tonytouch!

chovasie1
07-03-2011, 10:57 AM
Sorry if my `expression` term was confusing... By that i mean, if you use rotations on joints when you want to make `smile expression` for example. By expression i think about facial expression or pose.

So, problem is that if you trying to get some facial expression, and use rotations on joints beside translations, you will get some error in corrective BS that b-ling returns you. I think that even a scale of object makes difference... On smaller objects that are 1x1x1 unit for example error is bigger than on large objects as well...

Tony was faster than me... :)

Chad's solution works fine and fast and also Oleg Alexander's script from `The Art Of Rigging` book, cgTkShapeBuilder, but it need some changes to make it easy to use because CBS is just a part of that script... but math is 100% correct.

Chovasie

andyLP
07-04-2011, 04:19 AM
@Chovasie
Ahhh I'm with you! :) I'll have to test that out.
Cheers mate!

Boucha
07-05-2011, 05:48 AM
That my friend is a masterpiece in facial deformation cheers :)

tristratos
07-05-2011, 07:04 AM
There are some other solutions that works 100% accurate no matter how much translations and rotations you use in your expression.

Chovasie

Which other solutions are you refering to ?
Mel scripts ?
Plugins ?
Which ones ?
Do they workf lawlessly width Maya 2012 ?

chovasie1
07-09-2011, 12:08 AM
@tristratos

As i wrote above, cgTkShapeBuilder script from Oleg Alexander, which is described in `Art Of Rigging` book works just fine... It's mel script but for a little bit different use... so maybe you can use it to make your own script for making good corrective blend shapes. Original script do some other stuff too so in original form i complicated to use if you just want to make blendshapes.. But math works perfectly and no matter how much rotations you use on joints it will return you a 100% accurate corrective blend shape..
I had only one problem with this script, when rotation on joint is 90.00 degree it will make error.. if you use 90.0001 it will work correctly again.. but it have problems with clean 90 degrees... it will translate your vertices to infinity and wont work in that situation.. so try to avoid clean 90 degree rotations.
My point is that bling corrective shape which is most common script for making corrective blendShapes is not working properly, if you use rotations on joints blendShapes are off...
I didnt use maya 2012 but it's a mel script and im pretty sure it will work. If you make your UI and fix the script i think you will avoid all problems with compatibility... Most of problems with old mel scripts is with UI in maya 2011+

Chovasie

tristratos
07-09-2011, 06:24 AM
@ chovasie1

As mel is not my main expertise, have you edited the scripts in such a way that you've made it work for CBS too and you could possibly share ?

tonytouch
07-09-2011, 06:29 PM
hi guys ,

i actually did not try mr. Vernon's python-plugin yet , but i am sure it is very good ! and well .... compared to the point of time , when cgToolkit 1 was released , 5 years later i think , you really have an increase in polycount , since then .... so an API_plugin is surely much more comfortable than a mel script .

anyway ,
another great NEW method will be released some time soon , you might want to check this one out . it is NO CORRECTIVE , but a ADD-ON deformation ( normal based on top target-pose .... some kind of this / much more modern approach , and i think , it will be extremely cool , using corrections in combination with displacements.
http://forums.cgsociety.org/showthread.php?f=59&t=964350&page=1&pp=15

chovasie1
07-13-2011, 08:07 AM
Yes. Lbrush is very interesting solution and fun to work with! I already tested it an general idea is really good. But so far it have a lot of problems, lack of options and you cant do lot of things that you have on mind when try this approach...
I tried free beta version, so i hope that in time when final version is out it will be fixed and some features that are not included right now will be there in final version. But i think that this solution will need some time to get to the level when is usable in production...

Chovasie

adamdaly
07-18-2011, 02:03 PM
I really like the results of this method but wouldn't it quickly get very difficult to work with if your using a very dense mesh. I'm no expert with this stuff by any means so take this with a pinch of salt. Wouldn't it be better to use displacment maps for this sort of thing. I know that you'd run into problems with this but if you have your base displacement map with the pose displacement mats, added to each other and with masks, above it in a layer stack and used the controls to control the opacity of these pose maps; would that work?

adam

CGTalk Moderation
07-18-2011, 02:03 PM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.