Facial animation questions : CV AR question + Pose morph to Motion clip?


#1

Hi, I’m doing some tests here on facial animation with CV AR and I’m hitting a couple of roadblocks

  1. First, it appears that when baking the CV AR to keyframes, the morph data is lost. The sliders do move as expected, but the actual point deformation are wiped out (either absolute or relative morphs).

  2. More generally speaking, is it possible to bake the posemorph animation to motion clips? I’m trying to string together various small takes into one longer sequence, and it looked like the best option… except that it doesn’t seem to work even when checking “parameters”. Any idea about that?

Thanks,
Eric


#2

Hi Eric,

CV-AR doesn’t have any PoseMorphs at all. It uses vertex animation to animate the mesh directly (this is the only data that iOS gives you for the facial animation).

When you bake you are baking the blend shapes values. You can use these blend shapes to drive your own Pose Morphs, but you have to create these yourself (by sculpting them in C4D or zbrush etc…).

You won’t be able to turn the animated face captures into MotionClips since they don’t support vertex animation (as far as I am aware).

I don’t know for sure, but you might consider looking into a point based baking method, possibly to alembic files And then importing the multiple files together into one sequence in C4D.

Best,
Kent


#3

Hi Kent,

Thanks a lot for the reply, seeing that you worked on the plugin, that’s very helpful.

This being said, I did sculpt my own set of blendshapes/posemorphs, linked them, and it works well : I have my custom character’s face animating. That is until I try to bake it…

When I press “bake blendshapes” o “bake all”, it looses the connection to all the previously linked morphs (when in absolute mode) or simply looses the relative vertex offset when in relative mode. So I’m left with sliders moving and animating empty morphs, and I have to relink everything again by hand. I was wondering if that was a bug, limitation or something I missed.

On the second point, I was not considering baking the ARKIT animation directly to motionclips, but the result of the baked posemorph tag sliders. I thought the “parameters” option would do the trick, but apparently not.

I’ll keep looking into it and will post my findings if I get a working solution for longer “multi-take” sequences. Alembic is already on my list.

Cheers,
Eric


#4

Hi Eric,

The Bake BlendShapes and Bake All are intended as a helper for artists in the initial setup. You can use Bake BlendShapes to create a new PoseMorph, with a mesh (ie point data) for every pose morph/blend shape, ready to start sculpting them. It isn’t intended to be used to bake already exisiting expresso setups that you have done yourself.

So your requirements are for someone who has already setup all of their own Pose Morphs ( ie hasn’t used the Pose Morph Helper at all to get started) and would like to bake them down. I too would have thought that C4D has a way to bake these already but if not then I can add it to my todo list. It is a feature that is more general and not specific to CV-AR itself, since it could be used to make any driven PoseMorphs to keyframes.

If you find a solution in the mean time do let me know.

Cheers,
Kent