View Full Version : Automated lip sync animation in C4D?

01 January 2007, 04:44 PM
I was hoping someone could give me some tips on how to speed up animating lip sync animation. I have a project that calls for quick lip sync animation. The quality does not need to be the best. I just need a way to create this lip sync very soon. :)

I have looked into using Synth Eyes to drive facial capture and use those nulls to drive my characters face, but the results are not very good.

Then i was thinking that InterPoser Pro may work. I was thinking that i could rig the character in Poser and use DAZ Mimic for poser, then import that result into C4d.

Does Motion Builder do this very well too? I thought i saw a post where someone used MB for lip sync?

I know i should just animate it by hand, but there is an enourmous amount of lip sync to be done.

01 January 2007, 07:28 PM
Hi Danb,

If the quality does not need to be excellent you can animate some very decent lipsynching with only four phonemes.

If you create phonemes for EE (wide), for OO (narrow), for mouth closed (M,S,P,T and so on)and mouth open (AA) and then make a four way slider ala Jason Osipa you can do the job very fast.

Try to sense the vowels in words and make sure you hit those.

Motionbuilder has the ability to do this but not that well. You would spend more time cleaning up than using the four way slider and doing it by hand.

If you have CD morph setting it up would be a matter of hours if not minutes, that is if you are animating a simple mesh.

Have a look at Jason's book, you might find that lipsynching is not as time consuming as you might think.

hope this helps

01 January 2007, 07:54 PM
have to agree with Jannis here I'd still take hand animated with limited mouth shapes personally.

When I studied traditional animation one of the best pieces of advice I got for lipsyncing was to study the two best lipsyncers in the industry, who weren't actually animators, but Rather Frank Oz and Jim Henson. they have the ability to make a character who's mouth only opens and closes talk convincingly. We then went on to study John Kricfalusi's teams work on Ren and Stimpy. Where they'd often have say four very very extreme mouth shapes and still manage a good lipsync. It really shows how its more about timing than about every single mouth shape.

No granted 3D is a little more challenging as it is smoother than 2D and with a 3rd dimensio to take into effect, but timing should always be your priority over complex shapes and number of shapes. What often kills lipsyncing is so many bizarre shifts from on mouth shape to another as the animator tries to replicate each sound

01 January 2007, 08:43 PM
Alright, thanks guys. I will use Jannis' recommendation. I didn't know that MB wasn't good for lip sync.

Does Jason Osipa only use 4 phenomes, or are you saying 4 just to speed things up?

01 January 2007, 08:59 PM
Just hinge the back of the jaw (or better yet REMOVE the top half of the head from the bottom). THEN make the top of the head pop up and down using the random vibration expression. Nothing beats it.
I've been studying the Parker/Stone approach.

Sorry guys--a bit of Sunday humor. ;)

01 January 2007, 09:00 PM
hi danb,
animation is definately not my field,
but maybe this will help you out:
it's called Lip Synch Tool by Motion Gimmick

hope it helps,

01 January 2007, 09:10 PM
Strange i don't see that plugin on the motiongimick site. ??

01 January 2007, 09:31 PM
Strange i don't see that plugin on the motiongimick site. ??

just visited motiongimick site, it's strange allright.
another plug i found is (it does exist this time!, but it's old for v.6):
lipsynch automatically
Hiroshi Ikeda

(1) After setting markers in a sound file (.wav), load it to a sound track in the Timeline ( Markers are not necessarily displayed in the Timeline).
(2) By using "Petit-Target" command (including in the plugin "Petit-Morph"), you will get a set of targets (Petit-Target) for an original object (polygon). For the targets, see readme of Petit-Morph.
(3) Select the object which has the above sound track, and with the commmand "AutoSynch", you will see a table dialog, in which you can set table name, figure name to be morphed, target names created above, phoneme names set in the sound file's marker names, and target weights for each phoneme. (You can also see the table dialog, by adding a P/T Table Tag shown in tag menu of the Object Manager. In this case, you need the command "AutoSynch" for creating an "ASynch" track.)
(4) Click OK, then you will get an ASynch track with ASynch keys for the object specified by the figure name. At the same time, in the Object Manager, a tag is attached to the object. Information of phoneme/target table is stored within the tag (Phoneme/Target Table Tag).
(5) By double-clicking keys, you can modify ASynch key (which is a kind of PMorph key).
(6) By double-clicking the tag, you can edit phoneme/target table in a dialog. If you need a modified ASynch track with this new table, first delete or move the ASynch track made before, then the command "AutoSynch". Now phoneme/target table is loaded from the tag (if its object is selected).
(7) If you need another tag, delete or move the above tag or select another object with no Table tag but with sound track, then "AutoSynch". Again, a new table dialog will appear.

01 January 2007, 10:00 PM
I agree with Jannis & Lucent Dreams. The only part of lip-sync that takes time really is modelling face shapes. By limiting how many you can reduce that. Lip sync can be done manually really quickly, especially if you don't do lots of passes. I'd animate the jaw open/ closed, then wide/ narrow, then anything else after.

01 January 2007, 10:32 PM
Alright, thanks guys. I will use Jannis' recommendation. I didn't know that MB wasn't good for lip sync.

Does Jason Osipa only use 4 phenomes, or are you saying 4 just to speed things up?

Jason shows in his book as a starting exercise lipsynching with a spline circle that only uses four shapes, open, wide, closed, and norrow. Thats all you need if you want to be economical.

If you see creature comftorts animations they don't use more than that. Like Kai said timing is the most important thing.

One thing that might help is to open the mouth at the start of the vowels before the sound arrives. You have to think that the mouth anticipates the sounds before it makes them.

I don't know how rushed you are on this job, but if you have some time buy Jason's book and at least do the exercise with the circle spline, it will clear up alot of things for you.

Hope this helps.

01 January 2007, 11:13 PM
aardman is another great example good one Jannis.

Sorry for this slightly off topic comment but for anyone doing lipsync one thing I'd really like to emphasize is avoid blinking when your character stops talking. So often you see animations where the characters seem to always blink when they aren't talking which makes the character look very stiff or robotic even.typically in a dialogue you blink more while talking as your eyes are highly focused while listening. Subconciously we often can tell one some isn't interested or taking us seriously not only through avoiding actual eye contact but also by blinking more when listening.

CGTalk Moderation
01 January 2007, 11:13 PM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.