Compositing workflow question


#1

Hello, my friends,

I have got task to do. The client wants me to create a 3D model of a tablet and to composite it with real hands (from a real footage) in POV. How would you do this? The tablet should be controlled by the hands and respond to the touches of them. The sequence should be around 10 secs long.

I can work with Maya and After Effects.

Any suggestions?


#2

That’s actually pretty complicated. These are my initial thoughts. I’m positive im going to leave out something, but thought I might help get you pointed in the right direction. If I sat down and actually planned this out, the steps might be in a different order all together. I just wanted to get you thinking about it. Hopefully someone that has done this more will chime in.

Make sure you’re working in linear color space in both Maya and AE(set comp to 32bit(float).

1: Model the tablet, or purchase one online, depending on your budget.

2: Watch this video, https://vimeo.com/121829515 (there’s more than one video I believe, watch them all) Theres also a script there you will need, its linked in the description of the video.

3: Render out the hands in AE as .mov file and import that into the maya scene and attach it to your camera that you brought in from AE(explained in above videos). Make sure you read everything and watch the videos thoroughly as you need to keyframe certain things or it won’t work.
If I remember correctly, you can bring nulls from AE into Maya, if so, 3d track the footage, pick a point on the hand that is holding it in AE and create a null at that point. Export that null to Maya and attach your tablet to it. Make sure you create a hierarchy that allows you to do corrective/subtle animation to the tablet. Animate tablet in maya. Scale is going to have to be “eyeballed” for the tablet here, im not sure of a better way of figuring that out, but making the above mention hierarchy with a “Scale” node will help with any adjustments that need to be made here. Keep in mind, if you do have to make adjustments to the scale of the tablet, that will effect the placement of the elements in your “screen” animation.

4: Create your tablet “screen” composition, bring in the hands footage as a “guide layer” for timing purposes and to correctly place UI elements that will be controlled by the hands/fingers. You’ll most likely be making a lot of adjustments to this, as the animation and placement of elements won’t match correctly to the hand footage on the first attempt. I would suggest keeping it very simple until you have timing and placement figured out, which you can then use as a guide for the “final” animation of the screen. Make a playblast of your animation you’ve done in Maya(WITH the screen animated texture applied to the tablet) and bring that into AE and throw it under the hand footage, at this point your just testing to see if your “screen” animation is lining up correctly and timed correctly. Rinse and repeat until it’s working, then do your final screen animation and export that as a sequence to use on the screen in Maya as a texture.

5: Light/texture your Maya scene(tablet) to match the footage.

6: Setup render passes and render tablet scene to .exr sequence

7: Import EXR sequence into AE and make adjustments to the passes as needed. You’ll need to do some roto work to the footage to make the tablet appear it’s in the hand and also for any movement of the other hand that goes over the tablet. Make sure your sequence footage is interpreted correctly to match the video footage(24,25,30 fps).

8: Render final comp

If there wasn’t any planning done for this initially, and there wasn’t anyone on set for the hand footage(worse case scenario is that it’s stock footage) you’re going to find this extremely time consuming and challenging. If it was done correctly, they should’ve shot the hands WITH a tablet that had a green screen and markers on it that you would then comp your screen animation onto the tablet IN the footage(there would still need to be roto work done) for the most cost/time effective solution. If not, I hope you’re charging them accordingly. Best of luck on this one, and sorry if I missed any steps, I hope it helps.


#3

Hi, Remi,

writing of this post must have been really exhausting, so thank you very much for it!

Finally, I have chosen a little bit different solution, as I had only one week to finish it. I created a model in Maya, then I rendered a sequence with KeyShot with transparent background (including some 3D movements, so it looks like 3D). After that I imported the sequence into After Effects, green screened one hand (the one that holds the tablet) and then separately the other hand (the one that controls the tablet). Now I am trying to do my best to composite it somehow together. The most difficult part is the motion relationship between the tablet and the holding hand, which still looks a little bit robotic and unnatural… -_-

So thank you again and keep your fingers crossed for me! :slight_smile:


#4

I hope it works out for you. Sounds like a bit of a nightmare project. Best of luck!