Motion tracking on a panned video?

Become a member of the CGSociety

Connect, Share, and Learn with our Large Growing CG Art Community. It's Free!

 
Thread Tools Display Modes
  07 July 2018
Motion tracking on a panned video?

Hey guys,

I'm doing my first 3D tracking in C4D, and I feel like I might have been a bit too ambitious for a first 3D track...

My video is a continuous shot that lasts 10 seconds long and in terms of what's going on in the shot, you can divide it in 3 parts:

Part 1 : You see my feet walking on the sidewalk
Part 2 : I pan the camera up (from my feet) to the street in front of me
Part 3 : I pan the camera all the way to the right to show the other end of the street

Problem : None of my automatic or manual tracking efforts have worked.

Automatic tracking : The automatic tracking (800 points) seem to only start tracking at Part 3. Why? Also, every tracking point goes crazy between Part 2 and Part 3 because of a little bit of motion blur. Can I fix this manually? I've looked at so many tutorials, yet I can't seem to find how to fix automated tracks. I try replacing them on each frame but their new positions don't seem to save.

Manual tracking : I spent 2 hours creating manual tracking points that start and end at each parts of the video, giving a total of 42 perfectly and manually tracked points. Yet, when I run the 3D solver, it doesn't create any Nulls! Why is that? Does it need more points, still? This results in me not being able to put a Planar Constraint tag on the Motion Tracker.

Any help/guidance would be extremely helpful.

Thank you so much!
 
  07 July 2018
If at all possible, break the shot down to just the part where you absolutely need 3D tracking

Dollying/moving the camera is better than panning for tracking. You need parallax on your track points

hand held shots often have too much camera shake blur to track effectively. using a higher shutter speed can help but can make your video look strobey.

rolling shutter CMOS cameras are often a nightmare to track due to the shearing effect. Cheaper CMOS cameras may be worse for this. I recently tracked canon C100 footage (CMOS), which went pretty well.

I recommend practicing with a really simple really short shot first. place stickers or other markers over the walls an ground to make the track quick, easy and accurate. move (don't just pan) the camera slowly and steadily but keep it moving throughout the shot. Focus on one area and make sure some markers stay in throughout the shot. make sure you have more than one plane in your shot (eg not just a floor but also walls or other uprights). Once you have successfully tracke this type of shot you should have a much better idea of what is and isn't possible and how to plan your shots for tracking.
__________________
I take none sugar
 
  07 July 2018
Originally Posted by You need parallax on your track points: You need parallax on your track points:

This... You cannot "3D track" a pure panning shot because all points move at the same speed relative to the camara, so there's no way to interpret the depth of the scene.

You can do a 2D track or a planar track (AE, Mocha...) to get the rotation value, but you can't get translation values o depth information. That's not a limitation of C4D, it's a math thing.

Some 3D trackers like syntheyes get away with it by having a special pan/tripod mode where they project all points at an arbitrary value.

You can try to do the same in C4D by projecting your markers on some rough 3D model of the scene (even an environment sphere or cylinder), it may help. Look at Geometry assisted tracking.

In any case, try to split your scene in smaller simpler chunks with their own tracking strategy, you should be able to stitch the various solves later (with hand adjustment and camera morphs for instance).
 
  07 July 2018
I think the OP was dollying - "You see my feet walking on the sidewalk" - so I don't think he was stationary for the entire time. However, if he was walking, there would probably be a significant amount of bumping and wobbling, and that CMOS issue that bunter mentioned would be a killer.†

What kind of camera/lens were you shooting with? Lens distortion might be hurting the situation as well. My first (and only) track in C4D was a GoPro shot that I was never able to fully correct, even after I undistorted it.†
 
  07 July 2018
Hey guys, first off, a huge thanks for your replies! I've learned a lot already.

Originally Posted by EricM: In any case, try to split your scene in smaller simpler chunks with their own tracking strategy, you should be able to stitch the various solves later (with hand adjustment and camera morphs for instance).

Is there any way you could give me a tad more information on how I would be able to stitch the solves? Is there any tutorial on that that could get me started? I've learned a ton already but 3D tracking is still super new to† me, and this process feels a tad complex.

Originally Posted by LukeLetellier: I think the OP was dollying - "You see my feet walking on the sidewalk" - so I don't think he was stationary for the entire time. However, if he was walking, there would probably be a significant amount of bumping and wobbling, and that CMOS issue that bunter mentioned would be a killer.

I was only walking in Part A. Just before I pan up to the street (Part B), I stop walking.

I was shooting with a Fuji XT-1 (not full frame), 10mm lens. You are right, now that I think about it, the 10mm might not be helping at all. The GoPro also has a pretty large lens.

Thanks a ton for all the info @bunter!

I guess I will be doing 2 things;

1. Try to track Part A-B-C together and then stitch the solves together.

2. Try to reshoot the same scene, except with a framerate of 60FPS (as opposed to my current 30 FPS) and by panning in a much slower fashion. Do you guys think that would help? I could then speed up the parts I want to speed up in After Effects.

Thanks again, much appreciated!
 
  07 July 2018
I canít answer all your questions right now but quickly wanted to note that frame rate is not the same as shutter speed. Increasing the frame rate will not make any difference to your solve.†
__________________
I take none sugar
 
  07 July 2018
Originally Posted by JEEK: Is there any way you could give me a tad more information on how I would be able to stitch the solves? Is there any tutorial on that that could get me started? I've learned a ton already but 3D tracking is still super new to† me, and this process feels a tad complex.

There's no magic bullet I know of, it's very much a hand process : each solve will give you a new camera with its keyframes.

You can make a copy of the solved cameras, unparent them and move them around so that their path align (if you have a little bit of overlap between the solves it may help).

Ideally you have set the origins and orientiation constraints right so that the scale and direction of all solves match already as close as possible. Else it's very much eyeballing. Having a reference object in the scene should help.

At last, once the invidual solves are in place you can use a camera morph tag to smooth out the transition of the main cam from camera solve 1 to camera solve 2 to camera solve 3 in one continuous motion.
 
  07 July 2018
Originally Posted by Jeek: I was shooting with a Fuji XT-1 (not full frame), 10mm lens. You are right, now that I think about it, the 10mm might not be helping at all. The GoPro also has a pretty large lens.


Good golly, yes, the 10mm is absolutely killer here, as the warping makes tracking points move in a very awkward fashion, and the solver won't be able to properly understand what's going on with those manually tracked points.† In general, you want to shoot footage for tracking with longer lenses; the less lens distortion you have, the better.

For your existing footage, you'll want to bring it into AE, unwarp it, and bring the unwarped footage back into C4D for tracking / 3D work. You can then either use the unwarped footage w/ 3D for your final shot, or warp your renders to match the original footage.


Adding on to bunter's point: Frame rate is the amount of images you're taking per second. Shutter speed is the amount of time the camera takes in light per frame. The more light that's taken in, the more motion blur you get. The faster your shutter speed, the less motion blur you get, and the sharper your image is. However, it also means that fast pans may not look that good. So there's a tradeoff there, and some testing might be needed to igure out exactly where the happy medium is.
 
  07 July 2018
Actually focal length is not really a problem if the lense is equirectangular, and a prime is certainly a good thing to begin with.

The problem with gopro footage isn't the field of view, it's the fisheye distortion. Is the 10mm a fisheye too?

If you know the exact focal length and the sensor size it's already a great step in the right direction.

You can also create a lens distortion profile in C4D and feed it to the solver.

Next, take measurement on location so that you have points of references (ej : this trash can is 80cm high, this bank is 200cm long etc...), and add as many trackers as you can in the scene (they can be a plastic cap for instance, a piece of chewing gum... stuff that look natural in the scene but that you hand placed so that you know how far apart they are etc...). It can also help you create basic geometry to project you trackers on, and set the orientation constraints right.

Regarding shutter speed vs framerate, if he uses a 180į shutter it will actually always be twice the framerate (=1/50 for 25 fps, 1/100 for 50 fps) so it will make a sharper image. It depends on the camera setting.

Shooting with a high shutter speed (very sharp) is a actually a good idea : you can always add motion blur to your footage back in AE or Fusion (Reelsmart motion blur or equivalents) in the end. That way you'll motion blur everything at once, the plate and the CG.
 
reply share thread



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
CGSociety
Society of Digital Artists
www.cgsociety.org

Powered by vBulletin
Copyright ©2000 - 2006,
Jelsoft Enterprises Ltd.
Minimize Ads
Forum Jump
Miscellaneous

All times are GMT. The time now is 08:29 AM.


Powered by vBulletin
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.