I need to do what should be a fairly simple motion tracking job, though not entirely sure what the smartest approach would be in my particular case. Its a two-fold problem.
Part 1: Id like to do a highly accurate camera match to a segment of video where the camera is not moving. I have measured markers on my background plate, and what I believe to be correct camera info (lens length, image sensor dimension) and a good approximation of distance of subjects focal plane from camera.
Part 2: I need to track a moving object in the video, and match an exact CG replica of same object in a highly precise way. The object in the video was derived directly from the CAD-model I am trying to match-move so theoretically I should be able to get a very accurate match. Ive even gone so far as to put tiny tracker points on the object before the video was shot and of course took measurements of distances between those tracker points.
To clarify further, heres a link to the actual background plate video. On the left side, youll see some 5mm black dots stuck to the scrim (for initial camera match) and on right side, tiny black pen-point dots on the white handle of the product object. https://www.dropbox.com/s/owb00dxfupx52yz/DSC_4105_halfres.mov?dl=0
Regarding Part 1, Ive already taken a stab at a camera match using the Camera Calibrator tag and its plane helper object. Ive used this toolset on many projects but I never seem to get particularly accurate results. This time is no different. I ran through the typical process and the results calculated looked somewhat accurate in terms of the angle of reference plane to camera, but still a little bit off in the following way:
- After running Camera Calibrator, the dimensions of my track-marked rectangle in my photography should match exactly the dimensions of a CG rectangle I created, but theyre off by a visible bit, particularly in the horizontal dimension (see Image 1)
- The Camera Calibrator yields a focal length and sensor size that dont coincide with my real-world input. (Focal length should be 50mm, sensor size should be 23.5mm, instead, I get 80.71mm and 36mm, respectively) (see image 1 & 2 side by side)
After seeing this, I cloned the camera, deleted the tag on cloned camera and corrected the focal length and sensor size settings, then moved this new camera along its local axis vector to see if my reference geo would line up any better with the markers in my photography. The results are similar, but similarly imperfect, so its making me wonder which is more physically accurate; the camera whose settings were created by the Calibrator, or the one whose settings and position I dialed in manually. Wondering if anyone out there has dealt with the same little conundrum and how you mightve resolved it. The camera I shot with was a Nikon D7100.
I also wonder whether having all the real-world dimensional info is more a hindrance than a benefit. Assuming I do manage to get highly accurate settings and transform info for the camera itself, will the Motion Tracker toolset be able to give me anything more than an approximation of moving planes? Will having exacting dimensional data for the product Im trying to track actually help me get a highly accurate match?
This is quite long & winded but I do hope someone can give some pointers either on specifics, or general approach, or both.
Thanks ahead of time to all responders!