PFTrack Problem...Cannot solve SIMPLE shot!

#1

Hi everyone! How are you all doing?

I don’t know if this is the right place to post thus but I have been trying tot rack and solve a relatively simple shot for a very long time now and I have given it all I got at it and it just doesn’t seem to be working at all!

I have been trying to track and solve this shot that I’ve filmed myself and it doesn’t seem to be doing it correctly. Everytime I would track it it would seemingly do it well, the footage is by no-means horrible) but when I come to solving it , there is always drift and jitters on the ground plane. I can look in the Perspective or Point Cloud view and I can see that it is abit of a mess and not resembling the scene whatsoever. Not to worry I said, I originally thought this was due to the Rolling Shutter of my camera and thought I would re-shoot it with a Global Shutter instead of pouring more days/weeks into solving an unsolvable shot.

What annoyed me as well was that I also placed a good few dozen User Features throughout the shot and made them into ‘Tracking Guides’ and tracked the shot with that, but it still continuously failed at the Solving in a particular area…however what was interesting to see was that the Point Cloud was slightly more accurate but the motion path gradually got worse as the shot progressed.

I went ahead to re-shoot it with a CCD camera, quality is of course not on par with larger sensor cameras but reassuring to not see any ‘Jello’ nor was it terrible, it was definitely usable.

I have also taken the steps to:

  • Take note of the Focal Length and calculated it with the Wide-Angle lens attached.
  • Film progressively at 25p and at 1/100th shutter speed.
  • Shot my set-up on a Lens Distortion Grid (effectively correcting the overall geometry of the image)

I thought I was now onto a winner and went ahead to import the shot, setup my parameters for the Track and the Camera, undistorted the image (which looked pretty nice to be honest :smiley: ) and went ahead inputted well over 90 USER FEATURES as tracking guides and tracked the shot which went solid.

When it came to solving…once again it just could NOT solve whatsoever! Albeit it did do a slightly better job due to Global Shutter (dream finally came true) but it just for the love of God not get through all the way till the end. I have even lowered the threshold to remove/reduce ‘red trackers’ and even raised it up to use all the trackers but I either get a solve with bad solution or a failed solve completely. Not to forget to mention I manually removed trackers from ‘highlights’ from shiny surfaces from cars that would result in giving me ‘false positives’. What was interesting as well was that when I would solve without the trackers, it would work but still comes out incorrect.

I have been at this for sooo long I just really don’t know what to do anymore, and I really do need to know what is going on. I’m sorry if this is quite lengthy but I just wanted everyone to have a good idea what I have been doing and the measures I’ve taken to get things right.

I just don’t now what I’m doing wrong and I would really really appreciate it if anyone here can help me out with this or if anyone could advise me what I might need to do to solve this.

P.S. I’ve also included a link to how the shot looks like to probably give you a better idea. I hope it helps!

http://youtu.be/VnoETqGdBc8

#2

Anyone?

Anyone at all may I ask?

I gave it another go again but once again no luck! :frowning:

#3

maybe you could try to split your shot into different parts, sovle them seperate and try to connect them then back.

because you shot is really long and all the turn arounds with little perspective change makes them hard to solve.

so you could solve parts the work right now and try to improve other later on

#4

It is not simple at all. In a place full of reflective surfaces and
asphalt you pan and tilt like hell. Looking at a static street no
matter how long will not help the software. Pingking’s advice
is valid, you can also use image manipulation to soften reflections
or highlights but i think it is shot badly to begin with.

#5

Well how do I go abouts doing that? Do I do it in PFTrack itelf or outside it?

Either way, I’m sure long shots are solvable, and does the pans need perspective or parallax in them? It’s just a simple swirl around with the camera and all the User features I made from the beginning of the shot where later added in the same places when the camera goes back to the same area, surely that would’ve helped the solution to tell where everything is placed in 3D space?

As I’ve mentioned in my original post, I have tried m very best to remove all the necessary trackers from the highlights/reflections on the cars that would result in “false positives” and doing so improved the solution for abit longer but would fail later on where there were solid or ‘good’ trackers placed.

Would any of you two would like to have a go at it perhaps? I can send over the project files if you like? :slight_smile:

#6

do it in PFTrack, just handle you one very long shot like several shots. so limit the timeline to parts where you think you would make a cut in a edit

then solve them as different shots

#7

I’m sorry, i do not have access to PFtrack at the moment and
i am not that good at matchmoving. All i can say is that 1010-1140
frames seems to be the problematic area and it can be solved as a pan only
camera, but i am not sure if adding user tracks will result to jitter, you
have to try.

#8

Hiii guys! I am sooo sorry for the late reply (once again) I literally didn’t have a computer for over a week recently as prior to this I decided to rebuild it with new parts since I was having serious problems one of my 2GB ram sticks died (went from 8GB > 6GB) and with almost everything I did it kept crashing and BSOD’ing…I couldn’t even do any fo the PFTrack stuff you lot suggested!

Now I’m up and running with an i7 4770K and 32GB DDR3 ram babaaayyy!!! :smiley:

Ahem!..onto business

Are you saying that I:

  1. Track the entire shot and solve a certain section of the timeline and delete the solution and do the same thing for another section, thus I keep doing this until I reach the end of the clip?

or

  1. (If this is even possible) Track the entire shot and solve a certain section of the timeline and set a new In/Out points and solve that and continue until I’ve reached the end of the shot?

For either one, how do I go about this, especially when I come into my 3D/Compositing software and potentially have to put them together? Wouldn’t there be a difference in camera motion since they were not solved as once shot??

This relates to what I’ve just said, could I solve a certain section of the clip as ‘Free Motion’ and another section as ‘Pan Only’ on the same timeline? Or do I have to delete the whole solution and start again for the different camera motion?

I hope this makes sense guys! Please let me know!

#9

what’s happening man, hope you’re still at this shot otherwise i’ll just be wasting my time here.

so, lets get right down to it: YOU WILL NOT BE ABLE TO TRACK THIS SHOT WITH YOUR CURRENT METHOD

The reason why this shot is so damn hard is not because of “reflective” surfaces, but it’s more like it introduces too many new features that with PFtrack’s autotrack it’ll never figure out the relationship between them. Assuming you start tracking from when you tell the dude, “don’t move”, you see part of the road, then you don’t see much of it, then you see a lot then WHOA what’s this? the other side of the street? I don’t know how to factor this in the solve.

That’s basically how the software will react when you try to solve this.

So, does that mean this shot isn’t trackable? No, it certainly is possible. But realistically there’s only two possible methods:

  1. Do what pingking said, and split up your shot. This can be done by either just solving your shot in sections, or IMPORTING THEM IN SECTIONS and treat each section as a separate shot. Now the disadvantage to this method is keeping the consistency of position and scale of whatever you are inserting in the shot. Like if you were to solve one section of the shot, do your insertion, then you have another section solved, but if the distance between certain elements of the location is solved differently, then you gotta manually attempt to keep the continuity/consistency of inserted element by eye. Which is a pain in the butt.

now if you need whatever you are adding to appear throughout the entirety of your shot, then:

  1. Track the WHOLE shot, i mean that whole 1minute and 5seconds worth. and DON’T USE ANY AUTO TRACKED FEATURES, USE ONLY USER TRACKS. it sounds like you already got a bunch of user tracks. I will say that 90 isn’t enough, i’m thinking more like 150+ trackers since there’s quite a lot of stuff. then again i could be totally off. But also it depends on WHERE you tracked your 90 user tracks. one key thing to do is make your tracks last until that feature is OUT OF THE SHOT. Like that house you go back to, track some features on the first part of the shot, then when we come back to it, continue those tracks until it’s completely gone from the shot. it may sound obvious, but super crucial. and oh, make sure your tracks are accurate, otherwise it’ll miss some key jitters that may be present in your camera move.

Well i’d love to write more, but i might be blabbering. plus i don’t really know what version of PFTrack you’re using… so yeah let me know if you have any questions

#10

taz0x, that is incredible information you gave there and very helpful as well, so I really would like to thank you for replying with that!

Most of what you’ve said is something that I am kinda aware of, I initially thought it would’ve had problems knowing what area of the scene is where, this is why I created a bunch of Manual Trackers, all of which were tracked in areas of shot and getting the same tracker to track the same area if it appeared in shot, I thought this would’ve solved it, however this somehow also caused me more problems which has ultimately confused me! I’m pretty sure I described this in my original post.

It was also really hard for me to add anymore trackers (like what you suggested) as the footage (quality-wise) made it very difficult and also the fact that there really isn’t anymore features I could use (unless I got entirely exhausted from doing so for so many weeks). Mind you also, they were all set to ‘Tracking Guide’ which would help aid the Auto Trackers give a sense of flow of how the scene works when doing the autotracking process.

  1. This is what I was really afraid of, the fact that each different solved section would provide me with with inconsistencies in the position and scale of the matchmoved, this would actually be a nightmare and I would consider it unprofessional since trying to match an inserted object from the last frame of a solution to the first frame of the other section of a solution during compositing just seems like if the whole thing was done “wrong” if you know what I mean. You are referring to the compositing stage as well right?

  2. From the get go, the whole shot was tracked, both manually and with auto-features. Manual Trackers were tracked on features from the beginning and until the end they appeared in shot, and as I’ve previously mentioned, they were tracked in areas of shot and getting the same tracker to track the same area if it appeared in shot and were made to guide the autotracking features. This somehow proved to be a problem when it came to solving the shot (as a whole of course).

My questions fro you exactly are, in the case of no.1, isnt it possible to take each solved section and place them one after the other as one continuous shot? Like in 3ds max, doing so would create 3 separate cameras with it’s own field of tracking markers (which a 3D object would need to be attached), isnt there anyway to merge them as one working solution?

#11

You could make your sections overlap
for one frame and see what happens
with the cameras at the specific point.
Maybe you could match them with
some kind of offset but it is possible
they will be in totally different positions.

#12

I can’t answer all of this right now in depth, but to sum it up, yes it’s possible to do what you described, and to make it one camera, but will it be a “working solution”? that really depends on how each section gets solved. like you can make the cameras POSITION consistent with each other by eyeballing it and figuring out how you walked around with your camera and imitating that motion. but if the DEPTH of the solve, like how far apart one car is to the house on the streets (basically distance relationship between features) is inconsistent with each solve, you’re still going to have issues.

By the way man, what version of PFTrack are you using? the super old one? or the one that uses a bunch of nodes (like Nuke)?