I have solved the scene and everything looks great (FG/BG lots of parallax). I've separately solved an object in the scene that I want to attach geometry to. And its position and rotation is perfect too.
But when I attach geometry, its position drifts more and more out of sync as the timeline moves in either direction. So it looks like a scaling issue. I guess my tracked object feature cloud group is at the wrong scale relative to the scene.
Any tips on how to correctly size my object's feature group? (other than endless tweaking and testing?)