R19 Scene reconstruction with dynamic objects?

Become a member of the CGSociety

Connect, Share, and Learn with our Large Growing CG Art Community. It's Free!

Thread Tools Display Modes
  08 August 2017
R19 Scene reconstruction with dynamic objects?

will it be possible to reconstruct a scene frame by frame to get dynamic meshes from moving objects? Or is it restricted to static scenes?
  08 August 2017
Matchmoving software tracks "identifiable points" in footage regardless of whether they are on a static environment, or on a truck moving through the otherwise static scene.

Algorithms like SIFT and others are used to track the movement of clusters of similar identifiable pixels over successive frames.

I believe that some matchmoving software can indeed let you track moving objects instead of "static enviroments", and even track both at the same time, isolating the moving truck when you want to.

If Maxon's pixel tracker can do this, then being able to construct a mesh for dynamic moving objects shouldn't be a big deal.

How well that works in real life is another matter - a walking person for example might create a mesh that changes in many interesting ways each frame, including the arms occluding the upper body, and you might need to track a huge number of points where that person is to get a usable mesh out of the process.

Whether R19 can do any of this or just static environments I don't know.

But basically, once you have tracker-points-to-mesh working decently and enough tracking points on a moving object, it shouldn't be a big deal to see what points are "static" and "dynamic", and generate two different meshes for them.

In essence, you are just separating static points from moving points and then running the mesh generation twice.

Unless of course Maxon's solution builds meshes over many successive frames. In that case, a walking person might be impossible to "mesh" from tracking points.
  08 August 2017
Since R18 you can track moving objects. The scene reconstruction however will only work on static objects. What you can do though is shoot a survey shot (camera moving around the object while it is static) of the (later) moving object and reconstruct that. Then use the resulting mesh as reference geometry for object tracking in the actual shot and once it is done group it under the object tracker. Now you have a 3d reconstruction of your moving object.

Not sure if that answers your question. Subjects such as people walking around are not going to work out. Moving rigid objects are fine but any kind of morphing (bending limbs, cloth, etc) is not going to work.

Last edited by Havealot : 08 August 2017 at 09:20 AM. Reason: fixing an ambiguity & typo
  08 August 2017
Thanks guys,

the question comes down to the point if there is a single mesh carried over during all frames or if it is possible to let the system create a new polyobject every frame so a moving object would be generated into a C4D/obj mesh sequence or so.
  08 August 2017
C4D tracks only "non deforming" meshes : it assumes the scene or the objects moving through it keep their topology throughout the sequence, else it would not know if it's shrinking or moving away for instance.

So you can track a road, a car, but not a transformer.

I don't have R19, but based on that, I don't think the reconstruction can deal with deforming meshes and spit an obj sequence.

However, a moving object relative to a static camera being the same as a moving camera relative to a static object, I assume though you could track and reconstruct any object that doesn't deform over time.

For photogrammetry reconstruction, though, there are specialized tools which would be probably more suitable.
  08 August 2017
Originally Posted by zeden: Thanks guys,

the question comes down to the point if there is a single mesh carried over during all frames or if it is possible to let the system create a new polyobject every frame so a moving object would be generated into a C4D/obj mesh sequence or so.
As I said, not in one go. And it really depends what the moving object is. Rigid yes, morphing no. Tracking a transformer would work in theory by using several object trackers, one per moving part (like one for torso, one for upper arm, aso). But there is no way to track lets say a flag moving in the wind. That's simply a degrees of freedom problem. The solver needs several different views on every tracking or reconstruction point/pattern to determine their position and orientation in space. That's only possible with rigid objects.

Check out these examples. Are we looking at two identical, three-dimensional cubes? Sure...

No we are not. Only the left one is a cube. The right one is just a shape that looks like a 3d cube from a certain perspective. There is no way to determine the depth of the corner of the cube facing the camera. It could sit anywhere on a line drawn through it from the camera.

Thats why you need several view points/pattern (and parallax) to reconstruct them.
reply share thread

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Society of Digital Artists

Powered by vBulletin
Copyright 2000 - 2006,
Jelsoft Enterprises Ltd.
Minimize Ads
Forum Jump

All times are GMT. The time now is 09:06 AM.

Powered by vBulletin
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.