Insert 3D Piano model in video!?


Hi, Short story - I am a musician (guitarist) and I want to make a music video performing a song on a grand piano.

I had a nice location to shoot this, but the opportunity to film there slipped away. Basically it was nice living room with a Grand Piano in it.I’m now in another (smaller) living room in London and there are no grand piano in sight.

My thoughts are
Can I shoot the video playing a piece of wood or something (the music is allready pre-recorded anyway) and insert a CGI, 3D modell of a Grand Piano in a living room? (or in any other artificial environment, unreal engine 4 looks promising, though I suspect I will have to have a big green-screen and a larger place?)

How hard will this be to accomplish with realistic results? What steps (and what software) does this require? My knowledge of compositing and working with 3d is limited. Is it realistic that I can learn how to do this in a relatively short timespan (I’m familiar with non-linear editing and grading software)

There will be Two camera angles; First camera is moving from left to right across the room from the ”audience point of view” of the piano effectively hiding the fingers (There will be no scenes of my fingers actually playing the piano). The second camera is fixed on a tripod (may add camera movement in post). I would off course like to have the piano tracking properly in the imagery for the correct parallax effect etc. The project is a non-profit. My friend wrote a song on his iPad and now I want to surprise him with an ”eye catching” video performing the song he wrote. I already arranged and recorded the song - all thats missing is the video :wink: Attached are some images for reference.

My actual living room:


You’ll need first the piano geometry. Since you’re looking at a “relatively short timespan,” I recommend purchasing an asset from someplace like Turbosquid. A grand is usually pretty reflective, so in addition to your primary photography, also plan on shooting an HDR panorama of your apartment so you can get nice reflections and image-based lighting. The idea there is that you put the panorama on a huge sphere that surrounds the piano, and the piano will reflect the sphere, making it look like it’s in the room. If the panorama has enough detail and range in the brights, it can also create some of the light you need, although you’ll certainly want to supplement that with some additional, hand-placed lights in the 3d scene.

For 3d software, Blender should be sufficient to your needs. It’s open source and free, and there’s a pretty big community to help amateurs through the basics. The skills you’ll need to learn there are all related to lighting and rendering (assuming you find a piano model to purchase), so you can probably ignore anything about animation, modeling, and effects. Using free software will let you spend any budget you have set aside on the piano asset and equipment rental.

Completely artificial environments would be a much bigger and more complex project. For this first piece, I recommend just inserting the piano into footage.

For compositing and tracking, I recommend Fusion Studio (disclaimer: I am writing a book about Fusion). It costs $300, but that gets you not only a very good compositing program, but also a 3d camera tracker. You could instead use After Effects, which also has a 3d tracker, but I’m not sure off the top of my head how difficult it would be to export that data to Blender, which is where you’ll need it. There may also be a camera tracker in Blender. I know there was someone working on that at some point, but I’m not sure if it got done. If it’s there, then you can use the free version of Fusion because you won’t need any of the more advanced features. Natron would be another option—like Blender, it’s free and open source, but I don’t think it’s quite mature enough for general use. I haven’t looked at it in a while, though. There is also a non-commercial version of Nuke. Not sure of the limitations on that. Or you could spring for a temporary After Effects subscription.

Your notion of pretending to play a block of wood is a good one. Just make sure that it will be placed in such a way as to be completely hidden by the CG piano.

Good luck!


Hi Bryan,Thank you for taking the time to respond, much appreciated.

I bought a 3D-model at cgtrader. It’s also rigged… so I might be able to control the keys (hammer and mechanism) from the actual midi that is played in the DAW. (a bit overkill, but still… its cool)

The HDR panorama was a great tip, didn’t even think of that. I will play a huge roll in the realism of the final output.

I think I’m going with Cinema 4D, I’ve heard its relatively easy to learn compared to Blender. For compositing-software I think I’ll go with After Effects. It has a neat integration with Cinema 4D from what I understand. I realise that learning the node-based workflow in Fusion is easier one you get the hang of it. I have worked a bit in Davinci Resolve and like it very much.

I’ll make sure to post the result ones it’s done :wink:

Thanks again, Calle