CV-AR: Easy facial performance capture with iPhone X

Become a member of the CGSociety

Connect, Share, and Learn with our Large Growing CG Art Community. It's Free!

REPLY TO THREAD
 
Thread Tools Display Modes
  06 June 2018
Amazing plugin! I borrowed a phone from my collegue, watched tutorial on Cineversity and 2 minutes later I had my face animated inside Cinema!
Congrats to all involved in the development!

Anders
 
  06 June 2018
I just saw this yesterday. Gotta say, I'm beyond impressed.  It makes justifying the insane price tag a little easier if you can use it for actual mo-cap work!
 
  06 June 2018
Originally Posted by skeebertus: A technology giant like Apple cannot be depended upon to provide iPhone X functionality forever - they will pull that 3D camera off the market as soon as it doesn't make any money.
This plugin can't replace high-end professional systems, nor is it meant to. But it does inexpensively allow for surprisingly high-fidelity facial motion capture, and might find a spot in some professional workflows.

I think it is wise to be concerned about whether Apple continues to support this going forward, but I guess I am more hopeful than you. This relies on ARKit and the camera used in FaceID, which are two technologies Apple is betting heavily on.
 
  06 June 2018
Originally Posted by skeebertus: If you really need facial motion capture, open your email client and write to a producer of Stereo 3D cameras about it.

A technology giant like Apple cannot be depended upon to provide iPhone X functionality forever - they will pull that 3D camera off the market as soon as it doesn't make any money.

It is the opposite with manufacturers of Stereo 3D vision cameras - they are desperately looking for attention-grabbing real world applications of their Stereo 3D product that they can show off.

Send them an email detailing your needs and what you would use the product for - for example, animating CG characters for Youtube shorts.

If you are dealing with a good company, and more than one person inquires about the same thing, chances are that the company's management will be notified of the request.
Every other manifacturer can not guarantee a functionality forever. Also it's far more likely that manufacturers of stereo 3D product "desperately looking for attention" will fail faster than Apple if they struggle to sell a few pricey item while iPhone sell hundreds million of unit every years. On the other end with the iPhoneX you can use this now and for the foreseeable future, and they'll probably extend the X sensors to the entire line very soon and for cheaper price. Honestly seems just another of your attempt to blame Apple products.
__________________
www.3drenderandbeyond.com
www.3dtutorialandbeyond.com
www.facebook.com/3drenderandbeyond
 
  06 June 2018
Originally Posted by sirio: Honestly seems just another of your attempt to blame Apple products.


So you would rather spend 1000 Dollars on a dinky phone with a pseudo-3D IR dot camera with a tiny range - manufactured in China for less than 400 Dollars - than spend 500 - 800 Dollars on a real dual-lens Stereo 3D camera that can scan in 3D for 20 feet or more?

You do realize that a real Stereo 3D camera can be used to scan 3D objects and locations with surface texture, may provide multi-person full body-and-face mocap at the same time when paired with the right algorithms, and will let you shoot green screen type work without needing an actual green screen in the background if the camera can spit out a Z-depth map?

Your argument confirms my long held view that Apple users are borderline technology illiterate.

You have the opportunity to support other more innovative companies that may give you a far more powerful 3D product that can do much more for the same price, but you feel compelled to throw your money at something inferior "because it has the Apple logo on it".

Precisely this is the reason why almost everything Apple makes is extremely mediocre from a technology standpoint.

You see, Apple does not have to try very hard at all when tens of millions of Apple fanboys buy anything that has the Apple logo on it at any price Apple sets, regardless of whether it is good or not.


So if you want mediocrity - give your money to Apple like tens of millions of other naive customers. Apple are shameless. They'll gladly take your money and sell you a very ordinary product.

But if you want something that gives you far more 3D content creation firepower than Apple's overpriced crap, go and support a company that is actually trying harder than Apple.

Give your money to a company that is actually trying to be at the cutting edge of what is technically possible, not a profiteering mega-corporation with a fruit logo.
 
  06 June 2018
Hi skeebertus,

CV-AR gives you the ability to use an existing device that you might already own, for another useful purpose. And if you are considering buying a new phone at some point then it may make you consider getting an iPhoneX so that you can do this one additional thing with your spending power. 

Nobody is suggesting that it will replace purpose built solutions. Especially for large scale scanning and motion capture.

It's a phone, that just so happens to have a useful sensor in it that allows for some interesting solutions. I hope that more phones down the line also include such sensors and that one day we will all have a portable scanning and capturing device right in our pockets. 

Regards,
Kent
 
  06 June 2018
kbar is the voice of reason, and brings to light the key point here which skeebertus appears to have missed completely.  Add on to that with the notion that people who want to dip their toe into this kind of thing, and happen to be one of the millions of people who have this device (which practically no one ever pays for more than half the retail price for - btw) you have a convenient point of entry.  If your interest subsequently grows, and the nature of the work you do in that space grows congruently, then you'll plunk down on all the kit necessary to do this at a high level.  So, ultimately things like this will serve to help those companies which make that kit by introducing people into the space who would have never previously done so owing to the cost of entry.

I would say that your rabid, nonsensical hatred for a thing based upon the logo it carries is just as bad, and has blinded you to an equal degree as these 'fan-boys' you decry.  Pot, meet kettle.

And, I'd watch it with the broad insults.. it only serves to weaken your overall position to call me - someone who happens to enjoy the Apple ecosystem and sees plenty of mediocrity among its competitors - a technological illiterate, and just makes you come off as ignorant blowhard.  Were you to say that to my face - depending on the day - the next thing out of your mouth might well be a couple of bloody teeth, as insulting my intelligence just happens to be a hot button of mine.  I suspect that you only say these kinds of things when protected by the relative safety of internet anonymity, but may I suggest you try speaking to people here as you would if you were across the table from them.  Ultimately, when civility rules, we all win. 
__________________
Mac Pro 12 x 2.6 GHz 64GB Quadro K5000
OSX 10.10.4
MacBook Pro 4 x 2.3GHz 16GB GeForce GT 750M
OSX 10.10.5
C4D R18 Studio/CC/VizRT
Will's Works
 
  06 June 2018
Originally Posted by GruvDOne: kbar is the voice of reason, and brings to light the key point here which skeebertus appears to have missed completely.  Add on to that with the notion that people who want to dip their toe into this kind of thing, and happen to be one of the millions of people who have this device (which practically no one ever pays for more than half the retail price for - btw) you have a convenient point of entry.  If your interest subsequently grows, and the nature of the work you do in that space grows congruently, then you'll plunk down on all the kit necessary to do this at a high level.  So, ultimately things like this will serve to help those companies which make that kit by introducing people into the space who would have never previously done so owing to the cost of entry.

I would say that your rabid, nonsensical hatred for a thing based upon the logo it carries is just as bad, and has blinded you to an equal degree as these 'fan-boys' you decry.  Pot, meet kettle.

And, I'd watch it with the broad insults.. it only serves to weaken your overall position to call me - someone who happens to enjoy the Apple ecosystem and sees plenty of mediocrity among its competitors - a technological illiterate, and just makes you come off as ignorant blowhard.  Were you to say that to my face - depending on the day - the next thing out of your mouth might well be a couple of bloody teeth, as insulting my intelligence just happens to be a hot button of mine.  I suspect that you only say these kinds of things when protected by the relative safety of internet anonymity, but may I suggest you try speaking to people here as you would if you were across the table from them.  Ultimately, when civility rules, we all win. 

Agreed. Skee’s post was out of line. 

There are good arguments for—and against—any Apple product. The facial capture feature is potentially a meaningful value add...esp. for someone already invested in the Apple ecosystem.

Back to it: the plug-in looks super. Back to it: the plug-in looks super: elegant, easy, useful.
__________________
C4D R19 Studio, MODO 902, VRAY, Octane, Cycles. PC/Mac.

Last edited by IceCaveMan : 06 June 2018 at 07:25 AM.
 
  06 June 2018
Duplicate Post
__________________
C4D R19 Studio, MODO 902, VRAY, Octane, Cycles. PC/Mac.

Last edited by IceCaveMan : 06 June 2018 at 07:25 AM.
 
  06 June 2018
It's not just the hardware, it's the software that goes behind it that makes it worthwile, because unless the stereolabs Zed camera for instance has a facial mocap plugin built for it, it's just a 3D scanner. It's of no use for facial mocap. 

I also find the IphoneX and most Apple products stupidly overpriced, but the people behind FaceID is the same people behind Faceshift (Weise et al.), which was the best facial mocap I've ever used (ease of use / quality) before they got bought out.

How does the IphoneX + plugin compare to the original faceshift, I don't know yet, but it could be quite decent, and surely the next "entry level" iphone will have the faceID too (at "only" $700+ dollars, meaning you'll lose an arm but keep your leg).

If you look at alternatives you have facewear, which is way more expensive (even the cheaper iclone version is listed at $1190) and still needs some decent cameras on top of it or even a full rig if you want to make it wearable, or Facerig studio, much cheaper (around $300) but webcam based only and with less control.

So when you factor in all the costs to have a decent and portable facial capture solution, the ludicrous price of the iPhoneX as just a luxury phone, suddenly looks more acceptable as a professional tool, which you use to bill money, and can also use on a day to day basis.

It's still stupid money for anyone else, but for a 3D professional... it just got much more interesting. And mind you, my current phone is a $80 android model, so I'm really NOT an Apple fanboy.
 
  06 June 2018
Originally Posted by skeebertus: So you would rather spend 1000 Dollars on a dinky phone with a pseudo-3D IR dot camera with a tiny range - manufactured in China for less than 400 Dollars - than spend 500 - 800 Dollars on a real dual-lens Stereo 3D camera that can scan in 3D for 20 feet or more?

You do realize that a real Stereo 3D camera can be used to scan 3D objects and locations with surface texture, may provide multi-person full body-and-face mocap at the same time when paired with the right algorithms, and will let you shoot green screen type work without needing an actual green screen in the background if the camera can spit out a Z-depth map?

Your argument confirms my long held view that Apple users are borderline technology illiterate.

You have the opportunity to support other more innovative companies that may give you a far more powerful 3D product that can do much more for the same price, but you feel compelled to throw your money at something inferior "because it has the Apple logo on it".

Precisely this is the reason why almost everything Apple makes is extremely mediocre from a technology standpoint.

You see, Apple does not have to try very hard at all when tens of millions of Apple fanboys buy anything that has the Apple logo on it at any price Apple sets, regardless of whether it is good or not.


So if you want mediocrity - give your money to Apple like tens of millions of other naive customers. Apple are shameless. They'll gladly take your money and sell you a very ordinary product.

But if you want something that gives you far more 3D content creation firepower than Apple's overpriced crap, go and support a company that is actually trying harder than Apple.

Give your money to a company that is actually trying to be at the cutting edge of what is technically possible, not a profiteering mega-corporation with a fruit logo.
Your hate has blinded you as already said by other. For the record I don't own an iPhoneX and most of my recent hardware purchase are made on Windows hardware. It's just silly to think that every thing Apple made is bad and your assumption that Apple users buy whatever the company sells it's just plain wrong. There are stupid fanboys for sure(as well as Apple hater like you), many other choose one brand or another wisely and only after evaluating all pros and cons.
Also, you have totally miss the point of this post... nobody is comparing an iPhoneX to other high end dedicated solution.. it's just a phone but if can do also something useful for 3d then where is your problem? I'm 99% sure if this was achieved on an Android phone this conversation never exist.
__________________
www.3drenderandbeyond.com
www.3dtutorialandbeyond.com
www.facebook.com/3drenderandbeyond

Last edited by sirio : 06 June 2018 at 10:33 AM.
 
  06 June 2018
“... nobody is comparing an iPhoneX to other high end dedicated solution.. it's just a phone but if can do also something useful for 3d then where is your problem? I'm 99% sure if this was achieved on an Android phone this conversation never exist.”

This.
__________________
"...if you have faith as small as a mustard seed... Nothing will be impossible for you."
 
  06 June 2018
Originally Posted by NWoolridge: Here's another example, with some idling/reaction behaviour:

https://www.dropbox.com/s/btsbbn7nu...dling1.mp4?dl=0

Super! How long did that take you?
__________________
C4D R19 Studio, MODO 902, VRAY, Octane, Cycles. PC/Mac.
 
  06 June 2018
Originally Posted by IceCaveMan: Super! How long did that take you?
The major time spent was in the rig setup... now that the rig is working, swapping in a new performance takes a minute or two. These are all on the same rig, which has morph targets for various characters, male and female.
 
reply share thread



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
CGSociety
Society of Digital Artists
www.cgsociety.org

Powered by vBulletin
Copyright ©2000 - 2006,
Jelsoft Enterprises Ltd.
Minimize Ads
Forum Jump
Miscellaneous

All times are GMT. The time now is 03:48 PM.


Powered by vBulletin
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.