PDA

View Full Version : AUTOMATIC Enhancement of Video with Photos (resolution, dynamic range, etc)


gamedeveloper
08-17-2008, 03:34 AM
I just pulled this link off Engadget.com.

http://grail.cs.washington.edu/projects/videoenhancement/videoEnhancement.htm

Link includes extensive video demonstration of their technology. Forgive me if this has already been posted.

yenvalmar
08-17-2008, 05:12 AM
you still need to take still photos so we cant use this to fix any old existing video :(

Syndicate
08-17-2008, 07:22 AM
This is probably the most interesting tool I have seen in development.
Im sorry but the previous comment is ridiculous... how can you possibly complain that you cant improve footage you already captured without taking photographs?? Did you even watch the video properly?
You cannot magically add information to a video that wasnt there in the original... so please think before posting comments.

I am amazed at the HDR Feature and would love to use this as I find even full HD video suffers from colour loss and detail features that only film could provide.

Well done guys :)

McBradd
08-17-2008, 09:06 AM
It seems to me that the video suggest that you can add information to a video that wasn't there in the original. As long as you can go back and capture accurate photos, you should be able to use this technique to improve the resolution of old films, fix exposures, etc. Just as shown in the examples.

Amazing tech.

iC4
08-17-2008, 10:07 AM
wow, just wow. this looks awesome.

_mg_
08-17-2008, 11:51 AM
just, incredible.

fgdf
08-17-2008, 01:29 PM
Looks promising. Another intresting project from guys at Washington col. Good air out there

JoshBowman
08-17-2008, 01:49 PM
This looks super useful. That last part where they just make the no-parking sign disappear was the climactic WOW moment for me. I can't wait for this to become a commercial release or maybe a plugin.

If they can take this further and have enhanced backgrounds with moving objects I think my mind will explode.....I just realised though that the moving objects might be low quality compared to the rest of the background elements.

Kabab
08-17-2008, 01:55 PM
I love all the nerdy words sounds like an episode of star trek :cool:

SenorTron
08-17-2008, 02:04 PM
Most of that video was really cool, especially the stuff like changing the photos in the frames while keeping the reflections. I was impressed by the demonstration of it but it wasn't anything I would have thought impossible.

That last scene, removing the no parking sign though...that made my jaw drop in amazement.

Szos
08-17-2008, 02:08 PM
Wow. Just wow.

I especially love how while this is essentially taking "simple" technologies like a "photo modelling" and putting it on steroids, the fact that they have integrated it so well is just amazing. Is this the same team that created that "video modeling" program? (I believe that one had a car parked in a lot and a video camara drove past it. With that data they were able to manipulate the scene to do things like remove the car or add another car along side it.)

Would be nice to see some of these technologies actually getting released to the market.

JoshBowman
08-17-2008, 02:20 PM
http://www.youtube.com/watch?v=vda2RAEuW_g

Is this the one you mean?

The above tech in the video looks like it was created at my local university (they plug it at the end) while the original posted video looks like it's created by a team in the US.

lo
08-17-2008, 02:25 PM
the greatest freaking thing since sliced bread.
does ayone know if this is or will be a dedicated hardware system or just software? and if there are any plans to go commercial anytime soon?

mim-Armand
08-17-2008, 06:37 PM
Amazing! what the hell just next step will be?!
mocap for animators and now this new tech for modelers! thought next time is directors turn!

mim,

Self-Designer
08-17-2008, 07:01 PM
This is probably the most interesting tool I have seen in development.
Im sorry but the previous comment is ridiculous... how can you possibly complain that you cant improve footage you already captured without taking photographs??... You cannot magically add information to a video that wasn't there in the original...
Actually, it's not accurate. NASA uses many pictures of the same thing to combine one much higher resolution and details picture, so you can see really nice pictures of far galaxies and so. A video has a lot of common details between the different frames (even if the camera doesn't move, it's like adding more and more information, beyond the grain).

So, I don't know if you can really get a nice HD from a small sequence of PAL/NTSC, but I bet you can enhance the quality just by comparing the different frames.

yenvalmar
08-17-2008, 07:05 PM
obviously if you have no still photos, such as you are working with old video that you would like to edit, or video shot on a distant location by some client, that is why it becomes less cool that you need still photos taken at the same place..

mim-Armand
08-17-2008, 07:30 PM
btw the developer (or one of em) has some comments on their video on vimeo:
http://www.vimeo.com/1513129
his nam is "Pro"!

Unfortunately, all the software written for this project is research quality code. There is no UI, no error checking, and it's slow. I need to put in the engineering effort to create a user-friendly system around the research software and release it as an open-source beta. But that is going to be hard to do with the new research projects I'm working on.

However, the details on how to implement the software/algorithms have been published in an EGSR paper which is available from the project website. Ideally, a product team like Adobe would carry the torch from here on and create a professional quality implementation of the ideas presented in this work.

This work comes from academic research. It's not patented. The technical details are open for all to read. I would be happy if some community took on this work.

In fact how about I release my code under gpl license this weekend. I cant release Noah's structure from motion code. Though he plans to release it soon too. So dont expect the code to be an usable product. But it can serve as a great reference for people who want to implement these algorithms.

pixolpower
08-17-2008, 09:18 PM
wow masking an entire pole :D this is magic :D

Syndicate
08-18-2008, 12:32 AM
Actually, it's not accurate. NASA uses many pictures of the same thing to combine one much higher resolution and details picture, so you can see really nice pictures of far galaxies and so. A video has a lot of common details between the different frames (even if the camera doesn't move, it's like adding more and more information, beyond the grain).

So, I don't know if you can really get a nice HD from a small sequence of PAL/NTSC, but I bet you can enhance the quality just by comparing the different frames.


That is true to a degree... However Its not so much the grain but the resolution and things like washed out colour (overexposure) and motion blur that make extracting details very difficult. Either way my argument still stands.. you cannot "add" detail that was not there. I dont mean enhance. People have tried fractal resizing etc to make low res images appear less blurred when they are resized. Also NASA take more than one image because of things like subject distortion due to environment at a particular point in time.

Unless I zoom into my subject, the amount of detail gained by taking successive shots will be minimal. It will merely move the colour data on the ccd from one location to the next.

Syndicate
08-18-2008, 12:51 AM
Also just wanted to point out one more thing....

Its for STATIC objects in video only. So while it wont be that much use to most film applications, it will definitely be one of those tools that can speed up touch ups on "parts" of film that would otherwise need tedious roto.

Be sure to check out all the other cool projects

http://grail.cs.washington.edu/projects/





Quote "obviously if you have no still photos, such as you are working with old video that you would like to edit, or video shot on a distant location by some client, that is why it becomes less cool that you need still photos taken at the same place.."

Actually if you watch the video, you can use other textures not present in the original footage. So in theory you can enhance an old video bit by bit. Sort of like matte painting.

I think that is more fun anyway and artists keep a job ;)

gjpetch
08-18-2008, 01:47 AM
Also, check this out:
http://research.microsoft.com/unwrap/

Kel Solaar
08-18-2008, 10:30 AM
This is awesome, I would like to see some HD output. This project will enable some crazy stuff in compositing.

Self-Designer
08-18-2008, 06:17 PM
Actually if you watch the video, you can use other textures not present in the original footage. So in theory you can enhance an old video bit by bit. Sort of like matte painting...

Wow, that's could be a great idea! Thinking about toon, painted used on videos... Also could be interesting seeing old film enhanced or even new films getting faked older (but in that strange method which might bring a different result). :drool:

I wish someone has tighten all those 2d-3d-video analysing tools to bring a single environment for calculations from rotoscoping, video enhancements and stylishin, 2d->3d tracking and back and dunno what else...

frogspasm
08-18-2008, 08:57 PM
Did anyone notice that the guy actually posted the source code to this project?

It would be so cool to see someone massage this into an open source application.

RockinAkin
08-20-2008, 04:23 PM
The future of VFX work is certainly going to be interesting. ;)

yenvalmar
08-22-2008, 05:56 AM
[QUOTE=Syndicate]

Be sure to check out all the other cool projects

http://grail.cs.washington.edu/projects/


wow some real gems in there! thanks for pointing out the link. i especially thought this one is interesting-
http://grail.cs.washington.edu/projects/realface/

kookookrayzy
08-22-2008, 06:44 PM
This looks really promising, for editing news footage to sway public opinion. And I like that the narrator sounds like Mandark from Dexter's Lab!
Really though, these guys are on the cutting edge, we're going to be seeing this used a lot in the near future.

CGTalk Moderation
08-22-2008, 06:44 PM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.