Right On…Really cool

very interesting stuff.

great utility

Wow… That’s interesting. Not sure what to say about it either, but I look forward to utilizing those algorithm based depth maps and such to enhance my work.

amazing!

thats so amazing. My favorite would have to be the library shot and the camera stabalization.

Wow! This is the future. I love all these technical stuff like this or automatic 3D scene creation from one photograph etc.
Especially I loved the motion stabilization. It was even better than a dolly shot. Almost artificial, like a 3D model. The downside of this technology is that I have to make photos of everything I shoot, which almost doubles the filming time…

It is really impressive what people can do I hope this technology comes to the marked in a relatively low price soon in the future! Maybe Adobe should give them a big bag of money, and release it with the Creative Suite package? ;P

looks fun

This stuff is interesting. Awesome

this is amazing, i think this technology will be used a lot in the future

Yeah, I saw this a while back on Motionographer. I watched it 3 times I was so amazed. My favorite way to utilize this technology would be some cool stuff with the painterly effects. Like a live action version of Van Gogh’s “Starry Starry Night.”

Too bad you can’t have anything moving in the shot like a car or person. But still really amazing stuff.

mmmm.. a new science?!

So how do you incorporate this into AE?

One of the person (Aseem Agarwala) behind this technology is now working in Adobe so I hope we might see this thing very soon in After Effects or Photoshop.

sorry, ignore that above me

So this must have some kind of inherent 3D tracking in order to guess depth.

Very cool. Interesting how so many solutions for video are converging on 3D technology.

I don’t think it is anything commercially available but rather a study. Although I’m sure if it is viable something may come of it.

“The downside of this technology is that I have to make photos of everything I shoot, which almost doubles the filming time…”

Well I can certainly imagine someone creating a still camera that would attach to the video camera automatically taking pictures. If they could get that to work, we could see this technology in consumer video cameras before long.

What system/software is used to achieve this?

Thats incredibly useful, complicated but i see the obvious benefits. eagerly awaiting your 3D TRACKING tutorial andrew. Good Job, Your tutorials are increasingly helpful.

Incredible.

lol, so fascinating that i forgot why i opened my browser.. ah i remember now… lol.

That looks amazing. It produces very good results!

The object removal was flawless …Thanks for sharing this.

fantastic.they finally merged the two

amazing!!!! how they did the video with the parking label?
which program they use??

Credibly the most incredibly incredible feat of incredible incredibility.

amazing, thank you for sharing. I hope somthing useful comes from this technology.

Fascinating stuff. Technology pushing the envelope even further. This will definitely be available in the near future. All of our current applications started out this way. I myself cant wait for something like this to available commercially on some level. This will only offer artists more tools to get the look they after. So long to throwing out poorly shot video. Now it may have life after all…..

I saw this side about a year ago.

At that time some people on the Web where arguing that the Fx shown are fake…

To me the contents on the side havent change since that…

I mean, Im just wondering why big companys haven´t already bougt or at least are interested in the technology

One thing of note, is the reference images require stereoscopic photography, not just a standard still camera, but a specialized setup…so, not quite consumer friendly yet…but think about the still image sensors on a standard vid cam, always quite high res…imagine a consumer cam that shoots vid, but takes a high res still every 20 frames or so, and stores it separately, this could theoretically be processed and ready in-cam, as you shoot, so the information is ready to process by the time you load it up in your program of choice…anyway, certainly fascinating stuff….on a related note, Microsoft is working on similar themed technology (pretty cool also). Have a look here:
http://www.youtube.com/watch?v=mNLx9pclMKU&feature=related

This is a very interesting technology, hope to see it soon on the market, it will save lot of time (and money) in a project.
Once a again thanks to Andrew for find this kind of info.
Waiting for the new tuts.

I actually “Stumbled” upon this the other day, it is very awesome, like andy said, i dont think it’ll be commercially available for a while, but once the technology starts picking up and others start to emulate, or they start making a marketable software and/or hardware product, yea…… that’ll be fun! :p

I guess you could turn all footage into HD

You can do this with Photoshop CS3 on the cheap. If anything, why are you shooting over exposed footage to begin with, go out and buy some flags light the thing.

Ahh, Wow! Interesting development. I can see the possibilities with applying effects to the photos first.

Cool!
they must develop a software.

wow, that is awesome, thanks for sharing

don’t have any other words, i am ‘’speechless”

I think it’s kind of credit work from FX & animation university. It’s cool. But, with VC tutorials, we can do simmilar things

by Allen Ellis
“Well I can certainly imagine someone creating a still camera that would attach to the video camera automatically taking pictures. If they could get that to work, we could see this technology in consumer video cameras before long.”

I already know a professional photojournalist who does just that. He has a small HD video camera attached to the tripod collar on his lense what ever he is taking photos of he is also taking Hi-Def video. A cool little tip I picked up from other pros.

That looks so awesome. I am also currently studying that. It looks pretty advanced. I think you guys are interested in this too. http://www.vimeo.com/1513186?pg=embed&sec=1513186

I really love this one..its from the same makers that andrew kramer linked us.

Enjoy

somebody here reads engadget, huh andrew

Holy CARP!

I will never trust a goverment vid clip agaian seeing what they can do with this stuff…
http://uk.youtube.com/watch?v=FgzNfOXW_CM

interesting.

Microsoft have 3 guys on this
Adobe have 1 guy on this.

I don”t know, but i reckon they’ll be looking to make this os easy to do that they could make it comercially available. All the guys on the project have an experienced background in bringing complex ideas into a userfriendly format.

I predict this will be ot in the next 5years in a simple but effective form. Could be a collaberation between adobe and microsoft.

Its a very creative use of 3D motion tracking (or at least something similar to it).
I wish them good luck with it, pretty interesting!

vbCrLf.

Would be cool Plug In for my AE =]

maybe a fake? the flower shoot looks to unbelievable imho.

*correction shot not shoot^^

wow…i love the depth finder function…that is amazing…
i would hate to see the math that goes into that…

Awesome!!! Thank you for the video.

Well, check out section 7.1.1 of the paper: “The current processing speed of our system is quite slow with five
minutes being spent on each video frame (resolution: 853 x 480);”. Now think a 200 frame shot in 4K. That’s about 20 minutes per frame if your’re very lucky (and processing time scales linearly), which means 4000 minutes, or 66.66 hours, or 2.77 days. We’re going to need very lose deadlines, even if they manage to optimize the algorithm for better performance.

I downloaded their sample code and looked into it, and while I have seen much worse, the code is not necessarily very easy (nor very well suited) to be integrated into an AfterEffects or OpenFX plugin as it is right now. There are absolutely no comments in the sources whatsoever, but hundreds of different code files with zillions of utility classes and functions and third party libraries, so it’s not very easy to figure out what’s really going on and which part does what and is or can be used for what etc. So as much as I regret it, it’s just too much work for me to make an AE plugin out of the reference implementation. And that’s without counting the licensing nightmare that comes with all the third party libs. We’ll have to wait for a company like Adobe or Autodesk to help us out.

But while we’re on the subject of research papers: I found the following Siggraph 2008 paper on an adaptation of last year’s seam carving algorithm for video processing quite interesting. One of the authors works for Adobe now, and the algorithm itself is much less complex than the one Andrew linked to (I implemented the still image version of the seam carving algorithm last year as a spare time project, see my website link if you want to try it for yourself), so I think there is a fair chance that this is going to be in AfterEffects/Photoshop CS4.
http://www.youtube.com/watch?v=AJtE8afwJEg

Peter

Wow! I really liked the stabalizer and the removing of the sigh!

Very interesting, I bet maaltanon will dig this!

*maltaannon… whoopsy!

Woah! These concepts would be excellent for commerical and indy movie projects.
I love it. AS the person before states the code may not be as efficient as others with development and refining, plus a licensing agreement who knows how much better the software can get. For now the feat is amazing and welcome in all facets of the post universe. Thanks Andrew for pointing the way.

COOL check out each person in that project. all are far far far far futurestic BrainY guys. i too wanna move out!.

i found more than this invention. a croma without a green, blue or red screen. its just ausum!!!

First i thought, that is the direction that leeds to FX they use or better show in movies like Running Man (where Arnold´s face has been replaced digitally by clicking on ONE button only!)
But at the other hand i think a lot of the shown examples are possible with AE, too (remember: they are using still photographs for enhancement) or some special tool actually availible. I love such special tools a lot and they are making my live easier. Time will tell if it´s easier to enhance with that tool or still AE. But normaly you will get bad original footage in reality from private movie makers who will not (be able to) pay for enhancement of their footage. It´s the reason why they make their movies on very low budget cameras.

uhh… look at this:

youtube.com/watch?v=WRS9cpOMYv0

that is or could be a great tutorial – isn´t it Andrew?

It´s so amazing but will be easy if you are teached by VC

just watched the video demo. blew my mind! this could save a lot of work. I also felt a sudden drop my IQ after watching.

@ bijandisco

If I remember correctly, Andrew posted this on the blog some way back.

Also, on Andrew’s / Creatice Cow’s Serious FX Training DVD it has a whole tutorial on making groups of people from one person (Sam dancing in this case). Can easily be done, if you have the motivation and patience to do so

@ thomas_017 i meant that is easy to do for all of us vc students, but didn´t know andrew posted it some time ago.
My point is: that looks so amazing but is so easy to realize

you are awesome,
im getting into expressions,
and you are so right,
the power of controling my stuff ibecame much better now;
you should get payed by adobe too ^^ they cant show me their app like you did.

please continue with this,
this is much better then what ive learned at school:-)

best regards from germany

I’ll surely keep an eye or two opened for that

Hey Andrew, if you like that, you will LOVE this…

http://www.acvt.com.au/research/videotrace/

I can see the Department of Defense and Homeland Security putting this to good use to cover up and doctor footage in favor and against our nation as well as those abroad

Wow, the VideoTrace link is amazing … Really takes the hard work out of modelling 3D objects from just XYZ plane photographs.
Anyone know if there is already a commercially available version of this somewhere?

Those damn algorithms. I never liked math. Still don’t.

wow, this is really amazing. if you can do this now, what can you do in the future? they just have to build al photocamera in a videocamara that makes 1 photo a sec. ore something. you will have endless oppertunities in the editing whit just one handeling.

but when can you get this sofware, what is the name of it?

@Rob Moffet: That’s actually pretty simple to do manually, it’s just a bit more work. Just load your footage into a 3D tracking application (Boujou, Matchmover, flame*, whatever), then load the camera data that is generated into your favorite 3d app (Maya, XSI, 3ds Max, …) and load your footage as a background/reference. Then start modelling. If your tracking app can export locators for arbitrary points (or the point cloud it used for tracking), make sure to take advantage of that so that you already have the z-depth of the key points of the model so you only have to estimate the position of the vertices inbetween, essentially reducing the modelling problem to a drawing-by-numbers exercise. Move around in time and adjust and extend your model where necessary. Then take screenshots of your footage at key points in time (i.e. approximate front view, side view etc.), duplicate your camera and use it as a projector to create a texture for each view via simple camera projection. Stitch them together in Photoshop and apply it to your model.

On the other hand, if you have a tool that can generate an approximate depth map for your footage (via optical flow or whatever method) and you have the camera data from the 3d tracking app, writing a simple program that can do the modelling is quite simple since it could easily “unproject” the 2d screen coordinates you click with your mouse into 3d vertices for your model. You could then export a simple 3d .obj model or something like that for use in a 3d app or a compositor that can handle 3d geometry (like Nuke or flame*).

Peter

if ur talking about interesting technology video please take a look at http://www.istartedsomething.com/20080814/uni-washington-microsoft-research-yet-another-mindblowing-3d-photo-viewer/

Video Trace!!! It’s very cool!!!
http://www.acvt.com.au/research/videotrace/

wow… why didn’t YOU think of that Andrew?

jd

reminds me of the photosynth technology

video here

http://www.youtube.com/watch?v=s-DqZ8jAmv0

This has some great applications for video HDR and he sure does talk pretty…

Since everyone seems to be putting links up.. This is really cool

http://www.youtube.com/watch?v=d_A3GM5C9ps

Research Studies
http://www.graphics.rwth-aachen.de/index.php?id=55

Kinda seems like they took matchmoving software one step further.
Cool as hell though!

hey andrew can you do this to.. or this is for professional…