Interesting Video Technology

Not really sure what to say about this video but it’s pretty cool. The video discusses specialized methods & technology for enhancing altering video.

Check it out

POSTED:
6 years ago
RECENT POSTS
Clean Lighting with Element 3D V2
Here are a few experiments created in Element 3D V2 with simple lighting and Ambient Occlusion. In these examples I’m using the Ray Traced AO which is a bit slower but creates very nice results. The other important setting I like to use is called “Lighting Influence” under the Physical Environment. This setting helps blend the […]
Technical Details about E3D V2
Now that we have released the first look at Element 3D V2, you can see that this is not a small update to our plug-in. It is a massive overhaul with many big features. So, I thought I would take a little time to answer a few technical questions. NOTE: Pricing information and upgrade paths […]
Element 3D V2 Now Available! *Updated
Element 3D V2 is Now Available! We’ve been working on this plug-in for almost 2 years and we’re excited to release it! I’ve had to keep quiet for so long and now I can’t wait to show you what is possible! Top Features: Shadows and Improved Ambient Occlusion Dynamic Reflection Maps 3D Noise and Deform […]
The Day Before Element 3D V2! (Now Available!)
It’s been one incredible ride but on Tuesday Dec 2nd, we will be releasing Element 3D V2 but first check out some features we’ve been holding back! There is also a brief demonstration of each new 3D pack at the end of the video! Element 3D is Now Available If you can’t wait to watch the video… check […]
81 Comments

Wow, that's some pretty cool stuff.
Eshita
4 years ago
Okay i'm gonna give a frank opinion now.............i like tech stuff like this but....................i couldn't really stand the video after the first minute...............i know one thing for sure.............if Andrew had made that video instead of that person............i would have loved it.
Anyway the video was still informative and cool.
and1
6 years ago
hey andrew can you do this to.. or this is for professional...
Tem
6 years ago
Kinda seems like they took matchmoving software one step further.
Cool as hell though!
6 years ago
Since everyone seems to be putting links up.. This is really cool

http://www.youtube.com/watch?v=d_A3GM5C9ps

Research Studies
http://www.youtube.com/watch?v=d_A3GM5C9ps
digidave
6 years ago
This has some great applications for video HDR and he sure does talk pretty...
hungryhipp00
6 years ago
reminds me of the photosynth technology

video here

http://www.youtube.com/watch?v=s-DqZ8jAmv0
6 years ago
wow... why didn't YOU think of that Andrew?

jd
Marcio
6 years ago
Video Trace!!! It's very cool!!!
http://www.acvt.com.au/research/videotrace/
6 years ago
6 years ago
@Rob Moffet: That's actually pretty simple to do manually, it's just a bit more work. Just load your footage into a 3D tracking application (Boujou, Matchmover, flame*, whatever), then load the camera data that is generated into your favorite 3d app (Maya, XSI, 3ds Max, …) and load your footage as a background/reference. Then start modelling. If your tracking app can export locators for arbitrary points (or the point cloud it used for tracking), make sure to take advantage of that so that you already have the z-depth of the key points of the model so you only have to estimate the position of the vertices inbetween, essentially reducing the modelling problem to a drawing-by-numbers exercise. Move around in time and adjust and extend your model where necessary. Then take screenshots of your footage at key points in time (i.e. approximate front view, side view etc.), duplicate your camera and use it as a projector to create a texture for each view via simple camera projection. Stitch them together in Photoshop and apply it to your model.

On the other hand, if you have a tool that can generate an approximate depth map for your footage (via optical flow or whatever method) and you have the camera data from the 3d tracking app, writing a simple program that can do the modelling is quite simple since it could easily "unproject" the 2d screen coordinates you click with your mouse into 3d vertices for your model. You could then export a simple 3d .obj model or something like that for use in a 3d app or a compositor that can handle 3d geometry (like Nuke or flame*).

Peter
6 years ago
but when can you get this sofware, what is the name of it?
6 years ago
wow, this is really amazing. if you can do this now, what can you do in the future? they just have to build al photocamera in a videocamara that makes 1 photo a sec. ore something. you will have endless oppertunities in the editing whit just one handeling.
Jeremy
6 years ago
Those damn algorithms. I never liked math. Still don't.
Peter Somers
6 years ago
Wow, the VideoTrace link is amazing ... Really takes the hard work out of modelling 3D objects from just XYZ plane photographs.
Anyone know if there is already a commercially available version of this somewhere?
6 years ago
I can see the Department of Defense and Homeland Security putting this to good use to cover up and doctor footage in favor and against our nation as well as those abroad
Rob Moffett
6 years ago
Hey Andrew, if you like that, you will LOVE this...

http://www.acvt.com.au/research/videotrace/
Marco
6 years ago
I'll surely keep an eye or two opened for that
LonelyWalker
6 years ago
you are awesome,
im getting into expressions,
and you are so right,
the power of controling my stuff ibecame much better now;
you should get payed by adobe too ^^ they cant show me their app like you did.

please continue with this,
this is much better then what ive learned at school:-)

best regards from germany
bijandisco
6 years ago
@ thomas_017 i meant that is easy to do for all of us vc students, but didn´t know andrew posted it some time ago.
My point is: that looks so amazing but is so easy to realize
6 years ago
@ bijandisco

If I remember correctly, Andrew posted this on the blog some way back.

Also, on Andrew's / Creatice Cow's Serious FX Training DVD it has a whole tutorial on making groups of people from one person (Sam dancing in this case). Can easily be done, if you have the motivation and patience to do so
6 years ago
just watched the video demo. blew my mind! this could save a lot of work. I also felt a sudden drop my IQ after watching.
bijandisco
6 years ago
uhh... look at this:

youtube.com/watch?v=WRS9cpOMYv0

that is or could be a great tutorial - isn´t it Andrew?

It´s so amazing but will be easy if you are teached by VC ;-)
bijandisco
6 years ago
First i thought, that is the direction that leeds to FX they use or better show in movies like Running Man (where Arnold´s face has been replaced digitally by clicking on ONE button only!)
But at the other hand i think a lot of the shown examples are possible with AE, too (remember: they are using still photographs for enhancement) or some special tool actually availible. I love such special tools a lot and they are making my live easier. Time will tell if it´s easier to enhance with that tool or still AE. But normaly you will get bad original footage in reality from private movie makers who will not (be able to) pay for enhancement of their footage. It´s the reason why they make their movies on very low budget cameras.
6 years ago
COOL check out each person in that project. all are far far far far futurestic BrainY guys. i too wanna move out!.

i found more than this invention. a croma without a green, blue or red screen. its just ausum!!!
6 years ago
Woah! These concepts would be excellent for commerical and indy movie projects.
I love it. AS the person before states the code may not be as efficient as others with development and refining, plus a licensing agreement who knows how much better the software can get. For now the feat is amazing and welcome in all facets of the post universe. Thanks Andrew for pointing the way.
Lawrence
6 years ago
*maltaannon... whoopsy!
Lawrence
6 years ago
Very interesting, I bet maaltanon will dig this!
6 years ago
Wow! I really liked the stabalizer and the removing of the sigh!
6 years ago
Well, check out section 7.1.1 of the paper: "The current processing speed of our system is quite slow with five
minutes being spent on each video frame (resolution: 853 x 480);". Now think a 200 frame shot in 4K. That's about 20 minutes per frame if your're very lucky (and processing time scales linearly), which means 4000 minutes, or 66.66 hours, or 2.77 days. We're going to need very lose deadlines, even if they manage to optimize the algorithm for better performance.

I downloaded their sample code and looked into it, and while I have seen much worse, the code is not necessarily very easy (nor very well suited) to be integrated into an AfterEffects or OpenFX plugin as it is right now. There are absolutely no comments in the sources whatsoever, but hundreds of different code files with zillions of utility classes and functions and third party libraries, so it's not very easy to figure out what's really going on and which part does what and is or can be used for what etc. So as much as I regret it, it's just too much work for me to make an AE plugin out of the reference implementation. And that's without counting the licensing nightmare that comes with all the third party libs. We'll have to wait for a company like Adobe or Autodesk to help us out.

But while we're on the subject of research papers: I found the following Siggraph 2008 paper on an adaptation of last year's seam carving algorithm for video processing quite interesting. One of the authors works for Adobe now, and the algorithm itself is much less complex than the one Andrew linked to (I implemented the still image version of the seam carving algorithm last year as a spare time project, see my website link if you want to try it for yourself), so I think there is a fair chance that this is going to be in AfterEffects/Photoshop CS4.
http://www.youtube.com/watch?v=AJtE8afwJEg

Peter
tAlpA
6 years ago
Awesome!!! Thank you for the video.
G Man
6 years ago
wow...i love the depth finder function...that is amazing...
i would hate to see the math that goes into that...
Kamui
6 years ago
*correction :P shot not shoot^^
Kamui
6 years ago
maybe a fake? :) the flower shoot looks to unbelievable imho.
6 years ago
Would be cool Plug In for my AE =]
Ori Lahav
6 years ago
Its a very creative use of 3D motion tracking (or at least something similar to it).
I wish them good luck with it, pretty interesting!

vbCrLf.
Paul
6 years ago
interesting.

Microsoft have 3 guys on this
Adobe have 1 guy on this.

I don''t know, but i reckon they'll be looking to make this os easy to do that they could make it comercially available. All the guys on the project have an experienced background in bringing complex ideas into a userfriendly format.

I predict this will be ot in the next 5years in a simple but effective form. Could be a collaberation between adobe and microsoft.
Daryl Booth
6 years ago
Holy CARP!

I will never trust a goverment vid clip agaian seeing what they can do with this stuff...
http://uk.youtube.com/watch?v=FgzNfOXW_CM
Michael
6 years ago
somebody here reads engadget, huh andrew :P
6 years ago
That looks so awesome. I am also currently studying that. It looks pretty advanced. I think you guys are interested in this too. http://www.vimeo.com/1513186?pg=embed&sec=1513186

I really love this one..its from the same makers that andrew kramer linked us.

Enjoy
jon
6 years ago
by Allen Ellis
"Well I can certainly imagine someone creating a still camera that would attach to the video camera automatically taking pictures. If they could get that to work, we could see this technology in consumer video cameras before long."

I already know a professional photojournalist who does just that. He has a small HD video camera attached to the tripod collar on his lense what ever he is taking photos of he is also taking Hi-Def video. A cool little tip I picked up from other pros.
Karol
6 years ago
I think it's kind of credit work from FX & animation university. It's cool. But, with VC tutorials, we can do simmilar things :)
Flubber
6 years ago
wow, that is awesome, thanks for sharing

don't have any other words, i am ''speechless''
Infaas
6 years ago
Cool!
they must develop a software.
6 years ago
Ahh, Wow! Interesting development. I can see the possibilities with applying effects to the photos first.
6 years ago
You can do this with Photoshop CS3 on the cheap. If anything, why are you shooting over exposed footage to begin with, go out and buy some flags light the thing.
6 years ago
I guess you could turn all footage into HD :D
Phil
6 years ago
I actually "Stumbled" upon this the other day, it is very awesome, like andy said, i dont think it'll be commercially available for a while, but once the technology starts picking up and others start to emulate, or they start making a marketable software and/or hardware product, yea...... that'll be fun! :p
Fernando
6 years ago
This is a very interesting technology, hope to see it soon on the market, it will save lot of time (and money) in a project.
Once a again thanks to Andrew for find this kind of info.
Waiting for the new tuts.
6 years ago
One thing of note, is the reference images require stereoscopic photography, not just a standard still camera, but a specialized setup...so, not quite consumer friendly yet...but think about the still image sensors on a standard vid cam, always quite high res...imagine a consumer cam that shoots vid, but takes a high res still every 20 frames or so, and stores it separately, this could theoretically be processed and ready in-cam, as you shoot, so the information is ready to process by the time you load it up in your program of choice...anyway, certainly fascinating stuff....on a related note, Microsoft is working on similar themed technology (pretty cool also). Have a look here:
http://www.youtube.com/watch?v=mNLx9pclMKU&feature=related
illd
6 years ago
I saw this side about a year ago.

At that time some people on the Web where arguing that the Fx shown are fake...

To me the contents on the side havent change since that...

I mean, Im just wondering why big companys haven´t already bougt or at least are interested in the technology
Termite
6 years ago
Fascinating stuff. Technology pushing the envelope even further. This will definitely be available in the near future. All of our current applications started out this way. I myself cant wait for something like this to available commercially on some level. This will only offer artists more tools to get the look they after. So long to throwing out poorly shot video. Now it may have life after all.....
Ryan
6 years ago
amazing, thank you for sharing. I hope somthing useful comes from this technology.
Jim Jimenson
6 years ago
Credibly the most incredibly incredible feat of incredible incredibility.
dead
6 years ago
amazing!!!! how they did the video with the parking label?
which program they use??
6 years ago
fantastic.they finally merged the two
John
6 years ago
That looks amazing. It produces very good results!

The object removal was flawless ...Thanks for sharing this.
5ive&2wo
6 years ago
lol, so fascinating that i forgot why i opened my browser.. ah i remember now... lol.
Jim Jimenson
6 years ago
Incredible.
Kaine
6 years ago
Thats incredibly useful, complicated but i see the obvious benefits. eagerly awaiting your 3D TRACKING tutorial andrew. Good Job, Your tutorials are increasingly helpful.
6 years ago
What system/software is used to achieve this?
Allen Ellis
6 years ago
"The downside of this technology is that I have to make photos of everything I shoot, which almost doubles the filming time…"

Well I can certainly imagine someone creating a still camera that would attach to the video camera automatically taking pictures. If they could get that to work, we could see this technology in consumer video cameras before long.
6 years ago
I don't think it is anything commercially available but rather a study. Although I'm sure if it is viable something may come of it.
6 years ago
Very cool. Interesting how so many solutions for video are converging on 3D technology.
Kyle
6 years ago
So this must have some kind of inherent 3D tracking in order to guess depth.
6 years ago
sorry, ignore that above me ;)
sachin
6 years ago
One of the person (Aseem Agarwala) behind this technology is now working in Adobe so I hope we might see this thing very soon in After Effects or Photoshop.
6 years ago
So how do you incorporate this into AE?
Mohammed
6 years ago
mmmm.. a new science?!
6 years ago
Yeah, I saw this a while back on Motionographer. I watched it 3 times I was so amazed. My favorite way to utilize this technology would be some cool stuff with the painterly effects. Like a live action version of Van Gogh's "Starry Starry Night."

Too bad you can't have anything moving in the shot like a car or person. But still really amazing stuff.
Hoque
6 years ago
this is amazing, i think this technology will be used a lot in the future
Boddo1994
6 years ago
This stuff is interesting. Awesome
6 years ago
looks fun :)
Kim A. Strandli
6 years ago
It is really impressive what people can do :) I hope this technology comes to the marked in a relatively low price soon in the future! Maybe Adobe should give them a big bag of money, and release it with the Creative Suite package? ;P
Some Guy
6 years ago
Wow! This is the future. I love all these technical stuff like this or automatic 3D scene creation from one photograph etc.
Especially I loved the motion stabilization. It was even better than a dolly shot. Almost artificial, like a 3D model. The downside of this technology is that I have to make photos of everything I shoot, which almost doubles the filming time...
6 years ago
thats so amazing. My favorite would have to be the library shot and the camera stabalization.
Mastermadman
6 years ago
amazing!
Chris
6 years ago
Wow... That's interesting. Not sure what to say about it either, but I look forward to utilizing those algorithm based depth maps and such to enhance my work.
Math
6 years ago
great utility
Jeremy
6 years ago
very interesting stuff.
R-Bleezy
6 years ago
Right On...Really cool
Tutorials
147. Advanced 3D Light Rays
Create 3D Light Rays inside After Effects
149. Solar Atmosphere
Create a powerful sun effect in After Effects
Plugins
Stock Footage