A couple of months ago I was at Firebrand Saints and started wondering how the effects that they run on their videos work. At Firebrand, their main bar has a few flat screen TVs showing cable channels you’d expect but every now and then one of the TVs will start displaying the video through a filter. So as you’re sitting at the bar you’ll notice Sportcenter go from regular video to what looks like Sportcenter put through a pixelated filter. Unfortunately, the fall got a bit busy and I forgot to jot this down but it recently came back to me while watching the Family Guy – Take On Me skit.
So how do you programmatically apply effects to video? After some searching around, it seems like the preferred way to do this is to convert the video to a series of images, apply the effects, and then encode the images back to a video. Since we’re on Linux the weapons of choice for this are libav (ffmpeg fork) and ImageMagick. Additionally, I used youtube-dl to grab some source video from YouTube.
Playing around with manipulating videos and images is pretty CPU intensive so I decided to do this on a c3.xlarge. Once you have a machine up, just run the following to get everything setup:
Now, snag yourself a video from YouTube:
Next, you’ll want to extract the audio and then the individual frames from the video:
And now for the fun part – time to apply some effects to the image! As an fun first pass I ran the “paint” transform across the images. PS. Remember how we launched on that c3.xlarge? Well now we can run the transforms in parallel with xargs:
Finally, you’ll need to encode the images back together into a video format of your choice:
Here’s a clip of the video I ended up with:
So what else can you do with this? Well that’s the awesome part! An image manipulations you can do programmatically (ImageMagick, NodeJS, etc.) you can apply to your video.
Perhaps some Mystery Science Taylor Swift?
Unfortunately this adventure has left me with more questions than answers:
How do you apply filters to a video in real time?
Would it be possible to integrate this into a rooted (or stock) Chromecast?
Could you build something to support user interactivity? Maybe with Canvas/nodeJS?
It’s Friday and we have links! Grab some coffee (or a beer) because some of these are going to be a time sink.
Projects Come Alive With VideoBlocks
Enliven your creative projects with millions of unlimited downloadable content at the touch of a mouse on VideoBlocks.com. Check it out >
Download videos from Youtube (and more sites)
A small command-line program, youtube-dl lets users download videos from Youtube and other sites. The only requirement though is Python interpreter. Grab it now >
A Behind The Scenes Look Into AWS
At the re:Invent conference in Las Vegas this year, James Hamilton, vice president and distinguished engineer for Amazon Web Services (AWS), gave conference attendees a broad but insightful glimpse at the size and scope of AWS cloud. For example, Hamilton goes into detail on the ins and outs of the network AWS built from scratch. For more on the story, keep reading >
The Truth Behind W3C, the Web, and HTML5
In October of this year, the World Wide Web Consortium (W3C) officially recommended HTML5. For citizens of the Internet who aren’t familiar with the matter and want to know more about it, this article goes back in time, providing an in-depth history on how the Web, the W3C, and HTML5 came to exist. Keep reading >
Strategy Change For Firefox’s Search Partnerships
Last week, Firefox announced that it will be changing its strategy for Firefox search partnerships. In the announcement, it will no longer have a single global default search provider which was formerly held by Google. Instead, Firefox has chosen to promote choice and innovation on the Internet by partnering with local search providers, becoming the default search experience rather than Google, in every country. Keep reading >