23 May, 2011
Optic Flow with OpenCV
A labmate is working on how the brain processes movement via the visual system, and was working on ways to calculate "optic flow", or calculating how fast things are moving, in what direction, in particular regions of video stream. I figured that was good enough reason to learn about different methods of calculating optic flow, so I whipped up this little video (accompanied by the first track of Moby's new album! Turns out, that's the only track I like off the whole thing...).
So, what are you seeing in the video? White is movement, black is stillness. Specifically, the luminosity is the square of left-right and up-down components of movement (so, technically, I do have information about direction of movement, but I don't use it here).
I'm using the Horn & Schnunck algorithm for computing optic flow, and the various weird changes of the feel of the images that occurs throughout the movie is me tinkering with some of the parameters of the algorithm. Specifically, I modulate the degree of blurring of the input image, before optic flow calculation, and then I also change the size of the Lagrangian multiplier of the HS algorithm. The blur, well, blurs the image, and the Lagrangian term affects how "noisy" the image looks, and how sensitive the algorithm is to small movements near the noise floor. There's always going to be "fake" movement, as a result of compression artifacts in the video stream, and the blurrier the image and the smaller the Lagrangian, the less sensitive the output is to that. However, the bigger the blur and larger the Lagrangian, the less sensitive the output is to actual movement.
Neat! This is pretty fun stuff, and once you start to wrap your head around how OpenCV wants you to think, not too terribly difficult (I do have the Cinder library to thank for making a lot of the bookkeeping much simpler, though).
On another note, I've just picked up and hooked up a Microsoft Kinect, but haven't yet begun doing anything super rad with it, besides getting raw depth-field shown on my screens. I'd like to do limb-angle tracking with people bodies, and then little mice bodies, but that'll have to proceed as free time allows.
Oh! If you're interested in the code, do get in touch with me (I'm not terribly hard to track down), and I'll hook you up, straight up. It's comfortably under 250 lines, but it does depend on a lot of other libraries.
