13 May, 2011
Eye Tracking! A first attempt.
(download)
So, as a previous tweet has promised, I’ve started working on some gaze tracking software. Look above for my first attempt!
The road up to even getting the necessary software working has been arduous, but I have something simple (and very rough) to show.
Getting OpenCV installed on the Mac is an ordeal, and I have a thing or two to say about that in the future. Expect a blog post.
I’m writing it up using the Python bindings to OpenCV, which greatly simplifies the prototyping process. Although there are some existing libraries that would make some of the coding process easier (one called SimpleCV, being worked on by my good friend Kat Scott), it is thus far incomplete, and (at least in my experience) presumes a coding style that expects that you only use SimpleCV, and will never have to dip into the grittier OpenCV primitives should SimpleCV fail you. I’m sure that when SimpleCV is more thoroughly documented and covers more of the API, that it will be wonderful. For now, however, the pain of inevitably cross-converting data structures between SimpleCV and OpenCV creates more clutter than it saves.
SimpleCV devs — if you have suggestions for best practices, let me know! I'm open to any suggestions that simplify my workflow.
The next steps —
* More fluid eye tracking. Basically, seed the next frame’s eye tracking guess with the last frame’s solution. This should produce much smoother estimates.
* Actually tracking pupil position within the eye. This will be the variable I’ll use to directly map to screen position (the ultimate goal is to use your eyes as a cursor, especially useful for handicapped folks).
* Make a model of face-to-screen distance. This will be necessary to be able to translate eye angle to screen position reliably when the head is moving slightly.
* Better training of the eye detector. My eyebrows are apparently quite eye-like in the algorithm’s current iteration. I mean, they’re bushy, but jeez. Way to make a guy feel self-conscious.
