XMOS + Kodak image sensor vision system

[gibson]

Version: 1
Status: Under development
License: BSD
Download: http://github.xcore.com/repo_index/sc_xkcam_readme.html

A research collaboration between the University of Bristol and XMOS has developed a realtime adaptive tracking system. The system consists of a kodak image sensor connected to XMOS quad cores with an LCD and ethernet connection. XMOS threads program the sensor to sample an image as a sequence of Regions of Interest (ROIs), which overcomes memory constraints. Image processing on pairs of ROIs provides motion detection which is then used to re-program the sensor to 'look' more often at regions that may contain motion. A recursive Baysian filter is used to predict where motion is most likely to occur in the future. ROIs are read at rates of over 750 ROIs per second which gives an overall resolution of 320x240 pixels at 25fps. When motion is detected only a fraction of the sensor surface is sampled, system resources are focused on interesting things that are happening rather than grabbing pixels that are not changing. In the YouTube video blue ROIs are not being grabbed, instead many more ROIs are being grabbed around the moving object. The diagrams show the system layout and how the image processing component is structured (blocks=threads).
Video:

Images:

[phalt]

Thanks for posting a video of it working - are you going to be taking this project further?

[gibson]

Yes, a dedicated vision board is being designed, I will provide more details when it has been manufactured.