OpenCV

XCore Project reviews, ideas, videos and proposals.
Post Reply
User avatar
johanar
Active Member
Posts: 60
Joined: Tue Feb 16, 2010 8:21 am
Location: Sweden
Contact:

OpenCV

Post by johanar »

Has anyone got any experience with OpenCV? Would it be possible to port to XMOS, or is it too dependent on having a full blown OS underneath?

If it's not doable, a project to develop general computer vision libraries for XMOS might still be a good idea. Unfortunately I can't do it myself, as I'm just getting started with CV and I don't have a lot of time even for that. It would be a way to show how much more powerful these processors are compared to others though.


User avatar
shawn
XCore Addict
Posts: 238
Joined: Thu Dec 17, 2009 5:15 am

Post by shawn »

Xmos being c/c++ complient lends itself to multitude of ports and algorithm's.
I haven't looked at CV yet, but, Carmera hacking is prime for XCore, and would
be rewarding to those that can exploit IP, like CV apps.. I also think there's a
killer apps for such IP in the development of wearable IP, like WUW/6th sense.
User avatar
johanar
Active Member
Posts: 60
Joined: Tue Feb 16, 2010 8:21 am
Location: Sweden
Contact:

Post by johanar »

Indeed, the uses of computer vision are unlimited, and I'm sure XMOS could help hobbyist builders get into this exciting area.

Speaking of which, does anyone know if there's been any further progress in Leon Heller's and Ross' projects involving cameras and XMOS processors? Thinking about this thread over at xmoslinkers (and xcore is the place to be, right? or should I get an account at xmoslinkers instead?)
User avatar
leon_heller
XCore Expert
Posts: 546
Joined: Thu Dec 10, 2009 10:41 pm
Location: St. Leonards-on-Sea, E. Sussex, UK.
Contact:

Post by leon_heller »

I'd forgotten all about that. I'll see if I can find the PCB and finish it off.

I've just got a couple of TCM8230MD cameras. They are tiny.

Leon
User avatar
johanar
Active Member
Posts: 60
Joined: Tue Feb 16, 2010 8:21 am
Location: Sweden
Contact:

Post by johanar »

Sounds awesome! You should start a project for it so we can follow the development, and I'll post all my questions and suggestions there :)
User avatar
mikef
Member
Posts: 15
Joined: Sun Dec 13, 2009 3:17 am
Contact:

Post by mikef »

I don't think you'd be wanting to bring in even the raw algorithms from OpenCV. The issue on the XMOS is memory: even a 160x120 image, in 8-bit pixels, would consume something on the order of 19200 bytes -- that's a third of your core memory, and most vision algorithms to need the raw image, plus a second buffer to compute things in.

Interfacing external RAM as a scratch pad is a possibility, but it's more likely that if you had a thread or two acting as a framegrabber, and distributing parts of the image to different threads/cores, you could do quite a bit more -- but it's going to require paying close attention to the sort of algorithms you use.

If I ever find some more free time, it's something I'd like to play around with -- but my XMOS hasn't been powered on in several weeks.

-Fergs
User avatar
shawn
XCore Addict
Posts: 238
Joined: Thu Dec 17, 2009 5:15 am

Post by shawn »

mikef wrote: Interfacing external RAM as a scratch pad is a possibility, but it's more likely that if you had a thread or two acting as a framegrabber, and distributing parts of the image to different threads/cores, you could do quite a bit more -- but it's going to require paying close attention to the sort of algorithms you use.
I like the idea of QDRam, its real tight latency, complements CSP. looks like a job for G4.
I understand that most all source out there is not CSP. we need a smart cross compiler.
Especially for UNIX and all that Von Neumann code is unlikely to be at all CSP optimized.
Anyone porting said OS to CSP platform, Has to look at it that way. I want a CSP TEKO.
User avatar
errordeveloper
Active Member
Posts: 50
Joined: Wed Dec 16, 2009 6:04 pm
Location: London
Contact:

Post by errordeveloper »

ah, i'd say we really need to create a thread on memory solution on this forum.

one nice way i thought could be to implement a chip that would appear as a sort of channel accumulator ..

i am trying to think of a delay-line type of thing,
so on the high level you could say call a function to allocate it and then access it as a special channel/buffer to store very large chunks, like up to a few megs at least.

sorry if my explanation is not in those proper jargon terms, but i hope you got what i meant.
an accumulator facility which has a channel-like interface.

but yeah, i reckon we need an proper thread on this!
haven't heared if xmos is looking into expanding the size at all ..but may be a community project could work out.
User avatar
johanar
Active Member
Posts: 60
Joined: Tue Feb 16, 2010 8:21 am
Location: Sweden
Contact:

Post by johanar »

mikef wrote:I don't think you'd be wanting to bring in even the raw algorithms from OpenCV. The issue on the XMOS is memory: even a 160x120 image, in 8-bit pixels, would consume something on the order of 19200 bytes -- that's a third of your core memory, and most vision algorithms to need the raw image, plus a second buffer to compute things in.
Good point! Shows that I haven't really started looking into if this was actually possible :oops: But as you said, external RAM would be an option though I was really hoping for a one chip solution.

Just an idea, but maybe some algorithms could be rewritten to process the data linearly, so it doesn't have to save the whole image. Some functions might produce a compact result so you process one frame which is discarded after the calculation is done. Perhaps you grab a low res frame, and if you want to work with a specific area (for example if you suspect you see a face) the next frame from camera is cropped so you get a more detailed image of just that area. If you work with RGB separately you could do a 3-pass processing, but you'd still only get something like 320*240..

But as you say, it would be a lot easier with more RAM.
User avatar
andrew
Experienced Member
Posts: 114
Joined: Fri Dec 11, 2009 10:22 am

Post by andrew »

I'm up for helping with an OpenCV like library for XMOS chips. Especially writing algorithms that work in very tight memory footprints
Post Reply