FPGA supercomputer

Off topic discussions that do not fit into any of the above can go here. Please keep it clean and respectful.
User avatar
Berni
Respected Member
Posts: 363
Joined: Thu Dec 10, 2009 10:17 pm
Contact:

Post by Berni »

Oh and thats a great idea to also use some kind of code sharing system so that when a user gets something working they can submit it by just pressing a button. The code might help someone else get started lots faster


User avatar
jonathan
Respected Member
Posts: 377
Joined: Thu Dec 10, 2009 6:07 pm
Contact:

Post by jonathan »

Berni - good posts. I reckon if we can get enough interest we've got the skills on this board to make it happen. I've suggested XMOS do this with the XDK kits about 2 years ago - this would enable you to try out programming the XDK (complete with components, LCD, etc) over the web without having to shell out $999 for a kit!

Would be very worthwhile. There are lots of programmers out there who would want to try this out before they pay for hardware and then discover they need a soldering iron and a pile of components to do anything interesting. This is a route in!

The code sharing idea is great too - you hit one of the ways to make this system contribute real value back to the community too.
Image
User avatar
Berni
Respected Member
Posts: 363
Joined: Thu Dec 10, 2009 10:17 pm
Contact:

Post by Berni »

Yeah i think xmos should really try it as once the thing works there is no maintenance costs or work with it.(Apart from a mod to keep the code sharing system clean) As all you need is one fast PC running 24/7 and a bunch of xmos boards plugged in to it with a few Mbit of internet bandwidth on top. The users cannot mess up the PC as vmware can simply restore to a saved state once the user logs off and so fixing any mess.
User avatar
trousers
Active Member
Posts: 44
Joined: Fri Dec 11, 2009 10:20 am
Contact:

Post by trousers »

Berni wrote:Yeah i think xmos should really try it as once the thing works there is no maintenance costs or work with it.
I think jonathan should go for this as one of his xfund start-ups. If it can be done at a very low cost, and there are as many programmers as he says, then it could generate income from adverts and maybe from re-selling boards with peripherals pre-installed. Then those budding hardware engineers would never need to touch a soldering iron or a pile of components and he could turn any interesting projects into further start-ups. A virtuous circle with almost no up-front cost!
Best friends with the code fairy.
User avatar
jonathan
Respected Member
Posts: 377
Joined: Thu Dec 10, 2009 6:07 pm
Contact:

Post by jonathan »

trousers wrote:
Berni wrote:Yeah i think xmos should really try it as once the thing works there is no maintenance costs or work with it.
I think jonathan should go for this as one of his xfund start-ups. If it can be done at a very low cost, and there are as many programmers as he says, then it could generate income from adverts and maybe from re-selling boards with peripherals pre-installed. Then those budding hardware engineers would never need to touch a soldering iron or a pile of components and he could turn any interesting projects into further start-ups. A virtuous circle with almost no up-front cost!
Thanks for the suggestion.

The concept works well as an XMOS-supported project - even if the community builds it (which is why I have asked if XMOS might consider offering an XMP-64 as a prize).

To do more than that requires a lot more thought. Would you like to pitch the idea? Get in touch and we can sort out a time and a place.
Image
User avatar
XMatt
XCore Addict
Posts: 147
Joined: Tue Feb 23, 2010 6:55 pm

Post by XMatt »

trousers wrote:
Berni wrote:Yeah i think xmos should really try it as once the thing works there is no maintenance costs or work with it.
I think jonathan should go for this as one of his xfund start-ups. If it can be done at a very low cost, and there are as many programmers as he says, then it could generate income from adverts and maybe from re-selling boards with peripherals pre-installed. Then those budding hardware engineers would never need to touch a soldering iron or a pile of components and he could turn any interesting projects into further start-ups. A virtuous circle with almost no up-front cost!
This whole idea seems to be about creating a problem purely for the sake of solving it? Giving away a development board to anyone interested would be far more cost effective in this case and most likely achieve better results.
User avatar
jonathan
Respected Member
Posts: 377
Joined: Thu Dec 10, 2009 6:07 pm
Contact:

Post by jonathan »

XMatt wrote:
trousers wrote:
Berni wrote:Yeah i think xmos should really try it as once the thing works there is no maintenance costs or work with it.
I think jonathan should go for this as one of his xfund start-ups. If it can be done at a very low cost, and there are as many programmers as he says, then it could generate income from adverts and maybe from re-selling boards with peripherals pre-installed. Then those budding hardware engineers would never need to touch a soldering iron or a pile of components and he could turn any interesting projects into further start-ups. A virtuous circle with almost no up-front cost!
This whole idea seems to be about creating a problem purely for the sake of solving it? Giving away a development board to anyone interested would be far more cost effective in this case and most likely achieve better results.
Sense! Thanks.
Image
User avatar
lilltroll
XCore Expert
Posts: 956
Joined: Fri Dec 11, 2009 3:53 am
Location: Sweden, Eskilstuna

Post by lilltroll »

I'm writing DSP code (adaptive Volterra kernels (like a connected group of adaptive linear FIR filters that togheter makes a advanced realtime nonlinear adaptive filter) , MIMO adaptive realtime FIR, MultiChannel feedback adaptive FIR (of course realtime), Multirate ... and so on.

Things that I have only been able to simulate in MATLAB earlier.

Anyway - Low latency FIR-filters (That cannot use any FFT solution) works very nice with the 64-bit MAC instruction. Parrallell computation is no problem with FIR on a determenistic CPU.

When I need more power than the XC-1 - I hope to buy an X64 if other here reports that they succed to program the beast.

Competitive (more simple) solutions made by others is for an example based on the ADI Blackfin. Blackfin (that I played with earlier under uCLinux) can adress a larger outofchip RAM - which is nice for large FFT - but is of no use in my Point by Point processing.

I will not use my card 24/7 and I have a fast core i7 with 6 Gb RAM and I'm connected via optic fiber to the Internet. Also, I like the idea to share resources.
Probably not the most confused programmer anymore on the XCORE forum.
User avatar
nieuwhzn
Member++
Posts: 26
Joined: Sat Dec 12, 2009 6:45 am

Post by nieuwhzn »

I just received my XMP-64 yesterday. Because of lack of time I only was able to run the LED examples on it. What I noticed immediately is that the executables are huge. There is no intelligence in the compiler to figure out that you're trying to run the same code on all the different cores. Each code gets its own code in the executable which get loaded onto the individual cores. This is a big showstopper when trying to build systems with a couple of thousand cores.

What I will try to do is to set up the individual cores as mostly identical computing engines. Data will be dispatched to the system and will be picked up by a core if it is in idle state. Results will be send back into the grid to find their way back to a collector. Dispatcher and Collector will probably sit on the cores that have the ethernet connectors, which will have to be the gateways to move data on and off the board. Initially the data paths will be set up as a token ring, I had this already working for the 4 cores of an XC-1. As soon as that works I'll start thinking about using the hypercube topology in a smarter way, i.e. minimizing the hops that a package has to make to reach its destination. Fun ahead! I'll keep the forum updated.

Thousands of FPGA's as supercomputer? Aaaaargh.......Nightmare. Programming 1 Spartan is bad enough, thank you very much. My reason for using XMOS cores is that they come with XC, everything that you need for doing parallel computing wrapped into a nice C package. Keep in mind that a computing project usually has a completion date, so you always have to weigh software development time against just throwing more hardware at it. This is exactly the reason that vector computing failed so miserably, it needed utter knowledge of the hardware, most didn't have that and wrote lousy and slow programs.
Currently the most successful 'supercomputer' are computing farms, mostly a couple of hundred or thousend relatively cheap Linux boxes. As long as you are doing event parallelism you just develop on one interactive node and then dump your program and data on the farm. As far as I can see still nothing beats that model as far as time and cost is concerned. Of course it is utterly boring!
Currently we are trying to augment this model with attached GPU's, the results are promising, but the process (using CUDA) is painful. What I'm trying to do is to use the XMP-64 in a similar fashion, i.e. a co-processing-node.
Heater
Respected Member
Posts: 296
Joined: Thu Dec 10, 2009 10:33 pm

Post by Heater »

I'm no parallel processing expert but it seems to me that an array of FPGA's is not a good solution for building a "supercomuter".

My observation of FPGA designs is that yes they are great for implementing "odd" hardware solutions that would otherwise need piles of logic chips or an ASCIC. Here the complexity of designing in VHDL/Verilog or whatever is acceptable.

For many other designs the technique seems to be to alleviate a lot of the design complexity by implementing a processor core, from off the shelf IP, on the FPGA enabling a lot of the solution to be shunted into software. In C for example.

To go with that the I/O, Ethernet, USB, various buses etc, is also implemented from off the shelf IP. This minimizes the amount of actual custom logic design required for ones application.

Now a "supercomputer" with any generality in mind would probably be implemented in FPGA via the processor core approach. In which case on might as well use real processors in the first place. The XMOS devices look like what you would end up with in FPGA anyway.

I have worked with a team that used an array of FPGA's for checking out their ASIC design. It worked out very well but is hardly a "supercomputer" in the normal sense of the word.

As for the huge executables. Back in the day there was a WORM program for the Transputer that would boot up a network of nodes with the same code. The size and topology of the network was discovered as it booted. Seems like the XMOS needs such a WORM for arrays like this.
Post Reply