Runtime Negotiation of Xlinks

[Ali]

thanks Paul, a great piece of work and very helpful.

In case anyone cant find the project, its here: https://www.xcore.com/projects/runtime- ... ion-xlinks

and the details from this post are on the wiki. Please update and correct anything on the wiki: https://www.xcore.com/wiki/index.php/Ru ... _of_Xlinks

Ali

[spark]

Hi,Paolomio! I am trying to make XS1-G be a router which can store and forward the packet received. So i want to use LINK C and LINK D to connect two XS1-G devices. First plan i find recently is making the two XS1-G be 8 cores by modifying the XDK.xn. But it ends up with Failure. I can't boot two devices by only one PC. The attachments are the linking pictures and XN Files which are used to configure the two devices. I can't find the reasons why the connection is failed!!

[Paolomio]

Hi spark!

Sorry for the late reply...I don't always have a chance to read the forum.

Note that the code for (and the whole concept of) negotiating xlinks at runtime does not work with the G processors. They do not support the HELLO protocol that the L series does, and that feature is necessary in order to do runtime negotiation.

Note also that with the G processors there are constraints for topology. You can connect one G to another G, but you can't create a daisy chain of G's as your drawing implies. G processors must be connected in Hypercube topologies, so you can have 1, 2, 4, 16, 64..., but not any number of them other than that progression. This limitation is not present for the L series parts.

I also see from your drawing (very helpful, by the way!) that you are somehow trying to connect these boards using the xsys connector. Are you trying to use the xlink part of the xsys connector to make this connection between boards? I'm not sure this will work while at the same time booting the processors via XTAG2. There is a thread somewhere on this forum about daisy-chaining processors to boot multiple processor from XTAG2, but I'm not sure it applies to G processors. You might do some searching to see if that thread has anything interesting to say for your case.

I don't have much hands-on experience with the G processors--I only use the L's (in a chain-of-arrays configuration).

Sorry I can't be more helpful,

Paul

[spark]

Hi Paolomio!
Thank you very much for your detailed and helpful explanation. I plan to search for example about the daisy-chaining processors to boot multiple processor from XTAG2, just as you mentioned.
I will very appreciate it if we can communicate with each other about xmos programming again someday.
Spark

[pstnotpd]

Just found this post and the wiki entry! I was searching for this answer for some time now.

Gonna try it on a bunch of startkits :o)

[GerhardNorkus]

Hi Paolomio,

Is there any way you could post some code showing it modified for a port C to port D connection instead of A and B? I'm having trouble sifting through the documentation and understanding the direction bits (there are so many of them). I did download your wiki entry and the associated code.

Also, can you post a simple project that shows how you are calling your functions from within XC? Preferably, it would be nice to see a compilable example for two XK-1A's just shooting data from one XK-1A to the other. It should not, however, be a project where the XK-1A's are connected during flash programming. They should be flashed separately (on flashed to some slave code, the other flashed to some master code).

Of course, anyone else can feel free to reply, if possible!

Thanks ahead of time! I hope you get the post!
GJN

[rtimte]

Hi Paolomio,

Tried to download and make two StartKit's talk. The code is missing two functions:

int stchanendToInt(streaming chanend ch);
void setdNode(streaming chanend c, int targetNode);

Anybody else get this to work?

Richard

[pstnotpd]

@rtimte

I'm trying something similar and also looked at Paulomio's static forwarding code. Unfortunately I haven't found any good documentation on configuring the switch network on the fly.

@XMOS is there any more detailed documentation on configuring xlinks programmatically?


But some time later I found it mentioned that a "service" declaration in the XN allows any device which implements xlink to be connected (so including another startkit). This would, if I understand it correctly, leave the switch magic to the tools and would only involve the negotiation itself to set up a channel.

So that's the route I want to go, as it looks like a much more elegant way to achieve the same goal. But is again, the "service" declaration is very sparsely documented.

Any help would be appreciated.

[rtimte]

pstnotpdl I agree would be nice to have XMOS give us a little help,

I played with my design with 4 XK-1A's which was easy to network together as they have two IDS XSYS headers on them. Just plug them up and modify the XN file and you can code and debug all four from the xTIMEcomposer. I saw the new development kit startKIT and went out and got 16 of them to try and finish out my little project (should have looked harder at the docs before buying them). Saw that three of the four XLINKs B, C and D were all pined out to a jumper and I needed analog to digital in my project so off to Digikey to get them. Then looked at them closely and saw no XSYS connector.

Thought no big deal as I want my network to be dynamic (not plug and play), but able to connect any number of devices and have them self negotiate with each other at boot time. Looked at the xCONNECT Architecture documentation as it described that the silicon's switches have this capability but just a description of what is needed to be done but nothing on how to do it.

After much searching found this thread and it is exactly what I am trying to do but the code example is incomplete (missing two routines). Also, would have been more helpful to include a stub program that calls the dynamic channel routines and sends a few bytes of data between them.

Since I can't get this method to work (YET) and trying to debug a network of startKIT's without a debugger is not a option, I'm at a standstill.

Here is how my network is designed and mostly coded:

At boot the master chip tries to connect to all four links A,B,C and D and if successful it sends a message/command on that link to do a report connections. The report connections message is processed by the child chip by sending back to the parent chip ( that sent the message) what links this chip can talk to. All return messages flow back up to the master chip which creates a routing table. This process flows down the network tree until all chips on the network have reported back to the master. once this is complete the master holds a complete routing table of all devices on the network and how to get to them. Node numbers are assigned sequential from 1 (master) until the last node (chip) has reported back.

When the master has a complete routing table, it generates a routing table for each node in the network and sends this table to each node so it has a complete layout of the network. Now at this point every chip in the network has a routing table with the node address of every other chip in the network. The drawback with this design is that a message will take multiple hops (chip to chip) to get to its destination. But the good think is collisions are nonexistent as each hop in the chain ensures only one message in each direction is in play at any one time. This does not mean that only one message is in route but that between each chip only one message is on its what through the chain. Also the good think is each and every tile in the network can talk to each other (will talk about this later in the post).

An example of the routing table is below (first three are in the table, last 5 are for this discussion):

Code: Select all

Node  Node Type  Node Address                         Link A  Link B  Link C  Link D  Link Chain
 01   Master     -                                    02-A    03-A    04-A    05-A    -
 02              10001111                             01-A    06-A    11-A            A
 03              01001111                             01-B    17-A    18-A            B
 04              00101111                             01-C    19-A    20-A    21-A    C
 05              00011111                             01-D    23-A    28-A    29-A    D
 06              10000100-11111111                    02-B    07-A                    AB
 07              10000100-01001111                    06-B    08-A                    ABB
 08              10000100-01000100-11111111           07-B    09-A                    ABBB
 09              10000100-01000100-01001111           08-B    10-A                    ABBBB
 10              10000100-01000100-01000100-11111111  09-B                            ABBBBB
 11              10000010-11111111                    02-C    12-A                    AC
 12              10000010-01001111                    11-B    13-A                    ACB
 13              10000010-01000100-11111111           12-B    14-A                    ACBB
 14              10000010-01000100-01001111           13-B    15-A                    ACBBB
 15              10000010-01000100-01000100-11111111  14-B                            ACBBBB
 16              01000100-11111111                    03-B                            BB
 17              01000010-11111111                    03-C                            BC
 18              00100100-11111111                    04-B    19-A                    CB
 19              00100100-01001111                    18-B                            CBB
 20              00100010-11111111                    04-C                            CC
 21              00100001-11111111                    04-D    22-A                    CD
 22              00100001-01001111                    21-B                            CDB
 23              00010100-11111111                    05-B    24-A                    DB
 24              00010100-01001111                    23-B    25-A                    DBB
 25              00010100-01000100-11111111           24-B    26-A                    DBBB
 26              00010100-01000100-01001111           25-B    27-A                    DBBBB
 27              00010100-01000100-01000100-11111111  26-B                            DBBBBB
 28              00010010-11111111                    05-C                            DC
 29              00010001-11111111                    05-D                            DD

Note: the address is in 4 bit chunks and each bit is a link A= 1000, B=0100, C=0010 and D=0001, also 1111=end of chain. if the address ends in a half byte then fill the rest with end of chain 1111

Now when a message is sent to a node the message is processed if the first nibble is 1111 or forward onto the next node using the nibble as to what link to route to.

Examples:

1111 = this is your message
1000 = shift address by four bits and send to linkA
0100 = shift address by four bits and send to linkB
0010 = shift address by four bits and send to linkC
0001 = shift address by four bits and send to linkD

Tile zero on each chip is the network admin and processes or forwards the message as stated above. If the message is for this chip the first byte of the message is the tile on the chip that this message is for. The complete message is the address followed by a one byte tile number and then a two byte message length and then the message.

So let's say the master tile 5 wants to send a message "HELLO" to tile 3 of node 9 the message the tile would send to tile 0 would look like this: 00001001-00000011-00000000-00000101-H-E-L-L-O. Then tile 0 would convert the message to: 10000100-01000100-01001111-00000011-00000000-000000101-H-E-L-L-O and process the message (this conversion is to lookup node 9 and change this to the node address). Since the first Nibble is 1000 it would shift address by four bits and send to linkA. Now node 2 (link A connection) would process the message 01000100-01000100-11111111-00000011-00000000-000000101-H-E-L-L-O, since the first nibble is 0100 it would shift address by four bits and send to linkB. Now node 6 (Link B connection) would process the message 01000100-01001111-00000011-00000000-000000101-H-E-L-L-O, since the first nibble is 0100 it would shift address by four bits and send to linkB. Now node 7 (Link B connection) would process the message 01000100-11111111-00000011-00000000-000000101-H-E-L-L-O, since the first nibble is 0100 it would shift address by four bits and send to linkB.
Now node 8 (Link B connection) would process the message 01001111-00000011-00000000-000000101-H-E-L-L-O, since the first nibble is 0100 it would shift address by four bits and send to linkB. Now node 9 (Link B connection) would process the message 11111111-00000011-00000000-000000101-H-E-L-L-O, since the first nibble is 1111 it would route the message to tile 3 00000000-000000101-H-E-L-L-O (since the tile address is 00000011 or 3).

Now this is how I have it now... if anybody has suggestions for improvement My ear's are wide open. When I have the dynamic link logic working I will be glad to post the code to do the above!!!

Richard

[pstnotpd]

@rtimte

That sounds like roughly the same idea that I had. Due to time constraints I didn't pursue when I noticed the rather vague description of using static forwarding for port C and D. I believe it should be easier there are no shared pins for this port.

If I understand correctly you want to query all adjacent nodes at boot time in static forwarding (non-routed) mode and then set up the routing table and switch back to routing.

I was thinking of a "plug and play" configuration myself instead of a master-slave setup at boot time which would involve agents that keep listening for incoming hello's and then initiate reconfiguring.

If you use the TP1 header with a standard XTAG you do have very basic debug available as all startkits are visible on the JTAG chain (and show 2 tiles) . But I have not found a way to use the xe file in Sethu's example to set breakpoints etc. as these assume a 1 tile configuration per startkit.

How do you connect them? Do you have LVDS? I get a decent connection using <10cm jumper cables so hence my breadboard setup.