Max of 1 channel off core in a thread?

[bearcat]

I guess I missed this in the documentation. Took me a while to figure this one out, unfortunately.
On core can have multiple channels, but only 1 off core (in a dual or more core system). Maybe with some special programming of control registers, this may be possible, but is not what I am interested in doing.

Hey XMOS (or users here) how about a "cheat sheet" that lists issues like this. Would save non expert users alot of time as I did not notice any special mention of this in the documents I reviewed (or missed).

I had to use control tokens after all (which actually is working fine, almost). But you can't send control tokens in both directions, because it will block. We really need a "token available" instruction! As mentioned in the other thread, there is a control register to read, but I will re-work the software instead. This is needed if you don't want to send every value every time as this may cause many times more bandwidth needed.

Am I wrong?

Edited: misspell

[segher]

I guess I missed this in the documentation. Took me a while to figure this one out, unfortunately.
On core can have multiple channels, but only 1 off core (in a dual or more core system).

I very much doubt that is true. More likely, you are running into some other limitation; for
example, you're not closing the channels you use (by sending an END control token), which
means you'll quickly run out of available off-core channels.

Hey XMOS (or users here) how about a "cheat sheet" that lists issues like this.

There already is the knowledge base, and the wiki.

I had to use control tokens after all (which actually is working fine, almost). But you can't send control tokens in both directions, because it will block.

That's not true either as far as I can see.

We really need a "token available" instruction!

That would be nice in some (very) limited cases, you'll usually be better off just blocking a
thread though.

[bearcat]

I had hoped it wasn't true, but I have verified it with a hardware prototype. In my testing, I sent no control tokens at all, but would block (that's what happens when >1 channel). It always blocked. I started with code that worked single core (and multiple channels per thread). I scratched my head a couple days on this, until I reduced it to a single channel off core, and then did it work as expected. I tried lots of other possibilities, but only a single channel worked. I sent >250 million messages at one point on one channel, without error, to verify the hardware. The compiler gives no error, it just doesn't work when you try to send more than one channel at a time, the second channel never receives it. I also looked at the new MC Audio firmware, and it uses only a single channel off core. Of course, I am still learning and could be wrong.

I am only using the standard outuint and intuint instructions, and not driving the channel by the byte. Therefore I assume I need not send an end after each outuint, etc, which I am not sure I can even do.

The problem with using the chkct instruction in both directions, is it needs a token to process. I have to think on this some more, but I can say it didn't work on prototype.

[bearcat]

As a follow up. This is not a problem. My project is working just fine using a single channel between cores. This forced me to clean up the code and structure it better.

I am still not exactly sure of the implications or limitations, as the documentation doesn't cover anything about this that I could easily find. I have assumed this is by thread, but maybe not. I will be finding this answer soon enough. Now that I am aware of the limitation, it will be easier to discover and code around in the future.

[richard]

On core can have multiple channels, but only 1 off core (in a dual or more core system).

If you are using streaming channels or outuint() and inuint() without sending END tokens then each channel will consume a link. The number of channels between cores will be limited by the number of links between cores. For example on a G4 it wouldn't be possible to have more 4 streaming channels to other cores on any one core. Which device(s) are you using? If you have a multinode system with only a single XMOS link connecting two nodes then this would explain why you can only use 1 core. If not then something else is going on.

Communication can packetised by sending END tokens. Communication using the normal input and output operators (:> and :<) on normal (non-streaming) channels is packetised. So long as there is at least one link available (after taking into account streaming channels), then you can have as many channels using packetised communication as you want (until you run out of channel end resources).

But you can't send control tokens in both directions, because it will block.

I don't understand what you mean here - could you give an example.

We really need a "token available" instruction! As mentioned in the other thread, there is a control register to read, but I will re-work the software instead.

You can also use events to check if there is data available. Configure the channel end to event and enable then disable events on the current thread using setsr. If there is data available the channel will event immediately and thread will jump to channel end's event handler. If there is no data available then execution will continue. This is how the XC compiler implements select statements that have a default case.

[bearcat]

Thanks Richard for the reply. I had also thought maybe it was link related. So it is. That's why it worked on a single core, as (I believe) it has more internal links.

My system has 2 L1's with a single link between them. Looks like I am going to have to learn more about channels and tokens now as this presents more issues.

Is the end token the XS1_CT_END, as defined? I assume it is. So, if a thread ends without sending an end toekn, does this block the channel, etc, etc.. More to discover...

Using events is something I am going to have to learn soon as I need it to generate an interrupt from a digital in. It is interesting to hear that's how XC handles the select. I had been wondered how the case inuint_byref worked, as there doesn't seem to be an instruction to do this.

Now I am wondering if I should write my own packet driver or not and what overheads each method has. I estimate only needing about 20% utilization on the link, so maybe not critical either way. I do need another channel between the cores as I progress, but I could tunnel this through the existing channel and probably is the easiest since the throughput is low.

Again, thanks for the response.

[dave]

You can set up several channels between your two cores that share the same link.

However, to make this work you should send messages consisting of a series of data
or control tokens and ending with an EOM token. At the receiving thread, you can
check for the EOM and discard it with a CHKCTI instruction.

The path through the link is set up as soon as you send the first token and is
released as soon as you send the EOM, ready for a new message - which could
be using a different channel connecting different threads.

A further point is that you may want to ensure that the destination thread is
ready to input a message before you start to send it; this will avoid the link becoming
blocked by the message until the destination is ready. To do this, you can first
send a message consisting only of an EOM; wait for a similar EOM message from the
destination thread; then send the message (ending with an EOM).

The XC compiler will do all of this automatically if you use normal channels between
your two chips instead of streaming channels.

[lilltroll]

Using Many Client channel ends -> 1 server channel end

What happens with control tokens that are sent to a loose end?
If 3 clients are sending a EOM tokens to the same destination at the same time, maximum one will reach the receiver, (the one that the receiver has connected to)
What happens with the other 2 tokens? And what way is best to handle this?
Do I need to apply something like random time retry, or should the receiver send tokens to the clients to ask, or .....
I can think of several ways, but I do not know which one that is the best.

[paul]

In reply to your post here: https://www.xcore.com/forum/viewtopic.php?p=6589#p6589 (which says you didn't quite get your answer to your original question).

If you are using XC and not using streaming channels then it should do the arbitration for you. The only situation where it will look like this is not working is when you have a thread blocking because it is not receiving the data. The channel will remain open until the other end picks up the data.

Using these 'normal' channels (just declared using the chan and chanend types) you would see something like this on the producer and consumer sides respectively:

Code: Select all

outct res[r10], 0x1
chkct res[r10], 0x1
out r0, res[r10]
outct res[r10], 0x1
chkct res[r10], 0x1

Code: Select all

chkct res[r10], 0x1
outct res[r10], 0x1
in r0, res[r10]
chkct res[r10], 0x1
outct res[r10], 0x1

Obviously in between the outct and chkct's you can put as many in/outs as you want if you want to send multiple lumps of data - this is what a transaction does.

This is all you should need for multiple channels across a single link. In fact if you are just using XC then this should all be transparent if you are not using streaming channels and your threads are taking data off their respective channels in time.

Hope that helps,

[richard]

What happens with control tokens that are sent to a loose end?
If 3 clients are sending a EOM tokens to the same destination at the same time, maximum one will reach the receiver, (the one that the receiver has connected to)
What happens with the other 2 tokens? And what way is best to handle this?

In this case no data is lost - all data will be received eventually.

A packet is formed from when you start sending data on a channel end to when you send a CT_END control token. If a packet arrives while a channel end is in the middle of receiving another packet then the new packet waits until the first packet has been input by the receiver. The waiting packet will sit in the buffers in the route between the channel ends, making links along that route unavailable for other traffic while it is waiting. If the packet is bigger than the available buffering then the sender thread will pause when the buffers become full.