Most efficient way to pass variable-length data thru Interface
Posted: Thu Feb 09, 2017 5:11 pm
This is kind of a general question. What is the most efficient way to pass variable-length data through an interface in XC? Say, a null-terminated string or a variable-length packet of data like ethernet?
My understanding is that if we want to be able to pass from one tile to another, we can't use pointers or shared memory. So these are the other options I am aware of, (along with some implications according to my understanding):
1) Pass a fixed-size array, large enough to hold the largest possible packet (arrays are apparently always passed "by reference"?)
2) Break the packed into smaller fixed-size chunks and make multiple calls
3) Pass individual bytes or words using a raw channel. (handshaking for every word?!!)
4) Use a streaming channel (ties up resources for the life of the thread?)
5) Do some low-level I/O, using tokens to create your own protocol (seems non-standard, and possibly a lot of work)
My thoughts on each of the above:
1. This seems like the "preferred" approach, but does AKAIK involve using a fixed-size array. The client passes an array to the server (by reference) during an interface call, and the server writes data into it. XC and the hardware take care of marshaling this between tiles if necessary. This approach is probably ok for me, but I have a could questions about efficiency: Let's say that the largest packet is 4 kBytes, but most packets are less than 256 bytes, and packets are also transmitted infrequently (making a streaming channel overkill). So, to handle this, the client keeps a 4k array in memory, and passes it to the server on each interface call. The server writes data into the array and returns the size written. QUESTION: Assuming the server always writes to, and never reads from the array, what happens when this interface call goes across tiles. Does the entire contents of the array get copied over the link from client to server? I read somewhere that only "changes" are copied back from server to client; that's great. But if the server never needs to read from the array, only write to it, it seems like copying the all the data to the server is a waste of time and bandwidth.
2. I am guessing that each "call" involves some non-trivial amount of token "handshaking". So the inefficiency would be a trade off between moving garbage data (as in 1. above), vs. more handshaking.
3. This would avoid copying _any_ garbage data (see 1. above), but adds a the overhead of "handshaking" for every Byte or Word transmitted. BTW: Is data type preserved? What happens if I output a word to a regular chanend, and input a byte on the other side?
4. Seems like streaming channels will work very efficiently, but you lose the nice structure of "interfaces", and tie up some streaming channel resources permanently. (assume the streaming chanend is passed to two threads that each run in an infinite loop).
5. Apparently I can do whatever I want with the low-level I/O function calls, but I have to invent my own protocol entirely. Plus it won't be pretty, safe, XC.
Any ideas here, or am I just missing something really obvious?
Thanks!
My understanding is that if we want to be able to pass from one tile to another, we can't use pointers or shared memory. So these are the other options I am aware of, (along with some implications according to my understanding):
1) Pass a fixed-size array, large enough to hold the largest possible packet (arrays are apparently always passed "by reference"?)
2) Break the packed into smaller fixed-size chunks and make multiple calls
3) Pass individual bytes or words using a raw channel. (handshaking for every word?!!)
4) Use a streaming channel (ties up resources for the life of the thread?)
5) Do some low-level I/O, using tokens to create your own protocol (seems non-standard, and possibly a lot of work)
My thoughts on each of the above:
1. This seems like the "preferred" approach, but does AKAIK involve using a fixed-size array. The client passes an array to the server (by reference) during an interface call, and the server writes data into it. XC and the hardware take care of marshaling this between tiles if necessary. This approach is probably ok for me, but I have a could questions about efficiency: Let's say that the largest packet is 4 kBytes, but most packets are less than 256 bytes, and packets are also transmitted infrequently (making a streaming channel overkill). So, to handle this, the client keeps a 4k array in memory, and passes it to the server on each interface call. The server writes data into the array and returns the size written. QUESTION: Assuming the server always writes to, and never reads from the array, what happens when this interface call goes across tiles. Does the entire contents of the array get copied over the link from client to server? I read somewhere that only "changes" are copied back from server to client; that's great. But if the server never needs to read from the array, only write to it, it seems like copying the all the data to the server is a waste of time and bandwidth.
2. I am guessing that each "call" involves some non-trivial amount of token "handshaking". So the inefficiency would be a trade off between moving garbage data (as in 1. above), vs. more handshaking.
3. This would avoid copying _any_ garbage data (see 1. above), but adds a the overhead of "handshaking" for every Byte or Word transmitted. BTW: Is data type preserved? What happens if I output a word to a regular chanend, and input a byte on the other side?
4. Seems like streaming channels will work very efficiently, but you lose the nice structure of "interfaces", and tie up some streaming channel resources permanently. (assume the streaming chanend is passed to two threads that each run in an infinite loop).
5. Apparently I can do whatever I want with the low-level I/O function calls, but I have to invent my own protocol entirely. Plus it won't be pretty, safe, XC.
Any ideas here, or am I just missing something really obvious?
Thanks!