Streaming chanend, cores on same tile - buffer capacity?

[tuck1s]

I've got a producer and consumer task linked with a single streaming chanend. My consumer task is 'bursty' - it's committing data to an SD card and has to wait for the card to finish each write operation.

My streaming channel seems to have fairly limited (internal, hidden from xC) depth - I'm guessing around 200-400 bytes. Is there a way to
a) read out the channel depth to check it?
b) preset the channel depth (I want to basically dedicate most of my unused RAM to it)?

[infiniteimprobability]

Hi,
streaming channel ends have buffering determined by the path - how many switches they go through. The minimum you would see is 1 x 32b word (same tile) and could be up to 7 words (32b each) if you are going across 2 switches, such as in the L16. You can't set the depth manually (other than tile placement of task).

You can't see if the channel is ready to accept - you can only do an out, which blocks if the consumer hasn't emptied enough space. You can check to see if there is something in the channel at the consumer end by polling:

Code: Select all

select{
  case c_streaming :> int val:
  break

  default: //Drop through if nothing to read
  break;
}

If you need a predictable/configurable FIFO, then I recommend writing a task to do that.

Somewhat related item - https://new.xcore.com/forum/viewtopic.php?f=26&t=3464

[tuck1s]

Great, thanks for your reply.
I expect the trick is to get the FIFO buffer task to not block on either input or output, but keep looking for work to do on both sides.

I found the following example which looks to be similar:
https://github.com/xcore/sw_thread_comm ... ed_example

I also made up a simple test program to show that (on the StartKit cores) - as you expected - a producer blocks on the third 32-bit-write:

Code: Select all

/*
 * SimpleChannelTest.xc
 *
 *  Created on: 19 Oct 2015
 *      Author: steve
 */
#include <stdio.h>
#include <xs1.h>

void producer(streaming chanend c)
{
    unsigned i=1;
    while(i<=10) {
        printf("producer sends %8d\n", i);
        c <: i++;
    }
    while(1);                                   // Busy-wait at the end of the task
}

void consumer(streaming chanend c)
{
    unsigned v;
    delay_milliseconds(1000);                   // Deliberately wait to show up buffering
    printf("Now consuming\n");
    while(1) {
        c :> v;
        printf("\t\tconsumer = %8d\n", v);
    }
}

int main(void)
{
    streaming chan c;
    par {
        producer(c);
        consumer(c);
    }
    return 0;
}

Giving results:

Code: Select all

producer sends        1
producer sends        2
producer sends        3
Now consuming
		consumer =        1
producer sends        4
		consumer =        2
producer sends        5
		consumer =        3
producer sends        6
		consumer =        4
producer sends        7
		consumer =        5
producer sends        8
		consumer =        6
producer sends        9
		consumer =        7
producer sends       10
		consumer =        8
		consumer =        9
		consumer =       10

[tuck1s]

The github example https://github.com/xcore/sw_thread_comm ... ed_example

appears to not work as expected (on my StartKit, at least).
The tasks run, and there is interleaved printout. But thread2 is only reporting "busy" / "not busy". It's not showing

Thread2: Received notification

or
Thread2: Received data.

Here's what I'm getting in an example run. Looks like the thread1 -> thread2 comms are not correctly triggering thread2.

Code: Select all

Thread1: Putting data into fifo
Thread2:                                                Busy.
Thread1: Notifying thread2 that data is available.
Thread2:                                                Not Busy.
Thread1: Putting data into fifo
Thread2:                                                Busy.
Thread1: Putting data into fifo
Thread2:                                                Not Busy.
Thread1: Putting data into fifo
Thread2:                                                Busy.
Thread1: Putting data into fifoThread2:                                                Not Busy.

Thread1: Putting data into fifo
Thread2:                                                Busy.
Thread1: Putting data into fifo
Thread2:                                                Not Busy.
Thread1: Putting data into fifo
Thread2:                                                Busy.
Thread1: Putting data into fifo
Thread2:                                                Not Busy.
Thread1: Putting data into fifo
Thread2:                                                Busy.
Thread1: Putting data into fifo
Thread2:                                                Not Busy.
Thread1: Putting data into fifo
Thread2:                                                Busy.Thread1: Putting data into fifo

Thread1: Putting data into fifoThread2:                                                Not Busy.

Thread1: Putting data into fifo
Thread2:                                                Busy.
Thread1: Putting data into fifo
Thread2:                                                Not Busy.
Thread1: Putting data into fifo
Thread2:                                                Busy.
Thread1: Putting data into fifo
Thread2:                                                Not Busy.
Thread1: Putting data into fifo
Thread2:                                                Busy.
Thread1: Putting data into fifo
Thread2:                                                Not Busy.
Thread1: Putting data into fifo
Thread2:                                                Busy.Thread1: Putting data into fifo

Thread1: Putting data into fifoThread2:                                                Not Busy.

Thread1: Putting data into fifo
Thread2:                                                Busy.
Thread1: Putting data into fifo
Thread2:                                                Not Busy.
Thread1: Putting data into fifo
Thread2:                                                Busy.
Thread1: Putting data into fifo
Thread2:                                                Not Busy.
Thread1: Putting data into fifo
Thread2:                                                Busy.
Thread1: Putting data into fifo
Thread2:                                                Not Busy.
Thread1: Putting data into fifo
Thread2:                                                Busy.Thread1: Putting data into fifo

Thread1: Putting data into fifoThread2:                                                Not Busy.

Thread1: Putting data into fifo
Thread2:                                                Busy.
Thread1: Putting data into fifo
Thread2:                                                Not Busy.
Thread1: Putting data into fifo
Thread2:                                                Busy.
Thread1: Putting data into fifo
Thread2:                                                Not Busy.
Thread1: Putting data into fifo
Thread2:                                                Busy.
Thread1: Putting data into fifo
Thread2:                                                Not Busy.
Thread1: Putting data into fifo
Thread2:                                                Busy.Thread1: Putting data into fifo

Thread1: Putting data into fifoThread2:                                                Not Busy.

Thread1: Putting data into fifo
Thread2:                                                Busy.
Thread1: Putting data into fifo
Thread2:                                                Not Busy.
Thread1: Putting data into fifo
Thread2:                                                Busy.
Thread1: Putting data into fifo
Thread2:                                                Not Busy.
Thread1: Putting data into fifo
Thread2:                                                Busy.
Thread1: Putting data into fifo
Thread2:                                                Not Busy.

[tuck1s]

So I rolled my own fifo task - using similar construction to the Github example - and this seems to work nicely even under overflow & high traffic. Would appreciate any comments/feedback in case there are coding practices I should improve.

Here it is:

Code: Select all

/*
 * SimpleChannelTest.xc
 *
 *  Created on: 19 Oct 2015
 *      Author: steve
 */
#include <stdio.h>
#include <xs1.h>
#include <platform.h>

timer t;
// Functions to get the time from a timer.
unsigned int get_time(void) {
    unsigned time;
    t :> time;
    return time;
}


const unsigned clkPeriod = 690;                     // microseconds * 10ns XMOS timer period - tweak this for torture testing
timer t2;
void producer_task(streaming chanend cout)
{
    unsigned i=1;
    unsigned tick;
    printf("Producing....\n");
    delay_microseconds(10000);                      // At very high throughputs, give printf()s a chance to get done before we start

    t2 :> tick;                                     // grab the current timer ref value
    while(1) {
        select {
           case t2 when timerafter(tick) :> void:   // perform periodic task
             tick += clkPeriod;
             // printf("producer sends %8d\n", i);
             cout <: i++;
             break;
         }
    }
}

#define reportingInterval 1000000                   // only print every n good receptions to the console
void consumer_task(chanend cin)
{
    unsigned v;
    char ct;
    unsigned vchk = 1;                              // Test scaffolding only
    unsigned T;                                     // ""
    unsigned consumed_ctr = 0;                      // ""

    //delay_microseconds(5);                         // Deliberately wait to show up buffering / overflow handling on slow consumers
    printf("Ready to consume ...\n");

    T = get_time();
    while(1) {
        select {
          case inct_byref(cin, ct):
              // The other thread has notified us that data is ready
              // Signal that we wish to consume the data
              cin <: 0;
              cin :> v;                         // This should not block due to the case above

              // fixme: *** test code only:  check the values are expected, and print some stats
              consumed_ctr++;
              if((consumed_ctr % reportingInterval) ==0) {
                  unsigned now = get_time();
                  printf("Consumed %d values in %d ms\n", reportingInterval, (now-T)/100000);
                  T=now;
              }
              if(v == vchk) {
                  vchk++;                       // good - keep going
              }
              else {
                  printf("!!! Unexpected value %d consumed.  Resyncing.\n", v);
                  vchk = v + 1;                 // That's what we're expecting next
              }
              // fixme: *** end of test code
              break;
          // Could do other event-driven stuff in here
        }
        //delay_microseconds(1000);                // fixme: test code only, Make the receiver slow, so the buffer fills
    }
}

// fifo buffer
#define bufSize 0x2000          // MUST be a power of two, to permit logical-AND wraparounds
#define bufMask 0x1fff          // MUST be one less than bufSize

// Consume values from c
// Produce values out to a (non-streaming) channel through use of control tokens
void fifo_task(streaming chanend c, chanend d)
{
    unsigned bufHead = 0;                   // head = tail and count = 0 when empty
    unsigned bufTail = 0;
    unsigned bufCount = 0;
    unsigned notified = 0;                  // Used for control-token passing
    unsigned buf[bufSize];

    while(1) {
        select {
          case c :> unsigned v:
            // Add new value to the buffer head
            if(bufCount < bufSize) {           // we have space
                buf[bufHead++] = v;
                bufHead &= bufMask;             // Cheap way to do wraparound
                bufCount++;
            }
            else {
                printf("*");                    // fixme: Test code only:  Houston we have a problem, buffer overflow
            }
            if (!notified) {                    // Wake up the downstream consumer
              outct(d, XS1_CT_END);
              notified = 1;
            }
            break;

          case d :> int request:
            d <: buf[bufTail++];                // Don't need master/slave stanza for single uint32 values
            bufTail &= bufMask;                 // Cheap way to do wraparound
            if(--bufCount==0) {
                notified = 0;                   // If buffer's empty we'll need to renotify later
            }
            else {
                outct(d, XS1_CT_END);
            }
            break;
        }
    }
}

int main(void)
{
    streaming chan c;
    chan d;

    // This version connects with a fifo task, thus:
    // producer -> fifo -> consumer
    par {
        producer_task(c);
        fifo_task(c, d);
        consumer_task(d);
    }
    return 0;
}

on StartKit this gives

Code: Select all

Producing....
Ready to consume ...
Consumed 1000000 values in 6899 ms
Consumed 1000000 values in 6900 ms
Consumed 1000000 values in 6900 ms
Consumed 1000000 values in 6900 ms
Consumed 1000000 values in 6900 ms
Consumed 1000000 values in 6900 ms
Consumed 1000000 values in 6900 ms
Consumed 1000000 values in 6900 ms

The producer throughput, startup delay and consumer startup delay can be tweaked in the test program to deliberately provoke buffer overflow conditions for testing.

EDIT: Github version
https://github.com/tuck1s/SimpleFIFOTest

EDIT2: Now with four different test case setups added.

[tuck1s]

I came across this useful paper on buffering: https://download.xmos.com/XM-005982-PC-1.pdf