Lightest weight cross-tile synchronisation Topic is solved

If you have a simple question and just want an answer.
User avatar
lukehatpadl
Active Member
Posts: 34
Joined: Sat Jul 08, 2023 5:15 am

Lightest weight cross-tile synchronisation

Post by lukehatpadl »

What's the lightest weight way to synchronise between tiles (well, I suppose this is a specific instance of the general cross-core question).

Currently I'm doing this:

Code: Select all

      // synchronise with other tile
      if (bIsPrimaryTile) {
        outct(resetEpoch, XS1_CT_END);
        chkct(resetEpoch, XS1_CT_END);
      } else {
        chkct(resetEpoch, XS1_CT_END);
        outct(resetEpoch, XS1_CT_END);
      }
    }
View Solution
User avatar
xhuw
Verified
Member++
Posts: 23
Joined: Wed May 22, 2024 2:36 pm

Post by xhuw »

As each chanend has an internal buffer you could save the conditional check and just do

```
outct(resetEpoch, XS1_CT_END); // does not block
chkct(resetEpoch, XS1_CT_END); // blocks
```

Of course, this comes with the usual risks with chanends that if either thread does something unexpected with the chanend then it will trap.

(xmos employee)
XMOS Software Engineer

Image
User avatar
lukehatpadl
Active Member
Posts: 34
Joined: Sat Jul 08, 2023 5:15 am

Post by lukehatpadl »

Ah, nifty!

My use case is to ensure that after the two tiles synchronise with chkct(), pinseq() aligns them to the same LRCLK edge (LRCLK is being fed to both tiles).
User avatar
xhuw
Verified
Member++
Posts: 23
Joined: Wed May 22, 2024 2:36 pm

Post by xhuw »

I see, you could probably come up with some scheme that's a bit more thorough to guarantee synchronisation. The risk with this scheme is that it can take some (small amount of) time for the token to make it across to the other tile. So you might want to wait for an edge on the LRCLK on one of the tiles before initiating the sync so that you have a whole LRCLK period for exchanging the tokens. Rather than risking the exchange happening right on top of an edge which might lead to a timing error.
XMOS Software Engineer

Image
User avatar
lukehatpadl
Active Member
Posts: 34
Joined: Sat Jul 08, 2023 5:15 am

Post by lukehatpadl »

Another great idea. Thank you for the excellent support!
User avatar
lukehatpadl
Active Member
Posts: 34
Joined: Sat Jul 08, 2023 5:15 am

Post by lukehatpadl »

Interestingly where I try the simplified code suggested in your first reply, the second tile seems to hang waiting, at least when running with debug_printf() enabled:

Code: Select all

#784667
b100 0 // T1_Send (NB used different value to example below)
#784687
b0 0 // T0_Send
#2428351
b0 0 // T0_Send
#2428371
b0 0 // T0_Send
#2428374
b0 0 // T0_Send
#2428388
b0 0 // T0_Send
When running under xgdb I saw T1 hanging in some cases but not always.

With the original code, with debug_printf() enabled:

Code: Select all

#784686
b100 0 // T1_Wait
#2428369
b0 0 // T0_Send
#2428388
b1 0 // T0_Wait
#2428392
b101 0 // T1_Send
#2428409
b110 0 // T1_Complete
#2428414
b10 0 // T0_Complete
Haven't looked into this further yet.

Edit: fixed some typos in second example.
Last edited by lukehatpadl on Sat Jul 27, 2024 10:49 am, edited 2 times in total.
User avatar
lukehatpadl
Active Member
Posts: 34
Joined: Sat Jul 08, 2023 5:15 am

Post by lukehatpadl »

Code below, if it's not obvious :)

Code: Select all

        // synchronise with other tile
        if (bIsPrimaryTile) {
          TRACE_XTILE_SYNC(T0_Send);
          outct(resetEpoch, XS1_CT_END);
          TRACE_XTILE_SYNC(T0_Wait);
          chkct(resetEpoch, XS1_CT_END);
          TRACE_XTILE_SYNC(T0_Complete);
        } else {
          TRACE_XTILE_SYNC(T1_Wait);
          chkct(resetEpoch, XS1_CT_END);
          TRACE_XTILE_SYNC(T1_Send);
          outct(resetEpoch, XS1_CT_END);
          TRACE_XTILE_SYNC(T1_Complete);
        }
User avatar
xhuw
Verified
Member++
Posts: 23
Joined: Wed May 22, 2024 2:36 pm

Post by xhuw »

Hey, there is a chance this is an xc vs. C thing as I am more used to C in lib xcore.

I coded up an MVP to check my idea shouldn't block

In principle a there is a FIFO in both directions between 2 chanends. the FIFO can fit a token on it. However doing an `outct` to a chanend whos FIFO is already full will block. So to rely on the non-blocking behaviour you need to be certain that the FIFO is not full.

The usual "channel" API which is invoked when you use the `<:` operator in XC or "xcore/channel.h" in C has been designed to ensure the two threads are fully synchronised, but this will be more heavy weight with multiple transactions occurring and therefore possibly taking longer.
XMOS Software Engineer

Image
User avatar
lukehatpadl
Active Member
Posts: 34
Joined: Sat Jul 08, 2023 5:15 am

Post by lukehatpadl »

This is what I did in the end:

Code: Select all

// synchronise with other tile
if (bIsPrimaryTile) {
  outct(resetEpoch, XS1_CT_END);
  chkct(resetEpoch, XS1_CT_END);
} else {
  // XMOS support recommends an additional LRCLK period on one of the
  // tiles to handle inter-tile latency
  // https://www.xcore.com/viewtopic.php?p=41267#p41267
  *lrClkIn when pinseq(0) :> void; 
  *lrClkIn when pinseq(1) :> void; 

  chkct(resetEpoch, XS1_CT_END);
  outct(resetEpoch, XS1_CT_END);
}       
I am still running into some deadlocks but I think they are occurring elsewhere in the code. This stuff is challenging to debug!