Channel ids in simulation and execution of the XMP-64

[Jamie]

I'm making the assumption that the first channel returned by GETR has resource counter value 0. In the simulations I have run on with xsim this is true, but is not the case when actually running the code on the hardware. Core 0 on each node returns an identifer with resource counter 1 and the rest with 0. The network bringup loadable is untouched and runs fine, and I've disabled the runtime initialisation so only my code runs in the second loadable.

Is this behaviour expected?

[m_y]

The hardware (and simulator) will already return the lowest number resource that has not already been allocated. If you're getting channel 1 back it can only be because 0 has already been allocated. The boot ROM is a prime candidate in many cases.

It's for this reason that the standard startup code allocates all channel ends (i.e. until getr returns 0) then frees them all (except for 0 but only so that it can use it itself). It does this on all cores.

[Jamie]

Thanks for explaining that.

The boot ROM is a prime candidate in many cases.

Do you mean that a channel might not be freed by the network bringup phase?

[m_y]

The boot ROM is a prime candidate in many cases.

Do you mean that a channel might not be freed by the network bringup phase?

I'm pretty sure the network bringup code frees everything that it uses. However the boot ROM may not. xsim doesn't run the boot ROM by default so it would explain the discrepency you're seeing with the hardware.

When powered up in boot-from-link mode, the boot ROM allocates and listens on a channel end. When the debugger takes control via jtag it loads stuff into memory, changes the PC and so forth but it does not free the channel end. Thus when your application starts channel end 0 is already allocated (but it is safe to free it (or to use it after a setd)).

[segher]

The boot rom frees the channel end it uses fine, as far as I can see. Maybe this system
boots from OTP, instead?

[Jamie]

Thanks for your help, I now have channels working properly, and I have been able to run several simple examples on the XMP-64 but I'm getting some deadlock situations occuring. They seem to be related to channel communication between cores and I suspect this is due to the staggered execution start times caused by the JTAG loader. I know there is a synchronisation phase in the C/XC rutime to mitigate this effect but I'm not sure exactly how it works. I have tried implementing my own version by creating a pipeline and sending a token up then back down, but this immediately deadlocks when run on the hardware (not in the simulator). My only guess here would be that the first token is sent before the receiving channel end has even been allocated and is therefore lost, but I'm not sure...

Could you give me any idea about how this is performed in the XC runtime or just the best way to go about doing this?

Thanks!

[Jamie]

I know there is a synchronisation phase in the C/XC rutime to mitigate this effect but I'm not sure exactly how it works. I have tried implementing my own version by creating a pipeline and sending a token up then back down, but this immediately deadlocks when run on the hardware (not in the simulator). My only guess here would be that the first token is sent before the receiving channel end has even been allocated and is therefore lost, but I'm not sure...

I've been able to get around this by adding a long wait on core 0 before running the synchronisation, but this is obviously the wrong way to go about it...

[m_y]

Sorry for the delay in replying - I've been away.

The XMOS startup code uses a spare register in the switch (register number 3) of node 0 to synchronise. This is referred to at the scratch register.

On power up the scratch register has value 0. When node 0, core 0 is ready to synchronise it writes 1 to it then polls, waiting for it to reach num_cores.

Each of the other cores is assigned a unique number in the range 1..num_cores-1 (let's call it boot_id) in the linker-generated code. At startup each core polls the scratch register waiting for it to reach boot_id. When it does, it sets it to boot_id+1.

In order to reach num_cores each of the cores must have detected its own number and written the next. Thus they are all known to have reached a known point in the boot.

[Jamie]

Brilliant, thanks for explaining that.

Just to note though: on page 18 of the XS1-G system manual, the first and second bytes of the configuration resource identifier seem to be the wrong way round; byte 1 should be SSCTRL and byte 2 just 0.

[larry]

Just to note though: on page 18 of the XS1-G system manual, the first and second bytes of the configuration resource identifier seem to be the wrong way round; byte 1 should be SSCTRL and byte 2 just 0.

Thanks, Jamie. I'll get that corrected.