L2 switch latency

[Redeye]

I'm currently working on a project that will use 3 x L2 (because I'm running out of ports, code space and MIPS on a G4). As far as I can tell, this is effectively daisy chaining 6 x L1 together, each via its own switch. So, if I need to send data from L2(0)Core0 to L2(2)Core1, there's 5 links and 6 switch hops along the way so presumably there's a latency in that data transfer? You can work out the latency of the links from the data rates depending on link configuration, but what about the switches?

[segher]

You can measure it easily: run code on the cores at the ends (or whichever
you want to measure the latency between). Have the core at the far end
simply ping-pong back whatever you write to it; on the local end, send
the current timestamp, and compare what is received with the current
timestamp.

You probably want to measure the latency for establishing a connection
separately; and the latency for XC channels, if you use those (they do
handshaking).

Btw, If you connect the L2s not in a straight line, (e.g., in a "T" shape), you
can get a network diameter of 4 instead of 5. Whether that is good for your
app is something you yourself can answer best :-)

[Redeye]

Thanks, yes I know I can measure it, but what I want to do is be able to predict it so I can work out whether there's going to be any problems with my design before I go to the expense of making my 3 x L2 board.

[segher]

You probably want an official answer, or at least from someone better informed
than me, but failing that let me give my best estimate.

Since you want speed, let's assume your external links are 5w and 80MHz.
With switch frequency 400MHz (the default), I estimate 55ns delay per external
link, and 15ns per internal link. So that is 155ns for your three-L2 chain.

This estimate is based on the switch delay being the flight time for a single
token (which is what the switch buffers before sending it on), plus 5ns (two
ticks).

YMMV but now you at least have _some_ number :-)