Wait for interrupt and sleep

[rweickelt]

Hi,

when having a resource e.g. a channel configured to generate interrupts on incoming data. Is it possible to suspend the thread while waiting? Waiteu seems to work for events only. Or is it possible to switch into event mode when becoming idle? What has to be done in this case? A guess:

1. Disable interrupts in the status register
2. Reconfigure all acquired resources to generate events
3. Register a wake-up handler with all acquired resources
4. Enable events in the status register
5. waiteu

The wake-up handler would reconfigure the resources to react on interrupts before the event is handled. But I see a lot of room for race conditions here. What happens if interrupts/events are generated while the reconfiguration process?

[lilltroll]

Maybe this answers your question:
https://www.xmos.com/download/public/Th ... 1.0%29.pdf @ paperpage 10
Certain instructions cause threads to become non-runnable because, for example, an
input channel has no available data. When the data becomes available, the thread will
continue from the point where it paused. A ready request to a thread must be received
and an instruction issued rapidly in order to support a high rate of input and output.
To achieve this, each thread has an individual ready request signal. The thread identifier
is passed to the resource (port, channel, timer etc) and used by the resource to select
the correct ready request signal. The assertion of this will cause the thread to be restarted,
normally by re-entering it into the round-robin sequence and re-issuing the input
instruction. In most situations this latency is acceptable, although it results in a response
time which is longer than the virtual cycle time because of the time for the re-issued
instruction to pass through the pipeline.
To enable the virtual processor to perform one input or output per virtual cycle, a fastmode
is provided. When a thread is in fast-mode, it is not de-scheduled when an instruction
can not complete; instead the instruction is re-issued until it completes.

If you are not running in fast mode, the thread will automatically be paused /de-scheduled until data becomes available.

[rweickelt]

Many thanks to You for the detailed answer and referring to the manual.

I think it is clear to me, that threads are paused while waiting for events. But can we achieve the same behaviour when using interrupts? If an idle thread waits for an interrupt, then it is semantically the same thing as a thread voluntarily waiting for an event.

[lilltroll]

Check out chapter 16 in the same document.

Each thread has a Status Register SR that holds

EEBLE enable events
IEBLE enable interrupts
INENB determine if thread is enabling events
ININT determine if thread is in interrupt mode
INK determine if thread is in kernel mode
SINK determine if thread was in kernel mode
WAITING determine if thread is waiting to execute the current instruction
FAST determine if thread is in fast mode

and

In contrast to events, interrupts can occur at any point during program execution, and
so the current pc and sr (and potentially also some or all of the other registers) must
be saved prior to execution of the interrupt handler. This is done using the spc and ssr
registers. On an interrupt generated by resource r the following occurs automatically:

spc <- pc;
ssr <- sr ;
pc <- vres;
sed <- ed;
ed <- evres
sr [bit inint] <- true
sr [bit ink] <- true;
sr [bit eeble] <- false;
sr [bit ieble] <- false
sr [bit waiting] <- false

Meaning: If you have a thread that is paused/waiting since it is waiting for data on a channel, the statusbit will be set to waiting automatically.
If an interrupt happens, the status register will be saved to the shadow status register ssr, and thereafter the sr will be changed to not waiting automatically.
During the interrupt events also becomes disabled.
On return from the interrupt the old status register will be copied back to the original state.

KRET pc <- spc; return from interrupt
sr <- ssr
ed <- sed

So the thread will go back to waiting automatically (and any events becomes enabled again), until new channel data arrives.

[rweickelt]

Thanks again. I really appreciate Your help, although I've been aware of most facts. As I've written in my initial question: I have interrupts already running and just do not know, how to suspend an idle thread so that it resumes after an interrupt has occurred and the interrupt handler has done its job. That's how it works on other microcontrollers. XCORE is different here.

Nevertheless, I think that I've found a solution, which works somehow, but I do not foresee all consequences which this brings along. The thread that I am running, is a preemptive dispatcher for short run-to-completion tasks. This is different to XC where everything is cooperative.

Main thread:

1. Configure all used resources to generate interrupts
2. When done, suspend with waiteu, but leave interrupts enabled

Now, when an interrupt occurs, the following sequence is executed:

1. Run an interrupt handler
2. Before returning to the main thread, dispatch all tasks that might have been generated by the interrupt handler.

Dispatching involves

1. Picking up the next ready task
2. Execute the task with interrupts enabled

This means, that the main thread never gets over the waiteu instruction and that events are always dispatched in kernel mode. A problem is, that I might end up with stacked dispatchers because tasks are executed with interrupts enabled. But this is easy to solve.

But the more I think about that, the more I get unsure, if having a preemptive event-driven system (which does a good job on an uniprocessor system) can outdo a massive amount of cooperative communication sequential processes.

I am really curious, what other people think about that, but I guess, that this discussion dates back to the 80s.

[lilltroll]

Maybe I'm not understanding your question but.

If you have a channel waiting for incoming data

Something like (xc)

Code: Select all

c:> x;

That will translate to the instruction:
in for a streaming channel
or
inct for a (standard) channel

Both these instruction will deschedule the thread until data becomes available automatically.
I do not think you need to do anything at all regarding "how to suspend an idle thread"
The answer is that an idle thread will always be suspended if it is not set to run I fast mode.

Regarding the interrupt handler part I do not know, but you may find some examples on GitHub using interrupts!
Here is one example in the Ethernet module
https://github.com/xcore/sc_ethernet/bl ... terrrupt.S

Myself I have only written asm code using events.

[rweickelt]

Thank You for the link. I did not know, that the ethernet module uses interrupts.

My original question was, if the XCORE provides something like a "suspend with interrupts enabled and resume from that point after an interrupt has been handled" functionality. This is standard in most CPUs that I have used so far. Something like the waiteu instruction, not waiting for events, but interrupts. In my application, all resources are configured to produce interrupts rather than events. Interrupts allow preemption, events do not.

The answer seems to be "no", but I found a workaround. I acknowledge, that the XCORE architecture is unlike traditional CPUs.

Do You get it now?

[lilltroll]

Maybe I get it now, so what about the instruction SETSR and CLRSR !?

Set bits in SR
Set bits in the thread’s Status Register. The mask supplied specifies which bits should
be set. Note that setting the EEBLE bit may cause an event to be issued, causing subsequent
instructions to not be executed (since events do not save the program counter).
Setting IEBLE may cause an interrupt to be issued.
CLRSR is used to clear bits in the status register.

Enable interrupts , disable events and enable wait.

[rweickelt]

Setting IEBLE may cause an interrupt to be issued, but only if an interrupt has occurred. That's not the point. Interrupts work in my application, even when suspending with waiteu. Again, my problem is, that I want to resume after the waiteu instruction, once an interrupt has been handled.

Reading the architecture manual again opened my eyes:

WAITEU
Waits for an event. This instruction sets EEBLE and, if no event is ready it will suspend
the thread until an event becomes ready. When an event is available, the thread will
continue at the address specified by the event. The current PC is not saved anywhere.

That's it! The waiteu instruction is semantically a leaf in the thread control flow graph. In no case, the thread is supposed to resume after this instruction.

[edit] Of course it's not really a leaf, but a wall or something else my limited vocabulary does not contain[edit]

But, wait, there's an important information:

The current PC is not saved anywhere.

Really? This brought me to the following experiment:

Code: Select all

/*============================================================================
 * Interrupt handler for a single-stack system.
 *==========================================================================*/
.cc_top interruptHandler.function, interruptHandler
.align 4
interruptHandler:
    // The program is in kernel mode now. That means, we are bound to kret
    // for return and can not use retsp. kret requires krestsp and therefore
    // ksp has to be sp to achieve a single-stack.
    // I have not found a better way, yet.
    stw r0, dp[tempR0]    // Store r0 into scratch location
    ldw r0, sp[0]         // r0 = sp[0]
    stw sp, sp[0]         // Prepare modification of ksp
    krestsp 0             // ksp=sp+0, sp=ksp[0]
    stw r0, sp[0]         // Restore sp[0]
    ldw r0, dp[tempR0]    // Restore r0
    // Everything is sane now and ksp=sp

    kentsp 10         // Switch to kernel stack and allocate 10 words
                      // ksp[0]=sp, ksp-=10

    stw spc, sp[1]    // Store service registers...
    stw ssr, sp[2]
    stw sed, sp[3]
    stw lr,  sp[4]
    stw r0,  sp[5]    // ...and caller-save registers
    stw r1,  sp[6]
    stw r2,  sp[7]
    stw r3,  sp[8]
    stw r11, sp[9]

    get r11, ed       // Load handler index from 16bit event-data. ed has been
                      // loaded with an index i. Each interrupt source
                      // got its own index.
    ldc r0, 0xff      // Mask i because the high nibble in ed contains crap.
    and r0, r11, r0
    bl handle         // Finally call a high-level C-function handle(i)

    // Look, if the thread has been suspended by waiteu. In that case,
    // manipulate SPC to point to the next instruction in order to execute
    // a dispatcher. Otherwise just return as ususal.
    ldw r0, sp[1]
    ldap r11, __waiteuInstruction
    eq r11, r11, r0
    bf r11, __epilogue
    add r0, r0, 2     // waiteu is 2 bytes long.
    stw r0, sp[1]

__epilogue:
    ldw spc, sp[1]    // Restore service registers...
    ldw ssr, sp[2]
    ldw sed, sp[3]
    ldw lr,  sp[4]
    ldw r0,  sp[5]    // ...and caller-save registers
    ldw r1,  sp[6]
    ldw r2,  sp[7]
    ldw r3,  sp[8]
    ldw r11, sp[9]

    krestsp 10      // Contract the kernel stack and switch to user stack
                    // ksp+=10, sp=ksp[0]
    kret            // Return from kernel mode interrupt handler
                    // either to the _suspend function or to another
                    // interrupted program location.
.cc_bottom interruptHandler.function

/*============================================================================
 * Suspend the thread.
 *
 * Interrupts are enabled while sleeping and disabled when waking up.
 *==========================================================================*/
_suspend:
    setsr SR_IEBLE
__waiteuInstruction:
    waiteu          // Usually a thread is not supposed to resume after
                    // this instruction. Only _interruptHandler can do that
    clrsr SR_IEBLE
    retsp 0

 

In my high-level program I have something like a task loop:

Code: Select all

_disableInterrupts();
while(true)
{
    for (Task* task = readyList.takeFirst(); task != 0; task = readyList.takeFirst())
    {
        _enableInterrupts();
        dispatch(task);
        _disableInterrupts();
        
        // some more house keeping stuff...
        
    }

    _suspend();
}

 

Yes. It works! The saved program counter in the interrupt handler indeed points to the waiteu instruction where the thread has been suspended. After executing the interrupt handler, the program resumes after the waiteu instruction.

I think, this solves my original problem in an unorthodox way. :)

But one serious problem remains. There is still a possible race condition in _suspend because an interrupt could happen between setsr and waiteu while the program counter still points to _suspend and not _waiteu.

[lilltroll]

Trying to understand more about interrupts here.

Could hardware locks be used to avoid a race (Chapter 13)?

Mutual exclusion between a number of threads can be performed using locks. A lock is
allocated using a GETR l, LOCK instruction. The lock is initially free. It can be claimed
using an IN instruction and freed using an OUT instruction.

Example code: https://github.com/xcore/sc_util/tree/m ... dule_locks

Or have you found another way to solve the problem ?