Comtrol eCos - Manual (Page 36)

SMP Support
routines that can. Instead all such operations are deferred to an associated DSR routine that is run during the lock
release operation, when the data structures are in a consistent state.
By choosing a kernel locking mechanism that does not rely on interrupt manipulation to protect data structures,
it is easier to convert eCos to SMP than would otherwise be the case. The principal change needed to make eCos
SMP-safe is to convert the scheduler lock into a nestable spin lock. This is done by adding a spinlock and a CPU
id to the original counter.
The algorithm for acquiring the scheduler lock is very simple. If the scheduler lock’s CPU id matches the current
CPU then it can just increment the counter and continue. If it does not match, the CPU must spin on the spinlock,
after which it may increment the counter and store its own identity in the CPU id.
To release the lock, the counter is decremented. If it goes to zero the CPU id value must be set to NONE and the
spinlock cleared.
To protect these sequences against interrupts, they must be performed with interrupts disabled. However, since
these are very short code sequences, they will not have an adverse effect on the interrupt latency.
Beyond converting the scheduler lock, further preparing the kernel for SMP is a relatively minor matter. The main
changes are to convert various scalar housekeeping variables into arrays indexed by CPU id. These include the
current thread pointer, the need_reschedule flag and the timeslice counter.
At present only the Multi-Level Queue (MLQ) scheduler is capable of supporting SMP configurations. The main
change made to this scheduler is to cope with having several threads in execution at the same time. Running threads
are marked with the CPU that they are executing on. When scheduling a thread, the scheduler skips past any running
threads until it finds a thread that is pending. While not a constant-time algorithm, as in the single CPU case, this
is still deterministic, since the worst case time is bounded by the number of CPUs in the system.
A second change to the scheduler is in the code used to decide when the scheduler should be called to choose a
new thread. The scheduler attempts to keep the n CPUs running the n highest priority threads. Since an event or
interrupt on one CPU may require a reschedule on another CPU, there must be a mechanism for deciding this. The
algorithm currently implemented is very simple. Given a thread that has just been awakened (or had its priority
changed), the scheduler scans the CPUs, starting with the one it is currently running on, for a current thread that
is of lower priority than the new one. If one is found then a reschedule interrupt is sent to that CPU and the scan
continues, but now using the current thread of the rescheduled CPU as the candidate thread. In this way the new
thread gets to run as quickly as possible, hopefully on the current CPU, and the remaining CPUs will pick up the
remaining highest priority threads as a consequence of processing the reschedule interrupt.
The final change to the scheduler is in the handling of timeslicing. Only one CPU receives timer interrupts, although
all CPUs must handle timeslicing. To make this work, the CPU that receives the timer interrupt decrements the
timeslice counter for all CPUs, not just its own. If the counter for a CPU reaches zero, then it sends a timeslice
interrupt to that CPU. On receiving the interrupt the destination CPU enters the scheduler and looks for another
thread at the same priority to run. This is somewhat more efficient than distributing clock ticks to all CPUs, since
the interrupt is only needed when a timeslice occurs.
All existing synchronization mechanisms work as before in an SMP system. Additional synchronization mecha-
nisms have been added to provide explicit synchronization for SMP, in the form of
spinlocks .
SMP Interrupt Handling
The main area where the SMP nature of a system requires special attention is in device drivers and especially
interrupt handling. It is quite possible for the ISR, DSR and thread components of a device driver to execute on
different CPUs. For this reason it is much more important that SMP-capable device drivers use the interrupt-related
36