DragonFly kernel List (threaded) for 2003-11
:Since I'm very fond of the responsiveness effects
:of ULE in fbsd-current, I'd like to know, how this
:very responsiveness is supposed to be achieved
:in dragonfly via userland schedulers.
:As I understand it, the kernel-level thread-scheduling
:is not to be touched at all including time-slices.
:Also I understand that preemptive cpu-moves are against
:essential dragonfly design because of locking issues.
:(Which makes perfect sense for me, though I'm just an
: interested bystander/sysadmin.)
:I don't expect any complete explanations, just some pointers
:to relevant email-threads/documentation/ideas/whatever.
:I hope I don't sound too demanding...
Well, there are two things going on here... there's the
userland scheduler and there is the LWKT scheduler. The
LWKT scheduler is responsible for actually running a thread.
It uses a fixed priority scheme but you have to keep in mind
that the fixed priorities are differentiating major subsystems,
not user processes.
For example, hardware interrupt threads have the highest priority,
followed by software interrupts, kernel-only threads, then
finally user threads. A user thread either runs at user-kernel priority
(when it is actually running in the kernel, e.g. running a syscall on
behalf of userland), or a user thread runs at user priority.
The user scheduler uses the dynamic priority mechanism from FreeBSD-4.x.
It schedules *exactly* *one* user-runnable thread on each cpu in the
system at a time, and it is responsible for dealing with the timeslice
and forcing a reschedule to reschedule the next user-runnable thread.
What this means is that from the point of view of a user process,
preemption and dynamic priorities works as it always has. What has
changed is that preempt and dynamic priorities *ONLY* apply to threads
related to user processes now and are not used to schedule kernel-specific
activities such as interrupts. In FreeBSD-5 the same ULE mechanism is
use to schedule both user and kernel activities which imposes a great
deal of unnecessary overhead IMHO.
This also means that we do not have priority inversion based deadlocks,
because any thread that is in the kernel (whether working on behalf of
a user process, e.g. running a system call, or not), runs at a fixed
kernel priority which is higher then the priority we use when running a
process's thread that is currently in userland.
Be sure not to confuse user priority based time slicing with preemption.
A program running in user mode can be 'preempted' just fine. It's when
a thread is running in the kernel that preemption is either not allowed
or follows careful rules.
DragonFly does preempt... it just does it very carefully and only
under particular circumstances. An LWKT interrupt thread can preempt
most other threads, for example. This mimics what FreeBSD-4.x already
did with its spl/run-interrupt-in-context-of-current-process mechanism.
What DragonFly does *NOT* do is allow a non-interrupt thread to preempt
another non-interrupt thread. FreeBSD-5 uses priority borrowing to
allow virtually any thread to preempt virtually any other thread, either
directly or indirectly. This is a huge mistake IMHO which allows for
very sloppy interrupt thread programming. FreeBSD-5 will also preemptively
switch a thread to another cpu. DragonFly never preemptively switches a
thread to another cpu if the thread is running in the kernel.