Re: Fwd: SPLs and their ex-function / replacement

[Matthew Dillon <dillon@xxxxxxxxxxxxxxxxxxxx>]

Hi, I remember that SPLs were removed from DragonFlyBSD, but having readwhat
they are for (setting the various priority levels on interruptprocessing 
code so things are processed in the right order wheninterrupts/traps are
received), I wondered how replacing them with asingle tier of critical
sections works?

    For all intents and purposes there is no difference in performance,
    because critical sections are not typically held long enough for the
    additional latency to be noticeable.

    SPLs were never designed to deal with SMP systems so they have to 
    go no matter what.  Critical sections only protect against interrupts
    on the cpu entering the critical section, not on the other cpus.  The
    SPL->critical section change is therefore a temporary measure.

    Ultimately interrupts will have to use a serializing lock for interlocks.
    currently the EM driver (/usr/src/sys/dev/netif/em/if_em.c) is the only
    driver doing this (as a test).  On SMP systems the kernel is still
    using the Big Giant Lock for most things, including interrupts, and
    this effectively forces mainline kernel code and interrupt code to
    run on the same cpu, so critical sections have the proper effect
    until we can move to a better model.

Are there some interruptpriorities hard coded into the 
LWKT scheduler on each CPU? Also, just wondering how the above relates to 
converting the interruptprocessing stuff to run in a thread context for 
each driver...doesthat allow interrupts to be scheduled by normal means
(I would guessLWKT) rather than having a special mechanism? 

    The LWKT scheduler is a fixed priority scheduler with 32 priority
    levels.  Three of the higher priority levels are reserved for use
    by interrupt threads.  Generally speaking, the actual hard interrupt
    does nothing more then schedule the interrupt thread.  The interrupt
    can preempt lower priority thread and thus runs almost immediately
    (unless blocked by a critical section).

    The interupt thread abstraction does allow one interrupt thread to
    preempt another interrupt thread if it is at a higher priority, but
    at the moment nearly all of our interrupt threads run at the same
    priority.  Most modern I/O devices are DMA driven so interrupt latency
    isn't as important anymore.  The one exception is the serial port,
    but even there the serial FIFOs are large enough to handle quite a bit
    of latency without overflowing.

Thank you for
putting up with these sorts of questions, I am trying toget my head around
operating systems and some of the approaches inDragonFlyBSD are really novel 
so I cant find much information aboutthem. Thanks in advance, Alex Burke.

					-Matt
					Matthew Dillon 
					<dillon@xxxxxxxxxxxxx>