DragonFly kernel List (threaded) for 2003-07
Re: userapi: signals
:> never makes sense to have more then NCPU kernel threads dedicated
:> to a user program, regardless of how many userland threads that
:> user program is running.
:OK, but that's what it's like in the kernel as well, isn't it? Kernel
:threads are a lightweight per-cpu concept, really, with synchronization
:between them being deferred to the more expensive IPI token passing...
You mean in regards to multi-threading things like VFS devices? Right,
one shouldn't need more then NCPU threads per (high performance) device
to maximize performance.
I'm not sure how much kernel threading applies to user threading. Kernel
threads are more sensitive to memory issues since there is only one
KVM address space and very little of it is pageable.
:Maybe I'm confused or I'm not presenting myself well. Let me see if I
:can get this straight. We can draw some parallels...
: 1a. Each cpu in the system has its own self-contained LWKT scheduler.
: 1b. Each process in user mode has its own self-contained userland
Right. In *user mode* each process will be responsible for managing
its own threads. But a process *CAN* rfork() to take advantage of
other cpus if it wishes.
: 2a. A Thread will never be moved to another cpu while it is running,
:and a thread will never be preemptive switched to a non-interrupt thread.
: 2b. A userland thread will never be moved to another CPU, and will
:never be preempted by a non-signal thread.
No. In userland anything can happen. The system is free to interrupt
a userland program and migrate it to another cpu at any time.
As an abstraction it should not matter. The userland thread manager's
concept of a 'cpu' is rfork(). It is up to the userland thread scheduler
to determine which process to schedule a userland thread on and it doesn't
much matter *what* the kernel does with those processes. There is nothing
preventing you from rfork()ing four userland processes on a single cpu
system, and nothing preventing you from rfork()ing two userland processes
on a quad MP system.
: 3a. A Thread can access non-global information without having to obtain
A thread can access the per-cpu globaldata area without obtaining any
locks, but may have to enter a critical section if the information being
accessed can also be modified by an interrupt.
A thread can access any data in its thread structure or its process
structure without getting any locks, but it may have to enter a critical
section to protect against IPIs from other cpus manipulating data in
critical sections are cheap... basically just a counter.
: 3b. A userland thread can access non-global information without having
:to make any system calls...
The rfork()'d 'process' that a userland thread is running on is under the
complete control of the userland thread scheduler (or will be when this
baby is written!). This means that you can do whatever you want... you
can create a per-rfork memory area and access it like a kernel thread
would access its own per-cpu area, without locking.
: 4a. A cpu which attempts to schedule a thread belonging to another cpu
:will issue an IPI-based message to the target cpu to execute the operation.
Correct. Or anything belonging to another cpu. For example, if I had
a per-cpu cache and I wanted to free a piece of data to another cpu's
per-cpu cache I would send an asynch IPI to the other cpu with the data.
IPIs have high latency but low cpu wastage (if you send them asynch),
and they become more efficient as you get more heavily loaded down because
the message queue can build up. We could also implement a passive-IPI
that doesn't bother actually sending the IPI to the target cpu and
handle low-priority requests at task-switch time or something like that.
(I don't do that yet but I could). i.e. IPI messaging is potentially
: 4b. A userland thread which attempts to schedule a thread belonging to
:another CPU will make a system call (or send a message through a
:non-system kernel message port).
To another abstracted cpu (rfork). Yes, though perhaps all it really
needs to do is write() to a communications pipe that goes to the target
(abstracted) cpu. Thus this sort of operation would become more
efficient under heavier loads.
:None of this means the kernel needs to know about userland threads, it
:just means that the userland thread code is acting like the kernel
:thread code. So the userland process gets a signal and it schedules the
:signal thread... my question is more along the lines of:
Right. In fact, we could probably compile most of the LWKT subsystem
separately as part of the userland scheduler. The LWKT subsystem is
almost completely self contained. The messaging system *IS*
completely self contained.
:Can a process request a signal to fire when a message arrives on one of
:its kernel message ports, in which case it would be able to support
:preemptive signals, or would it have to wait for the signal to arrive on
:its reply port, in which case signal handlers (or the signal thread)
:would have to be scheduled synchronously when a Wait() occurs.
Probably. There are lots of ways to do this. Through brainstorming
and discussion we can figure out the most efficient mechanism to use.