|From:||"Simon 'corecode' Schubert" <corecode@xxxxxxxxxxxx>|
|Date:||Thu, 6 Oct 2005 02:43:49 +0200|
At the moment we still use FreeBSD-4's libc_r for threading which is POSIX compliant (I think) but implemented as 1:N, i.e. all threads of one process run in the same process *with the same kernel thread*. Userland does all scheduling of threads itself. This also means that we can't use MP for threaded programs: All threads run on the same CPU.
The plan for DragonFly is to switch to a 1:1 threading model. Libthread_xu already can do this, but it lacks kernel support mainly for signals. I've taken this oportunity to restructure the kernel to make it directly aware of threading. This will work in several steps as outlined below.
The idea is the following. We want one kernel thread (scheduling entity) per userland thread (1:1). While this can be done with pure procs alone, it quickly gets messy, because all userland threads belonging to one process share lots of state. We keep struct proc as the entity which is viewed as the "process" by the user: it has a pid, a virtual memory space, filedescriptors, a text-vnode (the binary) and so on. The new thing is that we move out all stuff which is in fact per-(userland)- thread: execution and scheduling control (lwkt), signals pending/blocked, statistics. Now we start (userland-) scheduling lwps instead of procs and thus we can run multiple threads of one process on different CPUs.
Furthermore this will help us doing MP-locking the data for processes: All thread-local data is in struct lwp and owned by the executing cpu. All (shared) process data is in struct proc and needs some sort of interlocking. But that's future, at the moment we still run under BGL.
I hope I described the concept well enough. Now on to the roadmap of implementation (mostly layed out by Matt Dillon):
1. Split thread-local members of struct proc out into struct lwp. Embed one struct lwp in struct proc and set up preprocessor linkage.
This doesn't change semantics for the rest of the kernel: There is one lwp per proc and all fields of that one lwp can be accessed via macros that indirect to the lwp: p_sigmask is p_lwp.lwp_sigmask.
2. Change functions which take a struct proc as parameter but instead should take a struct lwp, because they actually deal with the thread state and not the process (this will be the vast majority) to actually take a struct lwp. This will have to be done in chains, i.e. multiple functions could need to be changed at once. Patches still need to be small so that they can be easily reviewed and bugs can be spotted. Overall this is mostly mechanical. Also uses of td_proc which really mean to access thread-local data need to be changed to use td_lwp.
We still will have just one lwp embedded in struct proc, but we will try and avoid accessing it via curproc->p_lwp, but rather via curthread->td_lwp (maybe we want to introduce a curlwp macro).
This will change the function signatures but not the location of the data they are actually accessing!
This will be split in many small sub-steps. Every helping hand is appreciated! I will coordinate efforts.
3. Removal of the p_* compatibility macros and use of the embedded p_lwp. At this point all kernel accesses thread-local data via a passed struct lwp / thread->td_lwp and process data via a passed struct proc / thread->td_proc. Thus we have made thread-local data access explicit.
We still have just one lwp per proc, but the kernel isn't aware of this fact anymore.
4. At this stage we will allocate lwps directly and will remove the embedded p_lwp. We will implement neccessary kernel logic for multiple lwps per process. This means getting forking, exiting and signals working correctly with processes which have multiple threads. Once this is done we have a working 1:1 threading. This work will require some thinking but actually not much code.
(5.) Getting locking right before we can remove BGL (farer away).
If you have any questions, feel free to ask. And as I said before, feedback and helping hands are appreciated (and expected!).
-- Serve - BSD +++ RENT this banner advert +++ ASCII Ribbon /"\ Work - Mac +++ space for low $$$ NOW!1 +++ Campaign \ / Party Enjoy Relax | http://dragonflybsd.org Against HTML \ Dude 2c 2 the max ! http://golden-apple.biz Mail + News / \
Description: This is a digitally signed message part