DragonFly kernel List (threaded) for 2004-03
Re: Packaging (Re: Goals for first release (June/USENIX))
My own ideas on an installation framework have not yet solidified, but
the more I think about things like variant symlinks and VFS environments,
the more it seems to me that the actual physical install location should
be in a single, properly named and versioned subdirectory, e.g. like
/opt/<pkgnameandrev>, or /usr/local/<pkgnameandrev>, or something like
:haven't been updated), go through the usual configure/make steps but
:type some variant command of "make install", and have the whole thing
:not only installed exactly as usual, but registered in the package
:database with a proper +CONTENTS file so that it can be uninstalled
:cleanly if desired.
I would like to be able to do this too, which is how I cam up with
the VFS environment idea. The trick is to not burden the developer,
so if we were to do all of the above within a VFS environment then,
in fact, we could also 'make install' within that environment and
then tell the real installer which directories represent the installed
The real installer would pull them out of the environment and put
everything in /opt/<pkgnameandrev>/, so no 'CONTENTS' file would
actually be necessary, and would also create an environment file,
say /opt/<pkgnameandrev>/ENVIRONMENT, that tells the system how to wire
the package so we see it where we expect to see it.
In this example, the VFS environment itself is just a temporary affair
used to during the build and installation phase and would not have to be
used for the run-time phase (at least not for most packages).
The advantage of this is that there is a very clearly defined place to
look for (A) the installed package's raw files, and (B) the information
required to wire or unwire those files into the system (the ENVIRONMENT
Now if you think about this, it leads to all sorts of advantages:
* Human readable debugging and investigation. Sysops know exactly where
to look to understand where a package is installed and sysops know
exactly where t look to understand how a package is wired.
* Utilities. e.g. 'whoowns /usr/local/bin/soffice' would return
/usr/local/bin/soffice varsym for soffice-1.1, soffice-1.2, and
soffice-1.3, set SOFFICE_VERSION to select.
The idea being that you could identify any file in the system this
* Database rebuilding utilities. Everything would be sourced in
/opt/<packagename>/ENVIRONMENT. If our package management
subsystem uses a database, the database would be easily reconstructable
simply by scanning all the ENVIRONMENT files in /opt/*.
* Resynchronization and Validation. It is often the case where a system
is partially copied onto a new HD, or partally destroyed through
various means. This methodology would allow us to 'rebuild' the
installed varsyms from what we see in /opt, as well as get rid of
those that are no longer attached to anything, perhaps by consulting
an off-site database that indexes the ENVIRONMENTs for all
* Conflict management. We would use varsyms as much as possible to
deal with conflicts. Even more to the point, it would be posible to
have a two-level system where for those packages which are severely
manged between released (and e.g. install to totally different places),
the primary varsyms in e.g. /usr/local/etc and a/usr/local/bin, would
simply point to secondary softlinks within /opt/<packagename> itself.
In those hopefully few cases where varsyms are not sufficient resolve
conflicts, or paricularly badly written third party software (nah,
nobody would ever do that! Right!), then the actual run-time environment
could end up being a VFS environment too.
* Transparency. Since everything looks 'normally' installed from the
point of view of system management, we would easily be able to switch
between base-system packages, which are installed directly, and
package-managed packages, which are managed via /opt. This would allow
us to slowly migrate base-system packages to 'real' packages, at least
for the more complex base-system packages.
I just don't know if I can get to this before the first release, at least
not on my own. It's not just a matter of a chicken-and-egg issue, it's
that I think it is important not to leave too many loose ends in the
system. I had already made significant progress on asynch syscalls,
upcalls, IPC, and other subsystems. I need to finish those before I
embark on yet another new subsystems.
But I am not 'reserving' the package management subsystem either. I
think it would be an excellent project for a group of developers to
undertake. In fact, you don't even need to have working VFS environments
to start building it... a chroot plus unionfs mounts would be
plenty sufficient. And if unionfs is still broken, that is certainly
something I can help fix. I would definitely like to have a hand in
developing the ENVIRONMENT specification and in mapping out how the
package manager would work, since I do have my own requirements (e.g.
to accomplish the above) that I feel are necessary to create a compact
but easily managed packaging system. What I don't have time to do right
now is the actual major programming.