Binary Updates for DragonFly

[Matthias Schmidt <schmidtm@xxxxxxxxxxxxxxxxxxxxxxxxx>]

Hi guys,

I'm currently working on binary updates for DragonFly.  The main  
purpose is for security and other little fixes.  Binary updates are  
very handy if you have a lot of remote machines and can just type  
something like fetch_updates && install_updates and you get the latest  
fixes without fetching/compiling the source.  I know, this approach  
differs from the traditional Unix philosophy to distribute only source  
code diffs for fixes, but I think its really handy :)

I wrote a little piece of software which basically solves this task.

- A client for the user.  The client checks if updates are available,  
fetches and installs them.

- A tool for the server distributing the updates.  The tool compares  
two DragonFly installations and generates binary or source diffs and  
an index file containing informations about all modified files.

bspatch/bsdiff

To use the client and the server tool you have to install Colin  
Percivals bsdiff/bspatch tools at first.  I have a version ready for  
DragonFly here:

  # fetch http://www.mathematik.uni-marburg.de/~schmidtm/update-dragonfly/bsdiff_n_bspatch.tar.gz
  # tar xfz bsdiff_n_bspatch.tar.gz && cd bsdiff_n_bspatch
  # make && make install

Example

This example imitates a (security) fix: The original "/bin/ls"  
contains a bug (which is not true of course), the modified one fixes  
that bug.  I know its a constructed case but its sufficient for the  
demonstration.  If you use a DragonFly 1.10.1-RELEASE and your /bin/ls  
matches the following SHA1 sum, you can test the client.  If not you  
can find an appropriate ls in the above mentioned URL.

SHA1 (/bin/ls) = a15be4de58a78481eef558ab09614522dd5a5764

  # fetch http://www.mathematik.uni-marburg.de/~schmidtm/update-dragonfly/update-dragonfly.tar.gz
  # tar xfz update-dragonfly.tar.gz
  # cd update-dragonfly
  # sh update-dragonfly.sh -h
  usage: ./update-dragonfly.sh [-ghinv] [-f config]
                -g : Get available updates
                -h : Print this help
                -i : Install previous fetched updates
                -n : Do not actually install updates.  Just report all
                     install steps taken
                -v : Be more verbose

                -f config: Use this config file

The first step is to get all updates from the server (which is one of  
my university machines ;) ):

  # sh update-dragonfly.sh -f update-dragonfly.conf -v -g
  Fetch public key from http://globus.mathematik.uni-marburg.de: 
2323 ... done.
  Check for 1.10.1 updates
  New updates available.
  Get all available updates...
  Save permissions
  Save file status
  Patch /bin/ls
  Patching /bin/ls successful

  Use ./update-dragonfly.sh -i to install the following files:
  /bin/ls

Second step:  Install the updates.  Be sure to make a backup of /bin/ 
ls :)

  # sh update-dragonfly.sh -f update-dragonfly.conf -v -i
  Installing updates...
  Backup /bin/ls
  All updates installed

I modified "ls" to display the message "-i set" if the option -i is  
used.  If you can see it, the update was successful.

  # /bin/ls -i update-dragonfly.sh
  -i set
  2347 update-dragonfly.sh


Design and Implementation

The server tool (gen_update.sh) basically compares two DragonFly  
"installations".  The first one is the original one, the second the  
modified one.  I populated two directories with "make installworld"  
and changed the "ls" binary in the modified one.

  # ./gen_update.sh /usr/update orig mod		<- takes some time
  NOTE: /bin/ls and /bin/ls differ.  Create patch
  # ls -l /usr/update
  total 4
  drwxr-xr-x  14 root  wheel  -  512B Dec 16 11:38 mod/
  drwxr-xr-x  14 root  wheel  -  1.0K Dec 16 11:38 orig/

/usr/update contains "orig" (original installation) and  
"mod" (modified installation).  The tool compares all entries based on  
checksums and generates a patch (either binary or text) if two files  
differ.  The patch is copied to a well-known location and registered  
in an INDEX file.  The checksum of the patch, the modified and  
original file is recorded as well.  I experienced one problem during a  
run:  /etc/named/etc/named is linked to ".." so the tool got stuck in  
an endless loop.  Removing the link temporary fixed that problem :)

The client checks if the server is available and fetches the INDEX  
file as well as the INDEX checksum.  If the checksums matches, all  
listed patch files will be fetched.  The original files will be  
patched, validated and stored for later installation.  If the user  
decides to install the patched files, the client will copy them to  
their locations.

TODO

There are some areas needing improvement.  What to do about a self- 
build kernel/world with custom compiler flags etc where the checksums  
don't match?  freebsd-update (in its early days?!?) only supported the  
GENERIC kernel and worked only with a binary distributed userland.  I  
think most server admins also stick with the GENERIC kernel (at least  
I do it), so support for GENERIC would IMO be enough.

Regards,

	Matthias


PS:  The tool is undergoing work and needs testing so don't blame me  
if it eats some of your files :P

--
Dipl.-Inf. Matthias Schmidt <schmidtm@mathematik.uni-marburg.de>
Dept. of Mathematics and Computer Science, Distributed Systems Group
University of Marburg, Hans-Meerwein-Strasse, 35032 Marburg, Germany
Tel: +49-6421-2821591, Fax: +49-6421-2821573, Office C4347