DragonFly users List (threaded) for 2007-12
Re: Binary Updates for DragonFly
* Matthew Dillon wrote:
> Here's my opinion on binary updates. I considered two scenarios:
> (1) Distribute the object tree and use the normal make installkernel,
> installworld, and make upgrade mechanisms to update.
> (2) Distribute the binaries and write a custom installkernel and
> installworld mechanism, but keep using 'make upgrade' to deal
> with the final clean up.
> I think option #2 is probably our best bet as for the most part everything
> done by instalkernel or installworld can be packaged together into a
> single tar file that one just extracts, with only some minor renames
> of critical library files to avoid blowing the system up.
I agree with you on that.
> * The binary dist creates a tar file. The name of the tar file includes
> the rev and date... basically a unique naming convention that can be
> used to programmatically glue binary diffs together.
Something like DragonFly-<RELEASE>-<PATCH REV> (eg DragonFly-1.10-0 for the
first tar file. I think it would be good idea to distribute the "first/zero
tar file" together with a release (users updating manually have to
create this file themeselves, if they want to). If we have to update files
we can send out a binary patch relative to that first tar.
> * Binary diffs would be relative to older tar files and use a self
> identifying naming scheme so a script or application can verify
> that the correct binary diffs are available and applicable to the
> correct base file(s).
I can write a script for that task once the naming and update convention
discussion is done.
> * The tar files can be gzipped for distribution (but diffs are based on
> the non-gzipped version). The gzipped tar file would remain on the
> target host as a basis for future binary patches. The installed
> binaries would NOT be used as a basis for future binary patches, only
> the tar file.
Yes, zipping the binary patch would save another amount of bandwith.
> * A streaming patch to apply the binary diffs would be the best thing, so
> the tar file can be left on-disk in gzipped format. I don't know if
> the binary diff mentioned in the thread can do that. e.g. some sort
> of 'gunzip < tar | binary_patch <patchfile> | tar xvpf -' sequence
> to apply a binary patch and extract at the same time. A similar
> sequence could be used to generate the new tar file. Again, the
> binary patch has to be against the NON gzipped tar file or it will
> never work.
A sequence like the one above will work (slightly modified). Generating
binary patches from tar files works flawlessly.
> * Patches would contain the MD5 of the final result for validation
> (ungzipped MD5).
I used SHA1 (or SHA*) regarding to the latest discussion about "MD5
considered harmfull" :) We should send out a checksum for the original
file (the to-be-patched tar file), the binary patch itself and as you
said, for the result tar file.
> * The tar file is installed on the machine. Certain critical files
> (main library files like libc.so) are named differently in the tar
> file. The next-to-last step after application of the tar file
> renames them to the official names, replacing the current system
> library files as a final step.
> e.g. in the tar libc.so would be 'libc.so.new' and then as a
> second-to-last step after extraction libc.so.new would be renamed
> to libc.so.
A unique naming would be also necessary here. Eg rename all .new files
in the second-to-last step.
Dipl.-Inf. Matthias Schmidt <email@example.com>
Dept. of Mathematics and Computer Science, Distributed Systems Group
University of Marburg, Hans-Meerwein-Strasse, 35032 Marburg, Germany
Tel: +49.6421.28 21 591, Fax: +49.6421.28 21 573, Office C4347