DragonFly users List (threaded) for 2008-01
DragonFly BSD
DragonFly users List (threaded) for 2008-01
[Date Prev][Date Next]  [Thread Prev][Thread Next]  [Date Index][Thread Index]

Re: Futures - HAMMER comparison testing?

From: Bill Hacker <wbh@xxxxxxxxxxxxx>
Date: Fri, 18 Jan 2008 01:59:52 +0000

Matthew Dillon wrote:
:Sorry to hijack this thread.

Not to worry!

It was *intended* to be 'hijacked'. Welcome!

Just wanna mention a little write down of :mine about HammerFS features (and sometimes comparing it with ZFS):
:I can't await to try it out in real!
: Michael

    Nice.  There are a few factual mistakes but nothing major.  ZFS is
    reliable on a 386 system, you just have to limit its memory consumption
    with (I think) a sysctl.  HAMMER's resource use is still far lower,
    though.  There are still reports of total corruption for ZFS on FreeBSD
    but the issue doesn't sound insurmountable.

Not just on FreeBSD. There are some 'bad news' ZFS stories on the Solaris-on-UltraSPARC lists as well.

But - so far- more seem to be 'tuning' issues than structural.

    HAMMER doesn't journal.  Theoretically by carefully ordering
    certain I/O operations it will not have to journal.

As with 'softupdates' the net result - reduced risk of damage - is more important than the technique used to deliver it.

    More importantly,
    HAMMER can 'recover', as in regenerate, the contents of a cluster on
    the fly by scanning the records in that cluster and then rebuilding
    the B-Tree and allocation radix trees from scratch.  As long as
    recognizable records are present, it should be possible to recover a
    considerable amount of information even after the disk gets scratched.

Again - so long as it can be trusted to actually *deliver the goods*, the method is 'don't care' to the end-user.

    The historical nature and backup sections are correct, minus your
    journaling comment.  Basically you make backups by working from an
    as-of access.   You don't even have to make an as-of mount, there's
    an @@timestamp extension that allows you to access the filesystem
    as-of any time.  So, e.g. you can do things like:
    'diff /mnt /mnt/@@0x<timestamp>'.

From 'decision maker land' - the hottest of hot buttons are not speed or bragging rights but rather:

- The fs will not fail, embarrass, or cost lost time or money.

- It will need less time install, configure, and maintain than other options (in which is included backup/restoral costs).

Needing less effort to muck about with tarballs and rsync offsets a (possibly) slower fs. Bigtime.

Not so lost or corrupted or even just wrongly erased data.

    I am down to three major items for the release:  The Recovery, balancing,
    and vacuuming subsystems.  All are interrelated and I am making good
    progress.  Beyond that the spike code needs some major tweaking but
    the only effect of that is poor write performance (probably through
    the alpha release).

30% of the I/O speed of current alternatives is fine with me. 25% or less might be problematic.

50% is even OK for the long-term if the backup/restoral/rollback pays off. Those things are otherwise not 'free'.

> Of course, there are many other little issues
that need to be dealt with before the release as well.

    Post release I'll have a go at implementing backup/mirroring streaming.
    I have a pretty good idea how to implement it -- basically by storing a
    last-transaction-id in cluster headers, super-cluster headers, and
    volume headers, in order to reduce the amount of initial scanning
    required to resynchronize a stream.

Matthew Dillon <dillon@backplane.com>

That last part sounds like 'journaling' to me.

But - at the end of the day - how much [extra?] on-disk space will be needed to insure mount 'as-of' is 'good enough' for some realisitic span (a week?, a month?)? 'Forever' may be too much to ask.

How close are we to being able to start predicting that storage-space efficiency relative to ${some_other_fs}?


[Date Prev][Date Next]  [Thread Prev][Thread Next]  [Date Index][Thread Index]