Re: Futures - HAMMER comparison testing?

[Bill Hacker <wbh@xxxxxxxxxxxxx>]

Matthew Dillon wrote:
> :Sorry to hijack this thread.

Not to worry!

It was *intended* to be 'hijacked'.  Welcome!

> Just wanna mention a little write down of 
> :mine about HammerFS features (and sometimes comparing it with ZFS):
> :
> :http://www.ntecs.de/blog/articles/2008/01/17/zfs-vs-hammerfs
> :
> :I can't await to try it out in real!
> :
> :Regards,
> :
> :   Michael
>
>     Nice.  There are a few factual mistakes but nothing major.  ZFS is
>     reliable on a 386 system, you just have to limit its memory consumption
>     with (I think) a sysctl.  HAMMER's resource use is still far lower,
>     though.  There are still reports of total corruption for ZFS on FreeBSD
>     but the issue doesn't sound insurmountable.

Not just on FreeBSD. There are some 'bad news' ZFS stories on the 
Solaris-on-UltraSPARC lists as well.

But - so far- more seem to be 'tuning' issues than structural.

>     HAMMER doesn't journal.  Theoretically by carefully ordering
>     certain I/O operations it will not have to journal.

As with 'softupdates' the net result - reduced risk of damage -  is more 
important than the technique used to deliver it.

>     More importantly,
>     HAMMER can 'recover', as in regenerate, the contents of a cluster on
>     the fly by scanning the records in that cluster and then rebuilding
>     the B-Tree and allocation radix trees from scratch.  As long as
>     recognizable records are present, it should be possible to recover a
>     considerable amount of information even after the disk gets scratched.

Again - so long as it can be trusted to actually *deliver the goods*, 
the method is 'don't care' to the end-user.

>     The historical nature and backup sections are correct, minus your
>     journaling comment.  Basically you make backups by working from an
>     as-of access.   You don't even have to make an as-of mount, there's
>     an @@timestamp extension that allows you to access the filesystem
>     as-of any time.  So, e.g. you can do things like:
>     'diff /mnt /mnt/@@0x<timestamp>'.

From 'decision maker land' - the hottest of hot buttons are not speed 
or bragging rights but rather:

- The fs will not fail, embarrass, or cost lost time or money.

- It will need less time install, configure, and maintain than other 
options (in which is included backup/restoral costs).

Needing less effort to muck about with tarballs and rsync offsets a 
(possibly) slower fs. Bigtime.

Not so lost or corrupted or even just wrongly erased data.

>     I am down to three major items for the release:  The Recovery, balancing,
>     and vacuuming subsystems.  All are interrelated and I am making good
>     progress.  Beyond that the spike code needs some major tweaking but
>     the only effect of that is poor write performance (probably through
>     the alpha release).

30% of the I/O speed of current alternatives is fine with me. 25% or 
less might be problematic.

50% is even OK for the long-term if the backup/restoral/rollback pays 
off. Those things are otherwise not 'free'.

>     Of course, there are many other little issues
>     that need to be dealt with before the release as well.
>
>     Post release I'll have a go at implementing backup/mirroring streaming.
>     I have a pretty good idea how to implement it -- basically by storing a
>     last-transaction-id in cluster headers, super-cluster headers, and
>     volume headers, in order to reduce the amount of initial scanning
>     required to resynchronize a stream.
>
> 					-Matt
> 					Matthew Dillon 
> 					<dillon@backplane.com>

That last part sounds like 'journaling' to me.

But - at the end of the day - how much [extra?] on-disk space will be 
needed to insure mount 'as-of' is 'good enough' for some realisitic span 
(a week?, a month?)? 'Forever' may be too much to ask.

How close are we to being able to start predicting that storage-space 
efficiency relative to ${some_other_fs}?

Bill