Re: OT - was Hammer or ZFS based backup, encryption

[Jeremy Chadwick <jdc@xxxxxxxxxxxx>]

On Sun, Feb 22, 2009 at 01:36:28PM +0100, Michael Neumann wrote:
> Am Sat, 21 Feb 2009 19:17:11 -0800
> schrieb Jeremy Chadwick <jdc@parodius.com>:
> 
> > On Sun, Feb 22, 2009 at 11:59:57AM +1100, Dmitri Nikulin wrote:
> > > On Sun, Feb 22, 2009 at 10:34 AM, Bill Hacker <wbh@conducive.org>
> > > wrote:
> > > > Hopefully more 'good stuff' will be ported out of Solaris before
> > > > it hits the 'too costly vs the alternatives' wall and is orphaned.
> > > 
> > > Btrfs has been merged into mainline Linux now, and although it's
> > > pretty far behind ZFS in completeness at the moment, it represents a
> > > far greater degree of flexibility and power. In a couple of years
> > > when it's stable and user friendly, high-end storage solutions will
> > > move back to Linux, after having given Sun a lot of contracts due
> > > specifically to ZFS.
> > 
> > The fact that btrfs offers grow/shrink capability puts it ahead of ZFS
> > with regards to home users who desire a NAS.  I can't stress this
> > point enough.  ZFS's lack of this capability limits its scope.  As it
> > stands now, if you replace a disk with a larger one, you have to go
> > through this extremely fun process to make use of the new space
> > available:
> > 
> > - Offload all of your data somewhere (read: not "zfs export"); rsync
> >   is usually what people end up using -- if you have multiple ZFS
> >   filesystems, this can take some time
> > - zpool destroy
> > - zpool create
> > - zfs create
> > 
> > And if you add a new disk to the system, it's impossible to add that
> > disk to the existing pool -- you can, of course, create an entirely
> > new zpool which uses that disk, but that has nothing to do with the
> > existing zpool.  So you get to do the above dance.
> 
> Hm, I thought that would work easily with ZFS, and at least in theory I
> think that should work well with ZFS. Or what is wrong with:
> 
>   zpool add tank /dev/ad8s1

This will only work how you expect if you're using a ZFS mirror.  With
RAIDZ, it doesn't work -- you're forced to add the new disk into a new
zpool.  This is one of the shortcomings of ZFS (and it is documented,
but only lightly so).

> Okay "zpool remove" doesn't seem to work as expected, but it should
> work well at least for RAID-1 (which probably no one uses for large
> storage systems ;-). Maybe "zfs replace" works, if you replace an old
> disk, with a larger disk, and split it into two partitions, the one
> equally sized to the old, and the other containing the remainder of the
> space. Then do:
> 
>   zfs replace tank old_device new_device_equally_sized
>   zfs add tank new_device_remainder
> 
> But you probably know more about ZFS than me ;-)

In this case, yes (that I know more about ZFS than you :-) ).  What
you're trying to do there won't work.

The "zfs" command manages filesystems (e.g. pieces under a zpool).  You
cannot do anything with devices (disks) with "zfs".  I think you mean
"zpool", especially since the only "replace" command is "zpool replace".

What you're trying to describe won't work, for the same reason I
described above (with your "zpool add tank ad8s1" command).  You can
split the disk into two pieces if you want, but it's not going to
change the fact that you cannot *grow* a zpool.  You literally have to
destroy it and recreate it for the pool to increase in size.

I've been through this procedure twice in the past year, as I replaced
250GB disks with 500GB, and then 500GB disks with 750GB.  It's a *huge*
pain, and I cannot imagine anyone in an enterprise environment using ZFS
to emulate a filer -- it simply won't work.  For individual servers
(where disks are going to remain the same size unless the box is
formatted, etc.), oh yes, ZFS is absolutely fantastic.

> As for Hammer, I worked on some patches that will allow it to expand a
> Hammer FS while mounted. It's actually very easy to implement (~100
> LoC). And the shrink case should be at least in theory pretty easy to
> implement, thanks to reblocking. So with very little work, we can make
> Hammer grow/shrink natively (maybe it's in the next release). 

That's very cool.  I'm hoping HAMMER takes off, although the popularity
of it would greatly improve if *BSD could be made to work with it.  I
know this has been mentioned in the past (and Matt has responded as to
why it would be difficult -- not impossible, just a lot of work), but
really all of the BSDs could benefit from something great and new.

The one thing we all agree on is that, generally speaking, UFS isn't
cutting it.  :-)

-- 
| Jeremy Chadwick                                jdc at parodius.com |
| Parodius Networking                       http://www.parodius.com/ |
| UNIX Systems Administrator                  Mountain View, CA, USA |
| Making life hard for others since 1977.              PGP: 4BD6C0CB |