Re: Hammer or ZFS based backup, encryption

[Bill Hacker <wbh@xxxxxxxxxxxxx>]

Csaba Henk wrote:
> Hi,
>
> I need to setup a backup machine, and I intend to utilize today's
> snapshotty filesystems (which boils down to Dfly+Hammer or FBSD+ZFS --
> btrfs is not there yet, and I don't feel like dwelving into Solaris).
> Set up such an OS with such an fs, and backup by syncing to the
> snapshotty fs and create a snapshot.
>
> I wonder about the following things:
>
> 1) Any idea how does this approach scale related to more conventional solutions,
> like rdiff-backup or dump(8)? I see the the pros, but are there any
> cons? How effective is taking regular snapshots space-wise?

The advantage to snapshot-as-you-go is that it 'mostly' has just the 
rate-of-change to deal with - not the scanning or (external) DB'ifying 
of the whole mass of data. Easier to be near current if you don't have 
to keep digging through the big chunks to cope with a few bytes of change.

HAMMER has the advantage there in that it's basic structure is TID 
retentive. Downside is that reblock/prune are out-of-band operations.

ZFS, AFAICS, wants a great deal of RAM to work effectively with very 
large file systems. HAMMER seems *way* less greedy in that regard - 
OTOH, HAMMER really needs large *disks*.

> 2) Is there any practical argument for choosing between Dfly+Hammer and
> FBSD+ZFS? (Feel free to give biased answers :) )

Biased?  51 years into this game, I have no other kind....

On AMD-64, UtraSPARC, or Itanium, you can handle really large memory. On 
Intel lets-pretend-we-are-64-bit it is more challenging. Not that it is 
easy to find MB that support both anyway...

So ... IF I was to run ZFS, I'd probably bite the bullet and learn to 
put up with Solaris-on-SPARC, AND NOT Solaris on-Intel.

AFAICS, the FreeBSD port of ZFS has gotten quite good. But Solaris is 
where it ZFS was grown, and where it has the best 'fit' and integration, 
and the shortest list of out-of-step-with the rest-of-the-environment 
items. Acl's for example.

> 3) I'd like to encrypt stuff, either at device or fs level. For
> FreeBSD there is geli(8). I haven't found anything for DragonFly.
> Is there any way to get at it on DragonFly?
>
> Thanks,
> Csaba

Any fan-in/fan-out fs environment should be able to interpose an 
encryption layer between VFS and media by use of a loopback or nullfs 
method.

'Should', 'in the ports', and 'proven not to break' are not synonymous.

Soon having to make a similar choice, (and still-yet looking at Gfarm, 
Gluster, DFarm, Ceph, Chiron et al..)

- ZFS doesn't offer anything I actually need - or at least, not at the 
'price' it entails.

 - HAMMER does. And resource-cheaply. Large HDD are way cheaper than 
large RAM once the switch to a powerful 64-bit CPU and associated MB to 
hold and use it effectively is factored in.

HAMMER, OTOH, seems quite happy with a VIA C7, and very modest RAM.

- I expect to 'settle' on DragonFlyBSD for the next set of production 
'twins', accept (for now) roughly half the overall throughput FreeBSD 
could deliver, as we are network bandwidth and UPS-budget constrained 
anyway - not CPU or I/O bound.

But a DFLY choice is not *just* to have the hammerfs - but also because 
DFLY has more modern disk slicing and partitioning capability.

YMMV,

Bill