Re: [GSoC][RFC] HAMMER compression

[Naohiro Aota <naota@xxxxxxxxx>]

Michael Neumann <mneumann@ntecs.de> writes:

> Am Sonntag, den 03.04.2011, 21:02 +0900 schrieb Naohiro Aota:
>> Hi,
>> 
>> I wrote my GSoC project proposal for "HAMMER compression". Could you
>> take a look at it and provide me some comments or suggestions?
>> 
>> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2011/naota/1001#

Thank you for your comments.

> I would not concentrate too much to the chflags stuff and the user land
> utilities other than the "hammer reblocker" [1]. 

OK, I was confused somehow.

> 1) Imagine there would be compressed blocks stored in a HAMMER
> filesystem. How can they be distinguished from non-compressed blocks and
> how can they transparently be decompressed and returned to the user.
> This is a major milestone! If you would start with that major task,
> write yourself an ioctl syscall that generates for you a compressed file
> to experiment with.

ah, then the implementation steps would be following, right?

- first, implement ioctl to dump hardcoded compressed data to file
-- implement "compressed flag" metadata and setting it
   (where can I find some code to handle block "flag" or "metadata"?)
-- implement the ioctl (maybe like hammer_vop_write)
- implement uncompression
-- checking the flag and uncompression routine

> 2) A "hammer compress" user-level utility that scans the filesystem
> (similar to the reblocker and dedup) for uncompressed blocks. Start with
> compressing each block, then think about "compressing only historical
> blocks" or similar policies).
>
> That's how I would do it. I think setting compression for each
> individual file is not that important as we would have one PFS
> for /usr/src for example (where we want compression) and /usr/obj (where
> we don't want compression) so the hammer reblocker would be the ideal
> choice for that.

OK, I remove the features from my plan. I've revised my proposal.

> Regards,
>
>   Michael 
>
> [1]: For example
> http://gitweb.dragonflybsd.org/dragonfly.git/blob/HEAD:/sbin/hammer/cmd_dedup.c

Regards,
Naohiro