DragonFly kernel List (threaded) for 2008-06
HAMMER Update - 28 June 2008
Here's another update on the state of HAMMER for the release!
Large File Testing
I've added a large-file test to my testing rigimen. In addition to
running blogbench, buildworld -j 8, and fsx all at once I am now also
doing an unbounded "dd" to a test file until the disk fills up. The
file grows to approximately the size of the filesystem, 1.9TB.
I then 'rm' the file in full history mode, and later run hammer prune.
This sequence located a number of issues with the buffer cache, in
particular HAMMER was trying to delete all 28 million records in a
single flush cycle, which deadlocked the buffer cache.
In addition to that, retries due to deadlocks caused the delete scan
to start from offset 0, instead of picking up where it left off,
causing the delete operation to take several hours instead of
I am still testing but I believe I have fixed most of the issues.
FUN ASIDE: With my little 3-ware raid stripe HAMMER can maintain
full platter speed on writes on the dd when nothing else is running,
and still does a good job when all the other tests are running at
the same time. This is about 109MB/sec. It takes 5 hours for
the dd to actually fill up the 1.9TB test filesystem. And then
I get to 'rm' it and do a 'hammer prune' pass.
I have three mirroring-related items left to do for the release.
The first two are safety-oriented features, basically to disallow
user modification of the mirror-write target, and to allow a named
label to be associated with each mirroring target and require that
it matches the mirroring source. The safeties are needed to prevent
users from shooting themselves in the foot too easily as doing a
mirror-write to the wrong FS has a very good chance of destroying it.
The last item is to finish some B-Tree updates that have to occur
on the mirror master to make incremental and streaming mirroring work
Buffer cache work
HAMMER uses mixed block sizes and under heavy load testing this revealed
a couple of issues with the kernel's buffer cache which should now be
fixed. I may be doing a little more work in this area, though.