HAMMER update 23-jan-08

[Matthew Dillon <dillon@xxxxxxxxxxxxxxxxxxxx>]

    I continue to make excellent progress.  There are just a handful
    of outright bugs left, though I'm sure I will introduce new ones
    as I work on the balancer.

    * The recovery code is written and working but both the spike and
      recovery code need additional I/O ordering dependancies to operate
      safely.

      For example, the spike code copies a number of records but may sync
      the original cluster (where they've been deleted) before syncing the
      new cluster (where they were copied to).

      It is also possible for a file's link count to get out of sync after
      a crash, for example if you crash in the middle of a rm -rf.  The 
      worst that happens is we wind up with a directory entry pointing to a
      non-existant inode, or an inode whos directory entry no longer exists.
      Since inode numbers in HAMMER are forever-unique it can't actually
      hurt the filesystem but there will have to be a validator that will
      run through and fix those, or a transactional solution.

      There are still a few transactional issues but I'm not going to worry
      about them for the alpha release.  I have to get the I/O ordering
      worked out before I can even begin thinking about recovering
      transactions which span more then one cluster.

    * I finally put a simple heuristic in the spike code so B-Tree depth
      no longer spins out of control, though there are still some edge
      cases that aren't getting deleted (obvious if you cpdup, sync, rm -rf,
      sync, and use 'hammer ... show' to dump the tree).  The current
      heuristic doesn't pack files very well.

    * There are still a few fatal bugs, probably related to I/O ordering.

    Next up is the balancing code.  This code will be responsible for
    slowly balancing the topology, removing historical bits based on the
    retention policy, and straightening out the kinks.  Ultimately no matter
    how bad an initial layout, the balancer will be able to fix it.

					    -Matt