git: kernel - Fix 3:00 a.m. crashes (deadlocks) related to HAMMER VM use

[Matthew Dillon <dillon@xxxxxxxxxxxxxxxxxxxxxxx>]

commit 55b50bd522537a7b4e0810aa4cab05ad355d1381
Author: Matthew Dillon <dillon@apollo.backplane.com>
Date:   Mon Jan 16 11:33:33 2012 -0800

    kernel - Fix 3:00 a.m. crashes (deadlocks) related to HAMMER VM use
    
    When memory is low and the pageout daemon needs to write things out we still
    need to have at least some reserve to perform the supporting operations for
    the pageout.  HAMMER is particularly memory intensive and could get into a
    situation where insufficient reserve memory was available, deadlocking the
    system.
    
    With these changes DragonFly should run stable on systems with as little
    as 256M of ram, and possibly a bit lower.
    
    * The getblk/bread/bwrite/etc brelse/bqrelse sequence used to manage buffers
      had several bugs in it that prevented the low memory handling code from
      operating properly.  The b[q]relse() sequence was not properly detecting
      the low memory condition and freeing or caching the underlying VM pages
      (when possible).
    
    * Also change the low memory test used by the buffer cache from 'severe'
      to 'min' in kern/vfs_bio.c.  We may be able to change this back to 'severe'
      at a later date with further testing.  These tests are in brelse(),
      bqrelse(), and vfs_vmio_release().
    
    * Rewrite bio_page_alloc().  It effectively does the same thing that it did
      before but should operate more smoothly.  We also no longer try to recover
      pages from unrelated buffer cache buffers from this function, which could
      lead to deadlocks.  The warning kprintf is now also rate-limited.
    
    * Add a buffer overload test in the hammer dedup ioctl.  A hammer dedup
      could cause a buffer cache deadlock by allowing too many dirty buffers
      to build up.
    
    * Add a VM memory test to the core hammer flusher code that was previously
      only checking for the UNDO meta-data and buffer overload limits.  This
      is now done on a per-record basis and should prevent HAMMER from allocating
      too much memory during a flusher operation when the VM system is already
      too low on memory.
    
    * Add some vm_wait_nominal() calls in critical I/O paths, but make sure we
      do not use these calls in any I/O path used by the HAMMER pageout code.
    
      Probably the most important path is the vm_object_page_clean*() code
      path, effectively called via either msync() or via the 30-60 second
      system sync.
    
    * Properly bawrite() a buffer in hammer_vop_write() when IO_ASYNC is set
      (which is used by the pageout daemon), otherwise the pageout daemon will
      not be able to directly recover memory in low memory situations when
      paging to a HAMMER file mapped SHARED/RW.
    
    Testing-by: tuxillo, lentferj, ftigeot, dillon

Summary of changes:
 sys/kern/vfs_bio.c              |  195 +++++++++++++++++++--------------------
 sys/kern/vfs_subr.c             |    1 +
 sys/vfs/hammer/hammer_dedup.c   |   11 ++
 sys/vfs/hammer/hammer_inode.c   |    5 +-
 sys/vfs/hammer/hammer_reblock.c |    7 +-
 sys/vfs/hammer/hammer_vnops.c   |    7 ++
 sys/vm/vm_object.c              |    1 +
 sys/vm/vm_page.c                |    3 +
 8 files changed, 126 insertions(+), 104 deletions(-)

http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/55b50bd522537a7b4e0810aa4cab05ad355d1381


-- 
DragonFly BSD source repository