Commits on May 29, 2018

1. 9486 reduce memory used by device removal on fragmented pools …

   Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
   Reviewed by: Tim Chase <tim@chase2k.com>
   
   Device removal allocates a new location for each allocated segment on
   the disk that's being removed.  Each allocation results in one entry in
   the mapping table, which maps from old location + length to new
   location.  When a fragmented disk is removed, this can result in a large
   number of mapping entries, and thus a large amount of memory consumed by
   the mapping table.  In the worst real-world cases, we've seen around 1GB
   of RAM per 1TB of storage removed.
   
   We can improve on this situation by allocating larger segments, which
   span across both allocated and free regions of the device being removed.
   By including free regions in the allocation (and thus mapping), we
   reduce the number of mapping entries.  For example, if we have a 4K
   allocation followed by 1K free and then 4K allocated, we would allocate
   4+1+4 = 9KB, and then move the entire region (including allocated and
   free parts).  In this case we used one mapping where previously we would
   have used two, but often the ratio is much higher (up to 20:1 in
   real-world use).  We then need to mark the regions that were free on the
   removing device as free in the new locations, and also obsolete in the
   mapping entry.
   
   This method preserves the fragmentation of the removing device, rather
   than consolidating its allocated space into a small number of chunks
   where possible.  But it results in drastic reduction of memory used by
   the mapping table - around 20x in the most-fragmented cases.
   
   In the most fragmented real-world cases, this reduces memory used by the
   mapping from ~1GB to ~50MB of RAM per 1TB of storage removed.  Less
   fragmented cases will typically also see around 50-100MB of RAM per 1TB
   of storage.
   
   Closes #627

   

   ahrens authored and Prakash Surya committed May 29, 2018
   Configuration menu

   • View commit details
   • Copy full SHA for d6337dd
   • Browse repository at this point

   Copy the full SHA

   d6337dd View commit details

   Browse the repository at this point in the history