Commits on May 5, 2016

1. Fix loading huge series into RAM when points are overwritten …

   In some query scenarios, if there are a lot of points on disk spread
   across many blocks in TSM files and a point is overwritten near the
   begginning of the shard's timerange, the full series could be loaded
   into RAM triggering OOMs and huge allocations.
   
   The issue was that the KeyCursor code that handles overwriting points
   had a simple implementation that just deduped the whole series in this
   case.  This falls over when the series is quite large.
   
   Instead, the KeyCursor has been changed to only decode blocks with
   updated points.  It then keeps track of what section of the blocks
   have been read so they are not re-read when the later points are
   decoded.
   
   Since the points in a block are always sorted, the code was also changed
   to remove the Deduplicate calls since they end up
   reallocating the slice.  Instead, we do a sorted merge and re-use
   the slice as much as we can.

   

   jwilder committed May 5, 2016
   Configuration menu

   • View commit details
   • Copy full SHA for a0ac754
   • Browse repository at this point

   Copy the full SHA

   a0ac754 View commit details

   Browse the repository at this point in the history