Commits on Aug 14, 2019

1. sql: implement memory management system for caches …

   This PR implements a memory management system, intended to have
   control over the allocated memory for caches so that they can be
   freed at any moment and we can avoid out of memory errors.
   
   The main changes are the following:
   
   - MemoryManager in the sql package, which is just the component that
     tracks all caches. Memory of all freeable caches can be freed
     using the Free method of this component. The only way to instantiate
     new caches is using the NewXXXCache methods.
   - Rows, history and LRU cache implementations, accessed using the
     NewXXXCache methods of MemoryManager.
   - Reporters, which is a component that reports the maximum amount
     of memory the program is allowed to use and the currently used memory.
     This interface is meant for making testing easier. There is a default
     ProcessMemory reporter that returns the memory used by the process and
     the maximum memory defined in the `MAX_MEMORY` environment variable.
   - MemoryManager is passed down to every component through *sql.Context,
     which meant a little more boilerplate on the server SessionBuilder.
   - GroupBy, Sort, Distinct and Join now use the provided APIs of memory
     and cache management for their in-memory computations.
   
   Caveats:
   - We need to think of a good default so that memory usage won't grow
     forever and crash eventually, which is the behaviour when MAX_MEMORY is 0.
   
   Signed-off-by: Miguel Molina <miguel@erizocosmi.co>

   

   erizocosmico committed Aug 14, 2019
   Configuration menu

   • View commit details
   • Copy full SHA for 4bc20fc
   • Browse repository at this point

   Copy the full SHA

   4bc20fc View commit details

   Browse the repository at this point in the history