This repository has been archived by the owner on Jan 28, 2021. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 110
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
sql: implement memory management system for caches
This PR implements a memory management system, intended to have control over the allocated memory for caches so that they can be freed at any moment and we can avoid out of memory errors. The main changes are the following: - MemoryManager in the sql package, which is just the component that tracks all caches. Memory of all freeable caches can be freed using the Free method of this component. The only way to instantiate new caches is using the NewXXXCache methods. - Rows, history and LRU cache implementations, accessed using the NewXXXCache methods of MemoryManager. - Reporters, which is a component that reports the maximum amount of memory the program is allowed to use and the currently used memory. This interface is meant for making testing easier. There is a default ProcessMemory reporter that returns the memory used by the process and the maximum memory defined in the `MAX_MEMORY` environment variable. - MemoryManager is passed down to every component through *sql.Context, which meant a little more boilerplate on the server SessionBuilder. - GroupBy, Sort, Distinct and Join now use the provided APIs of memory and cache management for their in-memory computations. Caveats: - We need to think of a good default so that memory usage won't grow forever and crash eventually, which is the behaviour when MAX_MEMORY is 0. Signed-off-by: Miguel Molina <miguel@erizocosmi.co>
- Loading branch information
1 parent
875590d
commit 65596fc
Showing
69 changed files
with
1,061 additions
and
434 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.