Hashing speedup and unit tests
Summary
This release is the biggest release yet in the path to production usefulness outside of a few large objects. brain_plasma.Brain is mostly rewritten and entirely refactored; it now hashes names for direct access to speed up read and write operations by several orders of magnitude due to fewer and more lightweight calls. The API is mostly the same. Custom exceptions are added to help users catch and understand errors better. Most functions are unit tested and can be checked with pytest.
The sum of these changes means brain-plasma can be used as a fast production backend similar to Redis, but with fast support for very large values as well as for very small values, and for few as well as many values. I'm pretty excited about it.
Hashing speedup
Speedup results are drastic, especially when there are more than a dozen or so names in the store. This is because the old Brain called client.list() multiple times for a most Brain interactions. This was admittedly a horrible design that should never have existed. The new Brain doesn't call client.list() at all for most operations including all reads and writes. The script many_vals.py compares the old with the new Brains:
plasma_store -m 10000000 -s /tmp/plasma
# new terminal
python many_vals.py
>>>
100 items:
learn:
old: 3.6606647968292236
hash: 0.030955076217651367
recall:
old: 4.092543840408325
hash: 0.017110824584960938
10 items:
learn:
old: 0.32016992568969727
hash: 0.005012035369873047
recall:
old: 0.31406521797180176
hash: 0.002324819564819336Unit tests
Most functions are tested in tests/. Check yourself or test your changes with:
pip install pytest
pytestExceptions
Custom exceptions are added to help users catch and understand errors better. Most types of errors that are unique to the functions rather than to Python errors are defined as custom exceptions. Function docstrings mention which exceptions which may be caught. New exceptions are imported en masse like:
from brain_plasma.exceptions import (
BrainNameNotExistError,
BrainNamespaceNameError,
BrainNamespaceNotExistError,
BrainNamespaceRemoveDefaultError,
BrainNameLengthError,
BrainNameTypeError,
BrainClientDisconnectedError,
BrainRemoveOldNameValueError,
BrainLearnNameError,
BrainUpdateNameError,
)Other
Code is formatted with the excellent black. Markdown is formatted with Prettier.