Memory-efficient probabilistic counter namely Morris Counter. This module based on the following paper:
- Robert Morris. Counting large numbers of events in small registers. Communications of the ACM, vol. 21, issue 10, pp. 840-842, 1978.
Currently Morris Counter supports Python 3.5 and higher.
Basic idea of Morris Counter is described as follows:
- https://en.wikipedia.org/wiki/Approximate_counting_algorithm
- http://yukinoi.hatenablog.com/entry/2015/11/19/220721 (written in Japanse)
$ pip install morris_counter
While the Morris Counter works builtin modules, using third-party package (numpy and mmh3) leads to improve memory-usage and computation time.
$ pip install numpy mmh3
from morris_counter import MorrisCounter
mc = MorrisCounter(size=1000000, dtype='uint8', radix=2, seed=3282)
mc.count('ZOC')
# => 1
mc.increment('ZOC')
mc.count('ZOC')
# => 2
_ = [mc.increment('ZOC') for _ in range(2000)]
mc.count('ZOC')
# => 2048