PagedMemory: Use dicts to store items when there are only a few.

[ltfish]

TODO:

• Fix the type annotation for DynamicDictList to support DynamicDictList[element_type].
• Write a performance test case to determine the optimal threshold.

[github-actions]

Unit Test Results

     81 files  ±0       81 suites  ±0   1h 26m 46s ⏱️ + 17m 13s
1 089 tests ±0  1 059 ✔️ ±0  30 💤 ±0  0 ❌ ±0 
1 095 runs  ±0  1 065 ✔️ ±0  30 💤 ±0  0 ❌ ±0 

Results for commit 5cd4514. ± Comparison against base commit af6f718.

♻️ This comment has been updated with latest results.

[ltfish]

Apparently this is never merged (or finished). I'll get to it today.

[ltfish]

My profiling script:

import matplotlib.pyplot as plt
import timeit


NUMBER = 10000


def profile_list_dict(elements: int):
    t0 = timeit.Timer("dict(d)", setup=f"""
class A:
    pass

d = dict((v, A()) for v in range({elements}))""")

    t1 = timeit.Timer("list(l)", setup=f"""
class A:
    pass

l = [None] * 4096
for i in range({elements}):
    l[i] = A()""")

    d = t0.timeit(number=NUMBER)
    l = t1.timeit(number=NUMBER)
    print("#:", elements)
    print("dict:", d)
    print("list:", l)
    return d, l


def main():
    d_ = [ ]
    l_ = [ ]
    tt = [ ]
    for elements in range(0, 4096, 50):
        d, l = profile_list_dict(elements)
        tt.append(elements)
        d_.append(d)
        l_.append(l)

    # plot it
    fig, ax = plt.subplots()
    ax.plot(tt, d_, label="dict")
    ax.plot(tt, l_, label="list")
    ax.legend()

    plt.savefig("result.png")
    plt.show()


if __name__ == "__main__":
    main()

CPython 3.10:

PyPy 3.9:

Looks like we need two different thresholds :/