Persistent collections updates (part 6)

[lorentey]

• Implement the full API expected of a dictionary type.
• While we're at it, draft a protocol that lists member requirements for such types.
• Add some benchmarks for large-sized key/values.
• When copying or resizing nodes, don't copy the contents only to have to move them again to implement inserting a new item or making a new collision. Instead, create a unique node with the new item/child already inserted.
• Add underscored entry points to allow determining if internal consistency checking is enabled. Use it to have the benchmarks warn if you're trying to measure performance with consistency checks.
• Slightly clean up test fixture initialization. (Don't redo the preorder sorting every time we need to iterate over the fixtures.)
• Add operations to access bit ranks on UInt16, in case we want to reduce the maximum node size.

Two things I learned:

1. Optimizing node resizing/copying leads to extremely complicated/subtle code, but it measures as (at best) a marginal improvement, even for large Key/Value types. (I'm not sure it'd be worth implementing it for removals, but I sure would like to have a go at it later, anyway.) In hindsight, this is probably expected, as shared insertion performance is likely dominated by uniquing the ancestor nodes (which is a straightforward copying/replacement), rather than whatever happens to the final node. Still, it would be good to confirm this in an Instruments session.

2. Reducing node size to 16 measures worse overall than 32, even for large key/values, and even in the shared case. I would've expected the shared case to improve. I want to understand why this is, but first I want to add a PersistentSet and more test coverage.

Bonus thing I just (re)discovered: for performance, it is crucial for index types in tree-based collections to have an (unowned-unsafe) reference to the final node on the path they represent. However, this means that no collection backed by such an index can implement MutableCollection, as any CoW copying of nodes will potentially break that reference, so all mutations need to invalidate every index.

This means that PersistentDictionary.Values cannot be a MutableCollection, although technically it could come with a mutable subscript. (Also, unlike with the standard dictionary, updateValue(forKey:) and removeValue(forKey:) will need to always invalidate indices, even if they don't change the structure of the tree.)

I'll need to rework index invalidation logic to reflect this. I don't think mutations invalidating indices is a huge deal -- folks who need stable indices are better of using Key anyway. (With the gotcha that there is no stable key(after:)/key(before:) function.)

Checklist

• I've read the Contribution Guidelines
• My contributions are licensed under the Swift license.
• I've followed the coding style of the rest of the project.
• I've added tests covering all new code paths my change adds to the project (if appropriate).
• I've added benchmarks covering new functionality (if appropriate).
• I've verified that my change does not break any existing tests or introduce unexplained benchmark regressions.
• I've updated the documentation if necessary.

[lorentey]

The big win here has to do with the introduction of custom types for PersistentDictionary.Keys & .Values. Everything else is largely unchanged.

Tasks with difference scores larger than 1.01:
  Score   Sum     Improvements Regressions  Name
  2.420   2.420   2.420(#76)   1.000(#0)    PersistentDictionary<Int, Int>.Values sequential iteration (*)
  2.409   2.409   2.409(#76)   1.000(#0)    PersistentDictionary<Int, Int>.Keys sequential iteration (*)
  1.022   1.022   1.029(#56)   0.9937(#20)  PersistentDictionary<Int, Int> updateValue(_:forKey:), existing (*)
  1.021   1.021   1.028(#54)   0.9929(#22)  PersistentDictionary<Int, Int> subscript, remove missing (*)
  1.021   1.021   1.022(#67)   0.9986(#9)   PersistentDictionary<Large, Large> subscript, insert, shared (*)
  1.021   1.021   1.032(#45)   0.9886(#31)  PersistentDictionary<Int, Int> subscript, remove existing, unique (*)
  1.019   0.981   1.008(#20)   0.9731(#56)  PersistentDictionary<Int, Int> subscript, successful lookups
  1.017   0.983   1.006(#19)   0.9775(#57)  PersistentDictionary<Int, Int> defaulted subscript, successful lookups
  1.016   0.983   1.011(#27)   0.9725(#49)  PersistentDictionary<Int, Int> subscript, unsuccessful lookups
  1.015   0.984   1.008(#23)   0.9769(#53)  PersistentDictionary<Int, Int> defaulted subscript, unsuccessful lookups
  1.013   1.013   1.018(#62)   0.9952(#14)  PersistentDictionary<Large, Large> subscript, insert, unique
  1.012   0.988   1.002(#26)   0.9863(#50)  PersistentDictionary<Int, Int> subscript, insert, shared
  1.010   1.010   1.016(#49)   0.9941(#27)  PersistentDictionary<Large, Large> subscript, remove existing, shared
  1.010   1.010   1.021(#49)   0.9896(#27)  PersistentDictionary<Int, Int> random removals (missing keys)
  1.010   1.010   1.016(#46)   0.9940(#30)  PersistentDictionary<Int, Int> subscript, remove existing, shared

(All other results are beyond the usual ±5% cutoff, so they aren't necessarily statistically significant. Not that I'm doing any proper statistics here. The four charts below have scores above ±2%.)

[lorentey]

@swift-ci test

[msteindorfer]

LGTM