Persistent collections updates (part 5) #179

lorentey · 2022-09-18T01:14:06Z

Reverse the ordering of items in node storage, fixing copying colliding items in iteration order
Indices now represent paths into the hash tree instead of the preorder position of an item. This speeds up looking up indices (and the item addressed by an index), but it slows down index(after:) and index(before:).
Add index validation logic.
Add a way to dump trees in preorder walk order
Implement in-place mutations for the key subscript and the defaulted subscript. (This was a pain in the neck to implement.)
Add many more tests.
Fix a logic oversight in the node sizing logic that lead to nodes unnecessarily doubling their size with each new generation of copies.
Use the term "slot" rather than "offset" for positions within internal storage.

Checklist

I've read the Contribution Guidelines
My contributions are licensed under the Swift license.
I've followed the coding style of the rest of the project.
I've added tests covering all new code paths my change adds to the project (if appropriate).
I've added benchmarks covering new functionality (if appropriate).
I've verified that my change does not break any existing tests or introduce unexplained benchmark regressions.
I've updated the documentation if necessary.

If the key implements hashing properly, then this would be a clever way to try another hash seed to differentiate between colliding keys. However, if Key has proper hashing, then collision nodes are expected to be tiny, so it’s better to simply do naive comparisons. If the Key type has broken (i.e., partial) hashing, then loading colliding items into a dictionary would just make == even slower. So we’re better off comparing collidingh items via nested loops — so let’s just unconditionally do that.

These provide a barebones, read-only, non-generic abstraction, with just enough functionality to allow navigating around in the tree structure. We will use these to implement path-based indices.

The Index type now represents a path into the tree, consisting of a series of child offsets from the root node, as well as the item offset within the final node. (This all fits in a couple of words, as the depth of the tree is fixed, and the offset path can always be compressed into a single `UInt` value.) To speed up basic operations, we also cache the final node as an unsafe-unowned reference. Dereferencing this is extremely unsafe, so the Index type carries an unsafe reference to the root node along with a mutation counter, to allow reliable(*) index validation. If index’s stored root is identical to the root of the collection, and they have the same version value, then we can assume that the index is still valid.

The new fixtures provide an easier way to construct sample hash trees with specific structure, to run tests on them. Fixtures are able to determine the expected ordering for each sample tree, which comes super useful for verifying collection conformances.

(Internally) standardize on using the word “slot” to refer to an id for a location within a contiguous storage buffer that can hold a single item of some kind. “Offset” is sometimes used in public APIs (to refer to integer distances within collections, relative to some start position), and using the same term here causes internal naming difficulties. A “bucket” is therefore a more specific name for a slot within a hash table.

`RawCollider` doesn’t like to get called through the high-level hashing APIs.

…script

# Conflicts: # Tests/PersistentCollections2Tests/PersistentMap Fixtures.swift

Base the allocation request on the currently occupied space, not the full capacity. The original code tended to lead to unnecessary doubling of the node size in each copy, until we exhaust the 32 bit capacity value. (Oops.)

lorentey · 2022-09-18T01:15:00Z

@swift-ci test

lorentey · 2022-09-18T01:21:21Z

Iteration performance is boosted when the dictionary starts to go beyond a single layer of nodes.
(I suspect there is some constant overhead here that might be possible to eliminate. Retain/release traffic?)

lorentey · 2022-09-18T01:24:35Z

index(forKey:) performance is slower than I expect, reinforcing my suspicion about there being some bogus overhead. Looking at generated code will probably make it obvious where the time goes.

lorentey · 2022-09-18T01:30:05Z

The new in-place mutation support does show up as a bit of improvement on these microbenchmarks, but support for this is mostly exciting because it prevents CoW copies for the mutated value -- it's not necessary about raw lookup performance.

That said, in-place mutation for the defaulted subscript gets a solid 53% performance boost:

lorentey · 2022-09-18T01:31:47Z

Benchmark summary (compared to #177):

Tasks with difference scores larger than 1.05:
  Score   Sum     Improvements Regressions  Name
  17.51   17.51   17.51(#75)   0.9993(#1)   PersistentDictionary<Int, Int> indexing subscript (*)
  1.887   1.887   1.967(#55)   0.9207(#21)  PersistentDictionary<Int, Int>.Keys sequential iteration (*)
  1.881   1.881   1.963(#55)   0.9184(#21)  PersistentDictionary<Int, Int>.Values sequential iteration (*)
  1.738   1.738   1.870(#55)   0.8676(#21)  PersistentDictionary<Int, Int> sequential iteration, indices (*)
  1.596   0.626   1.001(#2)    0.6260(#74)  PersistentDictionary<Int, Int> unsuccessful index(forKey:) (*)
  1.532   1.532   1.532(#75)   0.9997(#1)   PersistentDictionary<Int, Int> defaulted subscript, _modify existing (*)
  1.108   1.108   1.113(#74)   0.9953(#2)   PersistentDictionary<Int, Int> subscript, _modify (*)
  1.086   1.086   1.099(#59)   0.9874(#17)  PersistentDictionary<Int, Int> random removals (missing keys) (*)
  1.081   1.081   1.082(#75)   0.9989(#1)   PersistentDictionary<Int, Int> defaulted subscript, _modify missing (*)
  1.065   1.065   1.069(#60)   0.9964(#16)  PersistentDictionary<Int, Int> subscript, remove existing, unique (*)
  1.064   0.940   1.010(#6)    0.9301(#70)  PersistentDictionary<Int, Int> updateValue(_:forKey:), insert (*)
  1.063   1.063   1.065(#70)   0.9981(#6)   PersistentDictionary<Int, Int> subscript, successful lookups (*)
  1.059   1.059   1.065(#65)   0.9943(#11)  PersistentDictionary<Int, Int> defaulted subscript, successful lookups (*)

lorentey · 2022-09-18T03:06:20Z

Last commit fixes index(forKey:) performance:

Tasks with difference scores larger than 1.05:
  Score   Sum     Improvements Regressions  Name
  17.33   17.33   17.33(#75)   0.9993(#1)   PersistentDictionary<Int, Int> indexing subscript (*)
  1.983   1.983   1.983(#76)   1.000(#0)    PersistentDictionary<Int, Int> successful index(forKey:) (*)
  1.867   1.867   1.949(#55)   0.9175(#21)  PersistentDictionary<Int, Int>.Keys sequential iteration (*)
  1.851   1.851   1.937(#55)   0.9142(#21)  PersistentDictionary<Int, Int>.Values sequential iteration (*)
  1.713   1.713   1.843(#55)   0.8699(#21)  PersistentDictionary<Int, Int> sequential iteration, indices (*)
  1.526   1.526   1.526(#76)   1.000(#0)    PersistentDictionary<Int, Int> defaulted subscript, _modify existing (*)
  1.272   1.272   1.272(#76)   1.000(#0)    PersistentDictionary<Int, Int> unsuccessful index(forKey:) (*)
  1.223   1.223   1.223(#76)   1.000(#0)    PersistentDictionary<Int, Int> random removals (missing keys) (*)
  1.184   1.184   1.184(#75)   1.000(#1)    PersistentDictionary<Int, Int> subscript, _modify (*)
  1.183   1.183   1.183(#76)   1.000(#0)    PersistentDictionary<Int, Int> subscript, noop setter (*)
  1.167   1.167   1.167(#76)   1.000(#0)    PersistentDictionary<Int, Int> subscript, remove missing (*)
  1.103   1.103   1.105(#64)   0.9983(#12)  PersistentDictionary<Int, Int> subscript, remove existing, unique (*)
  1.071   1.071   1.076(#66)   0.9957(#10)  PersistentDictionary<Int, Int> defaulted subscript, successful lookups (*)
  1.069   1.069   1.071(#71)   0.9978(#5)   PersistentDictionary<Int, Int> subscript, successful lookups (*)
  1.068   0.936   1.005(#10)   0.9312(#66)  PersistentDictionary<Int, Int> updateValue(_:forKey:), insert (*)
  1.065   1.065   1.067(#65)   0.9987(#11)  PersistentDictionary<Int, Int> random removals (existing keys) (*)

lorentey · 2022-09-18T03:06:41Z

@swift-ci test

msteindorfer

LGTM

Sources/PersistentCollections/Node/_Node+UnsafeHandle.swift

Sources/PersistentCollections/Node/_Node.swift

Sources/PersistentCollections/Node/_Level.swift

Sources/PersistentCollections/Node/_Node+Mutations.swift

…v1.1.0 (#906) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [com_github_apple_swift_collections](https://togithub.com/apple/swift-collections) | http_archive | minor | `1.0.6` -> `1.1.0` | --- ### Release Notes <details> <summary>apple/swift-collections (com_github_apple_swift_collections)</summary> ### [`v1.1.0`](https://togithub.com/apple/swift-collections/releases/tag/1.1.0): Swift Collections 1.1.0 [Compare Source](https://togithub.com/apple/swift-collections/compare/1.0.6...1.1.0) This feature release adds a number of new data structure implementations, along with minor changes to existing constructs. ##### New Data Structures - `Heap` implements a min-max heap, backed by a native array. (Contributed by [@AquaGeek](https://togithub.com/AquaGeek)) - `BitSet` and `BitArray` are two alternate representations of a bitmap type, backed by dynamically allocated storage. (Contributed by [@MahanazAtiqullah](https://togithub.com/MahanazAtiqullah)) - `TreeSet` and `TreeDictionary` are hashed collections implementing Compressed Hash-Array Mapped Prefix Trees (CHAMP). They provide similar API as `Set`/`Dictionary` in the Standard Library, but as persistent data structures, supporting incremental mutations of shared instances and efficient structural diffing. (Contributed by [@msteindorfer](https://togithub.com/msteindorfer)) ##### Other Changes - This version of the package can only be built using Swift 5.7 or later. - New methods: the `OrderedSet.isEqualSet` family of functions provide a way to test that two containers contain the same members, ignoring the order of elements. ([apple/swift-collections#183, [apple/swift-collections#234) - New method: `OrderedSet.filter` implements a version of the standard filter operation that returns an `OrderedSet` instead of an `Array`. ([apple/swift-collections#159) - `debugDescription` implementations have been updated to follow Swift best practice. (These are called by container types like `Array` to print their elements, so they work best when they're succinct variants of `description` that are suitable for embedding in structured output: specifically, they must not produce unpaired delimiter characters (`[`/`]`, `(`/`)`, `{`/`}`, `<`/`>` etc), raw top level commas, semicolons, colons, unquoted strings etc. `debugDescription` should not needlessly print type names etc.) ##### New Contributors - [@AquaGeek](https://togithub.com/AquaGeek) made their first contribution in [apple/swift-collections#61 - [@ejmarchant](https://togithub.com/ejmarchant) made their first contribution in [apple/swift-collections#82 - [@just-gull](https://togithub.com/just-gull) made their first contribution in [apple/swift-collections#115 - [@jPaolantonio](https://togithub.com/jPaolantonio) made their first contribution in [apple/swift-collections#121 - [@MahanazAtiqullah](https://togithub.com/MahanazAtiqullah) made their first contribution in [apple/swift-collections#83 - [@hectormatos2011](https://togithub.com/hectormatos2011) made their first contribution in [apple/swift-collections#155 - [@ktoso](https://togithub.com/ktoso) made their first contribution in [apple/swift-collections#159 - [@CTMacUser](https://togithub.com/CTMacUser) made their first contribution in [apple/swift-collections#116 - [@hassila](https://togithub.com/hassila) made their first contribution in [apple/swift-collections#297 Many thanks to our contributors for their great work (and patience)! ##### List of Pull Requests **Full Changelog**: apple/swift-collections@1.0.6...1.1.0 - Add a min-max heap implementation that can be used to back a priority queue by [@AquaGeek](https://togithub.com/AquaGeek) in [apple/swift-collections#61 - \[benchmark] Review and extend Heap benchmarks by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#76 - Add reference benchmarks for bit vector implementations by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#79 - Fix Markdown link in README by [@AquaGeek](https://togithub.com/AquaGeek) in [apple/swift-collections#77 - Fix documentation for types conforming to ExpressibleByArrayLiteral o… by [@ejmarchant](https://togithub.com/ejmarchant) in [apple/swift-collections#82 - \[Heap] Performance tweaks by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#78 - Fix typos: missing subscript parameters by [@ejmarchant](https://togithub.com/ejmarchant) in [apple/swift-collections#81 - \[Heap] Update implementation details section in docs by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#84 - Update CMakeLists.txt by [@compnerd](https://togithub.com/compnerd) in [apple/swift-collections#85 - Stop depending on swift-collections-benchmark by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#86 - \[OrderedDictionary] modifyValue → updateValue by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#91 - Add Benchmarks package to workspace by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#93 - \[OrderedDictionary] Deprecate `subscript(offset:)` for now by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#92 - Documentation: Remove in-place mutation comments by [@ejmarchant](https://togithub.com/ejmarchant) in [apple/swift-collections#96 - \[main] Freeze some types for consistency with their inlinable initializers by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#98 - Follow stdlib's leading underscore rule by [@ejmarchant](https://togithub.com/ejmarchant) in [apple/swift-collections#95 - \[Heap] Disable heap tests in release builds by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#100 - \[NFC] Merge release/1.0 to main by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#105 - Merge `release/1.0` into `main` by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#108 - \[README] Note that `Heap` hasn't been tagged yet & list other enhancements in progress by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#109 - PriorityQueueModule: remove `import Foundation` by [@compnerd](https://togithub.com/compnerd) in [apple/swift-collections#118 - \[Heap] Remove Heap's `ascending` and `descending` views by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#119 - \[Heap] Enable heap tests in optimized builds ([#101](https://togithub.com/apple/swift-collections/issues/101)) by [@just-gull](https://togithub.com/just-gull) in [apple/swift-collections#115 - Update CMakeLists.txt by [@compnerd](https://togithub.com/compnerd) in [apple/swift-collections#122 - Fix link to package internal documentation by [@jPaolantonio](https://togithub.com/jPaolantonio) in [apple/swift-collections#121 - Merge release/1.0 to main by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#130 - BitArray and BitSet data structures by [@MahanazAtiqullah](https://togithub.com/MahanazAtiqullah) in [apple/swift-collections#83 - Sorted collections by [@vihanb](https://togithub.com/vihanb) in [apple/swift-collections#65 - Merge `release/1.0` to `main` by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#141 - Remove Swift PM Artifacts to avoid Generated Schemes in Xcode by [@hectormatos2011](https://togithub.com/hectormatos2011) in [apple/swift-collections#155 - Reinstate custom schemes under Utils/swift-collections.xcworkspace by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#156 - \+OrderedSet add filter [#158](https://togithub.com/apple/swift-collections/issues/158) by [@ktoso](https://togithub.com/ktoso) in [apple/swift-collections#159 - \[Xcode] Update schemes & file template by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#161 - \[OrderedCollection] Use standard temp allocation facility, if available by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#160 - \[OrderedSet] Work around weird name lookup issue in compiler by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#162 - \=OrderedSet.filter Attempt to optimize filter impl by [@ktoso](https://togithub.com/ktoso) in [apple/swift-collections#163 - Force-inline \_modify accessors to work around a performance issue by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#165 - Merge release/1.0 branch to main by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#172 - Incubate persistent data structures by [@msteindorfer](https://togithub.com/msteindorfer) in [apple/swift-collections#31 - Persistent collections updates by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#174 - Persistent collections updates by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#175 - Persistent collections updates (part 3) by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#176 - Persistent collections updates (part 4) by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#177 - \[OrderedDictionary] Tiny documentation fix by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#178 - Persistent collections updates (part 5) by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#179 - Integrate PriorityQueueModule, BitCollections, PersistentCollections, SortedCollections into release/1.1 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#181 - Persistent collections updates (part 6) by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#180 - Persistent collections updates (part 7) by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#182 - \[BitSet] Fix decoding format on 32 bit architectures by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#185 - Persistent collections updates (part 8) by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#184 - Persistent collections updates (part 9) by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#188 - Add Sendable conformances to all public types by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#191 - \[test] Check baseline API expectations for set-like types by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#192 - Fleshing out `PersistentSet` by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#193 - Rename `PriorityQueueModule` to `HeapModule` by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#194 - \[BitSet] Fix invariant violation in member subscript by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#195 - \[1.1.0] Bump minimum required Swift toolchain to 5.5 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#196 - Restore support for building with Swift 5.5 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#198 - Merge release/1.0 to release/1.1 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#199 - Update CMake configuration in preparation for 1.1 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#200 - Update CMakeLists.txt by [@compnerd](https://togithub.com/compnerd) in [apple/swift-collections#202 - \[1.1]\[SortedCollections] Remove for now by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#205 - \[Heap] Change value of minimum or maximum element by [@CTMacUser](https://togithub.com/CTMacUser) in [apple/swift-collections#116 - \[Heap] Prerelease preparations by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#210 - Update README files by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#212 - \[manifest] Exclude CMakeLists.txt; remove unnecessary path settings; reindent file by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#213 - Review and finalize (?) set relation predicates for 1.1 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#216 - Merge.1.0→1.1 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#217 - Update CMake configuration by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#218 - Cherry pick changes from main to release/1.1 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#219 - Fix unusual build problems by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#221 - Review & update descriptions throughout the package by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#222 - Review and finalize (?) binary set operations for 1.1 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#223 - \[Xcode] Disable implicit dependencies by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#227 - \[OrderedSet] Improve sequence-taking initializer by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#226 - \[OrderedSet, BitSet] Fix custom mirror display style by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#225 - Finalize persistent collections API by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#224 - Start working on DocC support by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#228 - Update CMake build configuration by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#230 - \[PersistentSet] Iterator.next(): Make inlinable by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#233 - \[SetAlgebra types] isEqual(to:) → isEqualSet(to:) by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#234 - \[PersistentCollections] Doc & benchmark updates in preparation of API review by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#235 - Apply changes from in-progress review thread by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#237 - \[ShareableHashedCollections] API Review: add missing `mutating` keywords by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#238 - \[Benchmarks] Split default huge library up into individual files, one per type by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#240 - \[ShareableHashedCollections] Add missing import by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#243 - Unify unsafe bit set implementations by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#244 - Fix warnings in development versions of Swift by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#245 - \[HashTreeCollections] Change prefix `Shareable` to `Tree` by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#242 - \[TreeDictionary] Fix in-place merge operation to properly update the count by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#247 - \[cmake] Update CMake configuration by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#249 - Add not so experimental rope implementation by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#264 - Fix off by one error in BitSet by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#267 - Monomodule support by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#266 - Expose a handful of BigString.Index methods by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#269 - BitArray API refinements & additions by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#263 - Make most of Rope inlinable by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#270 - BigString: Fix String.Index.\_knownScalarAligned by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#272 - Rope: Fix `Sendable` conformance by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#271 - Require Swift 5.6 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#273 - \[OrderedDictionary] Explicitly mention in documentation that keys/values are ordered by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#275 - \[HashCollections] Ensure `self` doesn’t get destroyed before we’re done working with it by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#276 - Remove obsolete swift(>=5.5)/swift(>=5.6) checks by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#277 - \[Xcode] Update Xcode project by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#278 - Minor rope updates by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#279 - \[Rope] remove(at:): Fix assertion when removing the last item creates a deficiency by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#280 - Merge changes from release 1.0 to release 1.1 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#283 - \[CollectionUtilities] Silence a warning on 32 bit platforms by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#286 - \[BitSet] Fix a thinko in BitSet.isEqualSet by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#287 - Grab bag of fixes for small test issues by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#288 - \[Xcode] Set a code sign identity in the Xcode project by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#285 - Rope: Fix trap when replaceSubrange is called on an empty rope by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#290 - `Rope.find` returns a bogus remainder for the end position by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#291 - Update TreeSet.md by [@hassila](https://togithub.com/hassila) in [apple/swift-collections#297 - Fix CustomStringConvertible/CustomDebugStringConvertible conformances by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#302 - \[RopeModule] Fix issues in Swift's ABI stable dialect by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#318 - \[CMake, Xcode] Update configurations for alternate build systems by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#319 - \[RopeModule] Remove unnecessary typealiases by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#320 - \[Heap] Improve type-level doc comment by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#326 - \[Heap] insert(contentsOf:) Switch to Floyd’s if we’re inserting too many items by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#327 - \[1.1] build: support building in Debug mode on Windows by [@compnerd](https://togithub.com/compnerd) in [apple/swift-collections#336 - Merge release/1.0 to release/1.1 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#348 - \[Heap] Convert min() and max() to properties by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#328 - \[BitArray] Disable bitwise operators for now by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#353 - \[Heap] Final(?) Heap API adjustments for 1.1 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#354 - \[Benchmarks] Add Collection Equality Benchmarks by [@vanvoorden](https://togithub.com/vanvoorden) in [apple/swift-collections#351 - Release preparations for 1.1 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#355 - \[1.1] Fix typos by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#357 - \[TreeDictionary]\[Keys] Add Equatable and Hashable Conformance to TreeDictionary.Keys by [@vanvoorden](https://togithub.com/vanvoorden) in [apple/swift-collections#352 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://togithub.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://togithub.com/renovatebot/renovate).  Co-authored-by: Self-hosted Renovate Bot <361546+cgrindel-self-hosted-renovate[bot]@users.noreply.github.enterprise.com>

lorentey added 17 commits September 15, 2022 20:56

[PersistentCollections] Reverse ordering of items in node storage

2e523b1

[PersistentCollections] _Level: Store the shift amount in an UInt8

edaadc9

[PersistentCollections] Flesh out _RawNode, add _UnmanagedNode

6dd0211

These provide a barebones, read-only, non-generic abstraction, with just enough functionality to allow navigating around in the tree structure. We will use these to implement path-based indices.

[PersistentCollections] Rework basic node properties

547ac90

[PersistentCollections] Allow dumping hash trees in iteration order

16393ea

[test] LifetimeTracked: Implement high-fidelity hash forwarding

52e3ea6

`RawCollider` doesn’t like to get called through the high-level hashing APIs.

[PersistentCollections] Internal doc updates

6cf128e

[OrderedDictionary] Implement index invalidation

1324c4e

[PersistentDictionary] Implement in-place mutations for defaulted sub…

1661bd4

…script

[PersistentDictionary] Add some docs

3b7d241

[PersistentDictionary] Implement in-place mutations

57deed5

# Conflicts: # Tests/PersistentCollections2Tests/PersistentMap Fixtures.swift

[PersistentCollections] Fix node sizing logic

2321b34

Base the allocation request on the currently occupied space, not the full capacity. The original code tended to lead to unnecessary doubling of the node size in each copy, until we exhaust the 32 bit capacity value. (Oops.)

[PersistentCollections] Reduce _Bucket’s storage size

5240bf6

lorentey requested a review from msteindorfer September 18, 2022 01:14

[PersistentDictionary] Fix index(forKey:) performance

d45f264

msteindorfer approved these changes Sep 19, 2022

View reviewed changes

lorentey merged commit 7499bc2 into apple:feature/PersistentCollections Sep 19, 2022

lorentey deleted the PersistentCollections-updates branch September 19, 2022 17:56

lorentey added this to the 1.1.0 milestone Oct 10, 2022

lorentey added the HashTreeCollections Hash-array mapped prefix tree implementations label Nov 15, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Persistent collections updates (part 5) #179

Persistent collections updates (part 5) #179

lorentey commented Sep 18, 2022

lorentey commented Sep 18, 2022

lorentey commented Sep 18, 2022

lorentey commented Sep 18, 2022

lorentey commented Sep 18, 2022

lorentey commented Sep 18, 2022

lorentey commented Sep 18, 2022

lorentey commented Sep 18, 2022

msteindorfer left a comment

Persistent collections updates (part 5) #179

Persistent collections updates (part 5) #179

Conversation

lorentey commented Sep 18, 2022

Checklist

lorentey commented Sep 18, 2022

lorentey commented Sep 18, 2022

lorentey commented Sep 18, 2022

lorentey commented Sep 18, 2022

lorentey commented Sep 18, 2022

lorentey commented Sep 18, 2022

lorentey commented Sep 18, 2022

msteindorfer left a comment

Choose a reason for hiding this comment