Persistent collections updates (part 8) #184

lorentey · 2022-09-23T19:04:30Z

Implement logarithmic-time index(_:offsetBy:) and index(_:offsetBy:limitedBy:) operations.
Cache the shared hash value of items in each collision node. (At the front of the storage buffer, where child entries would otherwise be stored.)
Review and sanitize/simplify node allocation logic. Ditto for key-based lookups.
Implement better handling of compressed paths in isSubset(of:)/isDisjoint(with:)

Checklist

I've read the Contribution Guidelines
My contributions are licensed under the Swift license.
I've followed the coding style of the rest of the project.
I've added tests covering all new code paths my change adds to the project (if appropriate).
I've added benchmarks covering new functionality (if appropriate).
I've verified that my change does not break any existing tests or introduce unexplained benchmark regressions.
I've updated the documentation if necessary.

…tself Also, don’t check `DefaultIndices` — it’s fine to assume it implements Collection as long as the base collection is correct.

…ex(_:offsetBy:limitedBy:)

…ild in a non-unique node Oops, this did not account for reversed item order, so it was spawning the new child with the wrong item.

This will simplify merge operations as we no longer have to spend effort on preventing repeated rehashings of items within collision nodes. Also, review and simplify node allocation and lookup code.

…ubset/isDisjoint

lorentey · 2022-09-23T19:28:09Z

@swift-ci test

lorentey · 2022-09-23T20:17:15Z

Showing off the algorithmic improvement by striding over the collection in 10 equal steps using index(_:offsetBy:):

That's 2066x faster on average. 🤓

The new executable target `memory-benchmark` collects some memory usage statistics and outputs average memory efficiency for a range of sizes in CSV format. Bump required toolchain version for benchmarks to Swift 5.5.

lorentey · 2022-09-24T00:02:11Z

The last commit adds a very rudimentary memory benchmark, as a standalone executable target in the hidden benchmarking package. It produces CSV output, comparing the memory efficiency of the standard Dictionary vs PersistentDictionary. Current (untuned) results are:

This plots memory efficiency as a function of collection size. Higher values are better.

I define "memory efficiency" as the net amount of memory needed to store the items in the collection divided by the gross amount of memory allocated overall. This ratio generally depends on the memory layout of the Key/Value types -- smaller payloads generally lead to lower memory efficiency, especially for tree-based collections. (As the relative weight of node headers + child references will be higher.)

For Dictionary<String, String>, average memory efficiency (over a logarithmic scale of sizes from 1 to 1M) is about 55%, while for PersistentSet<String, String> we currently measure about 54% -- this is pretty good, considering we haven't really tuned the allocation patterns yet. (The HAMT is really efficient for representing small dictionaries, because of the node compression scheme -- this small-scale behavior is why it is able to catch up to the standard Dictionary. The log scale gives weight to small values, which would be a bit unfair, except it tends to match actual usage -- small data sets are more frequent.)

Of course, this is measuring a single collection instance -- the memory efficiency of PersistentSet goes through the roof in use cases that actually make use of persistency. (Mutated copies of the same original dictionary share their storage nodes, making them extremely memory efficient.)

lorentey · 2022-09-24T04:33:56Z

d87f5fd replaces recursion with looping for simple key-based lookups, resulting in a really nice 47-59% speedup.

Tasks with difference scores larger than 1.05:
  Score   Sum     Improvements Regressions  Name
  1.595   1.595   1.595(#75)   0.9998(#1)   PersistentDictionary<Int, Int> defaulted subscript, successful lookups (*)
  1.584   1.584   1.584(#76)   1.000(#0)    PersistentDictionary<Int, Int> subscript, successful lookups (*)
  1.486   1.486   1.486(#75)   0.9997(#1)   PersistentDictionary<Int, Int> defaulted subscript, unsuccessful lookups (*)
  1.474   1.474   1.476(#75)   0.9984(#1)   PersistentDictionary<Int, Int> subscript, unsuccessful lookups (*)

…StringConvertible, CustomReflectable conformances

… (untested) This adds an experimental _FastMembershipCheckable protocol to speed up these relationship predicates when possible — having separate overloads for each known collection with a fast containment check is not going to scale for long. This commit also introduces a set of isEqual(to:) methods that generalize ==.

Plus add tests covering these as well and mapValues.

…ring (untested)

…ns (untested)

…istentSet and a value transform

lorentey · 2022-09-25T00:22:46Z

Latest commits:

PersistentSet, PersistentDictionary: Add more protocol conformances
PersistentSet: Add isEqual/isSubset/isSuperset/isStrictSubset/isStrictSuperset/isDisjoint overloads
_FastMembershipCheckable: New utility protocol to make the overload set more manageable
PersistentDictionary: Add structural filter and compactMapValues operations
PersistentSet: implement structural filtering, subtraction, intersection
PersistentDictionary: Add an initializer to quickly convert a PersistentSet into a dictionary (without rehashing anything)

Note: The structural filtering/intersection/subtraction operations do not preserve unchanged nodes yet.

lorentey · 2022-09-25T00:24:10Z

It is of course also possible to implement structural union and symmetricDifference operations (as well as structural dictionary merges) -- this is still tbd.

lorentey · 2022-09-25T00:24:31Z

@swift-ci test

msteindorfer

LGTM

msteindorfer · 2022-09-26T13:59:39Z

Sources/PersistentCollections/Node/_Node+Lookups.swift

  internal func get(_ level: _Level, _ key: Key, _ hash: _Hash) -> Value? {
-    read {
-      guard let r = $0.find(level, key, hash) else { return nil }
-      if r.descend {
-        return $0[child: r.slot].get(level.descend(), key, hash)
+    var node = unmanaged
+    var level = level
+    while true {
+      let r = UnsafeHandle.read(node) { $0.find(level, key, hash) }
+      guard let r = r else {
+        return nil
      }
-      return $0[item: r.slot].value
+      guard r.descend else {
+        return UnsafeHandle.read(node) { $0[item: r.slot].value }
+      }
+      node = node.unmanagedChild(at: r.slot)
+      level = level.descend()
    }
  }


It's somewhat surprising just how much of an improvement this turned out to be. IIRC the recursive variant didn't have any ARC traffic, either, so the difference must be largely due to stack overhead. I'm happy to take it though!

msteindorfer · 2022-09-26T14:00:03Z

Sources/PersistentCollections/Node/_Node+Lookups.swift

  internal func containsKey(
    _ level: _Level, _ key: Key, _ hash: _Hash
  ) -> Bool {
-    read { $0.containsKey(level, key, hash) }
-  }
-}
-
-extension _Node.UnsafeHandle {
-  @inlinable
-  internal func containsKey(
-    _ level: _Level, _ key: Key, _ hash: _Hash
-  ) -> Bool {
-    guard let r = find(level, key, hash) else { return false }
-    guard r.descend else { return true }
-    return self[child: r.slot].containsKey(level.descend(), key, hash)
+    var node = unmanaged
+    var level = level
+    while true {
+      let r = UnsafeHandle.read(node) { $0.find(level, key, hash) }
+      guard let r = r else { return false }
+      guard r.descend else { return true }
+      node = node.unmanagedChild(at: r.slot)
+      level = level.descend()
+    }
  }


msteindorfer · 2022-09-26T14:17:44Z

Sources/PersistentCollections/Node/_Node+Builder.swift

+extension _Node {
+  @usableFromInline
+  @frozen
+  internal enum Builder {


msteindorfer · 2022-09-26T15:27:38Z

Sources/PersistentCollections/Node/_Node+Structural filter.swift

+    _ hashPrefix: _Hash,
+    _ isIncluded: (Element) throws -> Bool
+  ) rethrows -> Builder {
+    // FIXME: Consider preserving `self` when nothing needs to be removed.


You should be able to check sub-tree sizes. If the builder result would have the same size as self, you can return self.

That's a great shortcut -- I may just take it!

I'm hoping we can eventually prevent building the result when we'd be throwing it away. It feels like something that should be possible to express without too much additional complexity.

The same shortcut also applies to the other operations that make use of the builder abstraction.

…v1.1.0 (#906) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [com_github_apple_swift_collections](https://togithub.com/apple/swift-collections) | http_archive | minor | `1.0.6` -> `1.1.0` | --- ### Release Notes <details> <summary>apple/swift-collections (com_github_apple_swift_collections)</summary> ### [`v1.1.0`](https://togithub.com/apple/swift-collections/releases/tag/1.1.0): Swift Collections 1.1.0 [Compare Source](https://togithub.com/apple/swift-collections/compare/1.0.6...1.1.0) This feature release adds a number of new data structure implementations, along with minor changes to existing constructs. ##### New Data Structures - `Heap` implements a min-max heap, backed by a native array. (Contributed by [@AquaGeek](https://togithub.com/AquaGeek)) - `BitSet` and `BitArray` are two alternate representations of a bitmap type, backed by dynamically allocated storage. (Contributed by [@MahanazAtiqullah](https://togithub.com/MahanazAtiqullah)) - `TreeSet` and `TreeDictionary` are hashed collections implementing Compressed Hash-Array Mapped Prefix Trees (CHAMP). They provide similar API as `Set`/`Dictionary` in the Standard Library, but as persistent data structures, supporting incremental mutations of shared instances and efficient structural diffing. (Contributed by [@msteindorfer](https://togithub.com/msteindorfer)) ##### Other Changes - This version of the package can only be built using Swift 5.7 or later. - New methods: the `OrderedSet.isEqualSet` family of functions provide a way to test that two containers contain the same members, ignoring the order of elements. ([apple/swift-collections#183, [apple/swift-collections#234) - New method: `OrderedSet.filter` implements a version of the standard filter operation that returns an `OrderedSet` instead of an `Array`. ([apple/swift-collections#159) - `debugDescription` implementations have been updated to follow Swift best practice. (These are called by container types like `Array` to print their elements, so they work best when they're succinct variants of `description` that are suitable for embedding in structured output: specifically, they must not produce unpaired delimiter characters (`[`/`]`, `(`/`)`, `{`/`}`, `<`/`>` etc), raw top level commas, semicolons, colons, unquoted strings etc. `debugDescription` should not needlessly print type names etc.) ##### New Contributors - [@AquaGeek](https://togithub.com/AquaGeek) made their first contribution in [apple/swift-collections#61 - [@ejmarchant](https://togithub.com/ejmarchant) made their first contribution in [apple/swift-collections#82 - [@just-gull](https://togithub.com/just-gull) made their first contribution in [apple/swift-collections#115 - [@jPaolantonio](https://togithub.com/jPaolantonio) made their first contribution in [apple/swift-collections#121 - [@MahanazAtiqullah](https://togithub.com/MahanazAtiqullah) made their first contribution in [apple/swift-collections#83 - [@hectormatos2011](https://togithub.com/hectormatos2011) made their first contribution in [apple/swift-collections#155 - [@ktoso](https://togithub.com/ktoso) made their first contribution in [apple/swift-collections#159 - [@CTMacUser](https://togithub.com/CTMacUser) made their first contribution in [apple/swift-collections#116 - [@hassila](https://togithub.com/hassila) made their first contribution in [apple/swift-collections#297 Many thanks to our contributors for their great work (and patience)! ##### List of Pull Requests **Full Changelog**: apple/swift-collections@1.0.6...1.1.0 - Add a min-max heap implementation that can be used to back a priority queue by [@AquaGeek](https://togithub.com/AquaGeek) in [apple/swift-collections#61 - \[benchmark] Review and extend Heap benchmarks by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#76 - Add reference benchmarks for bit vector implementations by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#79 - Fix Markdown link in README by [@AquaGeek](https://togithub.com/AquaGeek) in [apple/swift-collections#77 - Fix documentation for types conforming to ExpressibleByArrayLiteral o… by [@ejmarchant](https://togithub.com/ejmarchant) in [apple/swift-collections#82 - \[Heap] Performance tweaks by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#78 - Fix typos: missing subscript parameters by [@ejmarchant](https://togithub.com/ejmarchant) in [apple/swift-collections#81 - \[Heap] Update implementation details section in docs by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#84 - Update CMakeLists.txt by [@compnerd](https://togithub.com/compnerd) in [apple/swift-collections#85 - Stop depending on swift-collections-benchmark by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#86 - \[OrderedDictionary] modifyValue → updateValue by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#91 - Add Benchmarks package to workspace by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#93 - \[OrderedDictionary] Deprecate `subscript(offset:)` for now by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#92 - Documentation: Remove in-place mutation comments by [@ejmarchant](https://togithub.com/ejmarchant) in [apple/swift-collections#96 - \[main] Freeze some types for consistency with their inlinable initializers by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#98 - Follow stdlib's leading underscore rule by [@ejmarchant](https://togithub.com/ejmarchant) in [apple/swift-collections#95 - \[Heap] Disable heap tests in release builds by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#100 - \[NFC] Merge release/1.0 to main by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#105 - Merge `release/1.0` into `main` by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#108 - \[README] Note that `Heap` hasn't been tagged yet & list other enhancements in progress by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#109 - PriorityQueueModule: remove `import Foundation` by [@compnerd](https://togithub.com/compnerd) in [apple/swift-collections#118 - \[Heap] Remove Heap's `ascending` and `descending` views by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#119 - \[Heap] Enable heap tests in optimized builds ([#101](https://togithub.com/apple/swift-collections/issues/101)) by [@just-gull](https://togithub.com/just-gull) in [apple/swift-collections#115 - Update CMakeLists.txt by [@compnerd](https://togithub.com/compnerd) in [apple/swift-collections#122 - Fix link to package internal documentation by [@jPaolantonio](https://togithub.com/jPaolantonio) in [apple/swift-collections#121 - Merge release/1.0 to main by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#130 - BitArray and BitSet data structures by [@MahanazAtiqullah](https://togithub.com/MahanazAtiqullah) in [apple/swift-collections#83 - Sorted collections by [@vihanb](https://togithub.com/vihanb) in [apple/swift-collections#65 - Merge `release/1.0` to `main` by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#141 - Remove Swift PM Artifacts to avoid Generated Schemes in Xcode by [@hectormatos2011](https://togithub.com/hectormatos2011) in [apple/swift-collections#155 - Reinstate custom schemes under Utils/swift-collections.xcworkspace by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#156 - \+OrderedSet add filter [#158](https://togithub.com/apple/swift-collections/issues/158) by [@ktoso](https://togithub.com/ktoso) in [apple/swift-collections#159 - \[Xcode] Update schemes & file template by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#161 - \[OrderedCollection] Use standard temp allocation facility, if available by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#160 - \[OrderedSet] Work around weird name lookup issue in compiler by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#162 - \=OrderedSet.filter Attempt to optimize filter impl by [@ktoso](https://togithub.com/ktoso) in [apple/swift-collections#163 - Force-inline \_modify accessors to work around a performance issue by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#165 - Merge release/1.0 branch to main by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#172 - Incubate persistent data structures by [@msteindorfer](https://togithub.com/msteindorfer) in [apple/swift-collections#31 - Persistent collections updates by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#174 - Persistent collections updates by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#175 - Persistent collections updates (part 3) by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#176 - Persistent collections updates (part 4) by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#177 - \[OrderedDictionary] Tiny documentation fix by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#178 - Persistent collections updates (part 5) by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#179 - Integrate PriorityQueueModule, BitCollections, PersistentCollections, SortedCollections into release/1.1 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#181 - Persistent collections updates (part 6) by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#180 - Persistent collections updates (part 7) by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#182 - \[BitSet] Fix decoding format on 32 bit architectures by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#185 - Persistent collections updates (part 8) by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#184 - Persistent collections updates (part 9) by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#188 - Add Sendable conformances to all public types by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#191 - \[test] Check baseline API expectations for set-like types by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#192 - Fleshing out `PersistentSet` by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#193 - Rename `PriorityQueueModule` to `HeapModule` by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#194 - \[BitSet] Fix invariant violation in member subscript by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#195 - \[1.1.0] Bump minimum required Swift toolchain to 5.5 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#196 - Restore support for building with Swift 5.5 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#198 - Merge release/1.0 to release/1.1 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#199 - Update CMake configuration in preparation for 1.1 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#200 - Update CMakeLists.txt by [@compnerd](https://togithub.com/compnerd) in [apple/swift-collections#202 - \[1.1]\[SortedCollections] Remove for now by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#205 - \[Heap] Change value of minimum or maximum element by [@CTMacUser](https://togithub.com/CTMacUser) in [apple/swift-collections#116 - \[Heap] Prerelease preparations by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#210 - Update README files by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#212 - \[manifest] Exclude CMakeLists.txt; remove unnecessary path settings; reindent file by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#213 - Review and finalize (?) set relation predicates for 1.1 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#216 - Merge.1.0→1.1 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#217 - Update CMake configuration by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#218 - Cherry pick changes from main to release/1.1 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#219 - Fix unusual build problems by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#221 - Review & update descriptions throughout the package by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#222 - Review and finalize (?) binary set operations for 1.1 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#223 - \[Xcode] Disable implicit dependencies by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#227 - \[OrderedSet] Improve sequence-taking initializer by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#226 - \[OrderedSet, BitSet] Fix custom mirror display style by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#225 - Finalize persistent collections API by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#224 - Start working on DocC support by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#228 - Update CMake build configuration by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#230 - \[PersistentSet] Iterator.next(): Make inlinable by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#233 - \[SetAlgebra types] isEqual(to:) → isEqualSet(to:) by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#234 - \[PersistentCollections] Doc & benchmark updates in preparation of API review by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#235 - Apply changes from in-progress review thread by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#237 - \[ShareableHashedCollections] API Review: add missing `mutating` keywords by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#238 - \[Benchmarks] Split default huge library up into individual files, one per type by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#240 - \[ShareableHashedCollections] Add missing import by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#243 - Unify unsafe bit set implementations by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#244 - Fix warnings in development versions of Swift by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#245 - \[HashTreeCollections] Change prefix `Shareable` to `Tree` by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#242 - \[TreeDictionary] Fix in-place merge operation to properly update the count by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#247 - \[cmake] Update CMake configuration by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#249 - Add not so experimental rope implementation by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#264 - Fix off by one error in BitSet by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#267 - Monomodule support by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#266 - Expose a handful of BigString.Index methods by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#269 - BitArray API refinements & additions by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#263 - Make most of Rope inlinable by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#270 - BigString: Fix String.Index.\_knownScalarAligned by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#272 - Rope: Fix `Sendable` conformance by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#271 - Require Swift 5.6 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#273 - \[OrderedDictionary] Explicitly mention in documentation that keys/values are ordered by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#275 - \[HashCollections] Ensure `self` doesn’t get destroyed before we’re done working with it by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#276 - Remove obsolete swift(>=5.5)/swift(>=5.6) checks by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#277 - \[Xcode] Update Xcode project by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#278 - Minor rope updates by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#279 - \[Rope] remove(at:): Fix assertion when removing the last item creates a deficiency by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#280 - Merge changes from release 1.0 to release 1.1 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#283 - \[CollectionUtilities] Silence a warning on 32 bit platforms by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#286 - \[BitSet] Fix a thinko in BitSet.isEqualSet by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#287 - Grab bag of fixes for small test issues by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#288 - \[Xcode] Set a code sign identity in the Xcode project by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#285 - Rope: Fix trap when replaceSubrange is called on an empty rope by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#290 - `Rope.find` returns a bogus remainder for the end position by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#291 - Update TreeSet.md by [@hassila](https://togithub.com/hassila) in [apple/swift-collections#297 - Fix CustomStringConvertible/CustomDebugStringConvertible conformances by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#302 - \[RopeModule] Fix issues in Swift's ABI stable dialect by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#318 - \[CMake, Xcode] Update configurations for alternate build systems by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#319 - \[RopeModule] Remove unnecessary typealiases by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#320 - \[Heap] Improve type-level doc comment by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#326 - \[Heap] insert(contentsOf:) Switch to Floyd’s if we’re inserting too many items by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#327 - \[1.1] build: support building in Debug mode on Windows by [@compnerd](https://togithub.com/compnerd) in [apple/swift-collections#336 - Merge release/1.0 to release/1.1 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#348 - \[Heap] Convert min() and max() to properties by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#328 - \[BitArray] Disable bitwise operators for now by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#353 - \[Heap] Final(?) Heap API adjustments for 1.1 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#354 - \[Benchmarks] Add Collection Equality Benchmarks by [@vanvoorden](https://togithub.com/vanvoorden) in [apple/swift-collections#351 - Release preparations for 1.1 by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#355 - \[1.1] Fix typos by [@lorentey](https://togithub.com/lorentey) in [apple/swift-collections#357 - \[TreeDictionary]\[Keys] Add Equatable and Hashable Conformance to TreeDictionary.Keys by [@vanvoorden](https://togithub.com/vanvoorden) in [apple/swift-collections#352 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://togithub.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://togithub.com/renovatebot/renovate).  Co-authored-by: Self-hosted Renovate Bot <361546+cgrindel-self-hosted-renovate[bot]@users.noreply.github.enterprise.com>

lorentey added 8 commits September 23, 2022 11:15

[PersistentCollections] Remove dead code

8c9ae08

[PersistentDictionary] Add index striding benchmark

9407eff

[tests] Delay checking Indices until we’re done with the collection i…

a453335

…tself Also, don’t check `DefaultIndices` — it’s fine to assume it implements Collection as long as the base collection is correct.

[PersistentCollections] Small path-related adjustments

eefef1e

[PersistentCollections] Implement structural index(_:offsetBy:) & ind…

77f205f

…ex(_:offsetBy:limitedBy:)

[PersistentCollections] Fix indexing oversight when spawning a new ch…

287c677

…ild in a non-unique node Oops, this did not account for reversed item order, so it was spawning the new child with the wrong item.

[PersistentCollections] Cache the item hash within collision nodes

9150bbf

This will simplify merge operations as we no longer have to spend effort on preventing repeated rehashings of items within collision nodes. Also, review and simplify node allocation and lookup code.

[PersistentCollections] Slightly optimize compressed path case in isS…

2193e79

…ubset/isDisjoint

lorentey requested a review from msteindorfer September 23, 2022 19:04

[PersistentDictionary] Add a quick&dirty memory benchmark

92d1d12

The new executable target `memory-benchmark` collects some memory usage statistics and outputs average memory efficiency for a range of sizes in CSV format. Bump required toolchain version for benchmarks to Swift 5.5.

[PersistentCollections] Optimize key-based lookup operations

d87f5fd

lorentey added 7 commits September 24, 2022 17:02

[PersistentSet] Add ExpressibleByArrayLiteral, Codable, Custom[Debug]…

00ae83d

…StringConvertible, CustomReflectable conformances

[PersistentDictionary] Add CustomReflectable conformance

1c6e5cb

[PersistentDictionary] Implement structural filter & compactMapValues

826e22c

Plus add tests covering these as well and mapValues.

[PersistentSet] Implement structural subtraction, intersection, filte…

c92b0fa

…ring (untested)

[PersistentSet] Implement (naive) union, symmetricDifference operatio…

590dfeb

…ns (untested)

[PersistentDictionary] Add a structural initializer that takes a Pers…

ba303dc

…istentSet and a value transform

msteindorfer approved these changes Sep 26, 2022

View reviewed changes

lorentey merged commit ac35613 into apple:release/1.1 Sep 26, 2022

lorentey added this to the 1.1.0 milestone Oct 10, 2022

lorentey added the HashTreeCollections Hash-array mapped prefix tree implementations label Nov 15, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Persistent collections updates (part 8) #184

Persistent collections updates (part 8) #184

lorentey commented Sep 23, 2022

lorentey commented Sep 23, 2022

lorentey commented Sep 23, 2022 •

edited

lorentey commented Sep 24, 2022

lorentey commented Sep 24, 2022

lorentey commented Sep 25, 2022 •

edited

lorentey commented Sep 25, 2022

lorentey commented Sep 25, 2022

msteindorfer left a comment

msteindorfer Sep 26, 2022

lorentey Sep 26, 2022

msteindorfer Sep 26, 2022

msteindorfer Sep 26, 2022

msteindorfer Sep 26, 2022

lorentey Sep 26, 2022

msteindorfer Sep 26, 2022

Persistent collections updates (part 8) #184

Persistent collections updates (part 8) #184

Conversation

lorentey commented Sep 23, 2022

Checklist

lorentey commented Sep 23, 2022

lorentey commented Sep 23, 2022 • edited

lorentey commented Sep 24, 2022

lorentey commented Sep 24, 2022

lorentey commented Sep 25, 2022 • edited

lorentey commented Sep 25, 2022

lorentey commented Sep 25, 2022

msteindorfer left a comment

Choose a reason for hiding this comment

msteindorfer Sep 26, 2022

Choose a reason for hiding this comment

lorentey Sep 26, 2022

Choose a reason for hiding this comment

msteindorfer Sep 26, 2022

Choose a reason for hiding this comment

msteindorfer Sep 26, 2022

Choose a reason for hiding this comment

msteindorfer Sep 26, 2022

Choose a reason for hiding this comment

lorentey Sep 26, 2022

Choose a reason for hiding this comment

msteindorfer Sep 26, 2022

Choose a reason for hiding this comment

lorentey commented Sep 23, 2022 •

edited

lorentey commented Sep 25, 2022 •

edited