Skip to content

0.16.0

Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 19 Aug 10:18
· 1 commit to main since this release
0.16.0
a03d0f1
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

0.16.0

The release that lands the NodeStore migration. Symbol storage and the declaration model move off retained class Node trees onto flat arena storage, every remaining holder that kept trees alive is paid down, and diagnostics stop being printed by the library. Steady-state memory in a long-lived host drops from 842 MB to 262 MB on the same load. This is a breaking release — the eager wrappers and trees that several public properties vended no longer exist.

  1. Symbol storage is a flat arena. SymbolIndexStore interns demangled symbols into a NodeStore (12 bytes per node, immutable after freeze()), SymbolTable holds 16-byte rows of canonical offset + packed name reference instead of 494k interned Strings, and single-element row buckets inline rather than each paying a heap allocation.
  2. The declaration model retains descriptors, not wrappers. TypeDefinition / ExtensionDefinition / ProtocolDefinition materialize their Mach-O wrapper on demand; instance sizes go 1272 → 384 B, 640 → 224 B, 440 → 384 B.
  3. Degradation reporting goes through events. printRoot() renders per definition, failures dispatch .definitionPrintFailed / .renderingDegraded, and library code no longer writes to a process stream at all — the host picks the sink (GUI → os_log, CLI → stderr). Closes #102.
  4. Legacy LC_DYLD_INFO binaries parse. Pre-macOS 12 / iOS 16 binaries carry no chained fixups, so every external reference in them used to be misread as a bare pointer. Their interface output goes from nearly empty to complete (SwiftUI on the iOS 15.5 simulator: 139 → 81,157 lines).
  5. Four silent wrong-result paths are fixed — opaque-type rewriting printing illegal Swift, a multi-payload enum cache falling back to a wrong layout after one bad record, reference storage over class-bound existentials sized one word too narrow, and dyld-cache image matches taken first-hit rather than ranked.
  6. Breaking API changes, all in the same direction: Symbol.nlist → Symbol.isExternal, the three *Name types' node is a NodeReference (and they lose Codable), the three definition types' wrapper properties become materialized*(in:), and ConsoleEventHandler writes to stderr.
  7. Dependencies move up — swift-demangling to 0.6.0 ..< 0.7.0 and MachOObjCSection to 0.8.105 ..< 0.9.0.

Rendered output for modern binaries is byte-for-byte unchanged, verified pair by pair (see Testing and CI). The two intended differences are item 4 (legacy binaries render far more) and item 5's opaque-type fix (21 sites in SwiftUI stop printing illegal Swift).

Highlights

842 MB → 262 MB

Measured in RuntimeViewer indexing five system images (Foundation + libswiftCore + AppKit + SwiftUI + SwiftUICore), same load throughout:

Stage Steady state
0.15.2 baseline 842 MB
NodeStore migration + follow-ups 470–480 MB → ~450 MB
Evolution 0001 — symbol-name offsetization 322 MB
Evolutions 0002 + 0003 262 MB

Live class Node instances 208,809 → 44; NodeStore instances 14,451 → 15; the parsing cluster 33.4 → 3.3 MiB; transient indexing peak 808 → 613 MB.

Indexing got cheaper and faster, not one at the other's expense: the build-time increment went 272 MB → 68 MB while running 14% quicker than the old pipeline, and the symbol sweep went 1317 ms → 701 ms per 100k symbols.

Symbol storage

  • SymbolIndexStore.Storage holds a NodeStore arena, immutable after freeze() and therefore lock-free Sendable. The build sweep demangles into a transient tree, runs the existing classification on it, and interns into the arena — the process-wide NodeCache, which never shrank, is no longer touched.
  • SymbolTable (evolution 0001) is a 16-byte SymbolRow of canonical offset + packed name reference. Image rows point straight at the mapped LINKEDIT string table (zero copy, clean pages); file rows and export-trie names go into one private contiguous buffer. Name lookup is a byte-level binary search over a name-ordered permutation, replacing the by-name dictionary. 494k interned Strings (68.7 MiB) disappear.
  • SymbolRowBucket (evolution 0003) inlines the single-element case of the four [UInt32] row-number buckets — 87.6% of buckets on the fixture. The downstream cluster goes 38.8 → 7.2 MiB.
  • Symbol drops its nlist existential (64 → 32 B) and DemangledSymbol compacts to 32 B.

Declaration model

  • Evolution 0002: the three definition types retain a descriptor reference and materialize on demand via materializedTypeContext(in:) / materializedProtocolConformance(in:) / materializedProtocol(in:) — at most once per operation, never cached back. DeclarationModelInstanceSizeTests pins the sizes.
  • Definition node fields hold NodeReference, and MetadataReaderCache's three Node-tree dictionaries were retired onto references too. MetadataReader.removeCache(for:) wires into SwiftDeclarationIndexer.deinit, so that cache is no longer append-only.
  • Cross-store lookups go through StructuralNodeReferenceKey. NodeReference's intrinsic Hashable is store identity, so a structurally equal key from another store silently misses — which is what once dropped override keywords and vtable-offset comments in bulk. The rule is now written down in AGENTS.md.

Diagnostics — the library stops choosing where output lands

The previous rounds' fix for "one bad definition kills the whole block" printed to a process stream from library code. That is not a choice a library can make correctly: os_log is invisible to a CLI (empty terminal, empty 2> redirect, empty CI log — exactly the case #102 reported), and stderr is meaningless to a GUI host.

  • Failures and degradations dispatch as events; the host attaches the handler that matches it. Dispatcher falls back to os_log when no handler is attached, so a silent build is not the failure mode either.
  • ConsoleEventHandler — the CLI's default sink — moved from stdout to stderr, where it stops interleaving with the product output.
  • The interim FileHandle.standardError.write(_:) calls were themselves a bug: that overload is ObjC-bridged and raises NSFileHandleOperationException, which Swift cannot catch, so 14 sites could abort the host process. All are gone, and a source-scanning test keeps them out.
  • The diff renderer's header resolution became three-state (absent / rendered / failed), fixing a case where one side failing deleted the declaration from both sides.

Correctness fixes found along the way

  • Legacy LC_DYLD_INFO bind support. resolveBind(fileOffset:) only understood chained fixups. iOS 15.5 simulator interfaces: Combine 10 → 6,907 lines, WidgetKit 17 → 2,795, SwiftUI 139 → 81,157, parse errors to zero. The opcode decoder is bounded against hostile input (a repeat count could otherwise hang or wrap).
  • Opaque-type rewriting substituted a generic parameter with its own depth literal, printing illegal Swift (SwiftUI.StaticIf<A1, 1, C1>, 21 sites in SwiftUI). Present since 2025-12-16.
  • Multi-payload enum cache: a loop-level catch bailed on the first bad record, after which every later multi-payload enum silently fell back to calculateTaggedMultiPayload — a wrong layout rather than a missing one, remembered by SharedCache for the rest of the round.
  • Reference storage over class-bound existentials, dyld-cache image ranking across sub-caches, PackedNameReference no longer trapping on binary-supplied name geometry, per-image cache eviction handed to an image's last live indexer (with both TOCTOU windows closed), and several subclass-map / conformance-provider paths that dropped results silently.

Compatibility

Breaking source changes. Each has a direct replacement:

Before After
Symbol.nlist Symbol.isExternal
Symbol.init(offset:name:nlist:) Symbol.init(offset:name:isExternal:)
TypeName.node / ProtocolName.node / ExtensionName.node, and ExtensionDefinition.genericSignature, typed Node typed NodeReference — materialize through the owning store when a tree is needed
Codable on those three *Name types Removed. A mangled symbol is already the tree's serialization, and upstream dropped Node: Codable in 0.5.0
TypeDefinition.type (an eagerly parsed TypeContextWrapper) materializedTypeContext(in:)
ExtensionDefinition.protocolConformance materializedProtocolConformance(in:)
ProtocolDefinition.protocol materializedProtocol(in:)
TypeDefinition.parentContext, the ParentContext type Removed — it was an index-time carrier held as if it were a permanent field
ConsoleEventHandler writing to stdout Writes to stderr

Additions:

  • SwiftIndexEvents.Dispatcher, injectable and shared by indexer and printer, so a host attaches its handlers once and covers both.
  • SwiftIndexEvents.Payload.definitionPrintFailed / .renderingDegraded, carrying the failing definition's name and the seam that gave way.
  • MetadataReader.removeCache(for:) and SwiftDeclarationIndexer.removeSubIndexer(_:) / removeSubIndexer(at:).
  • StructuralNodeReferenceKey, promoted to @_spi(Internals) public.
  • MachOTestingSupport gains SwiftIndexEventCollector and ExclusiveImageAccess (a TestScoping trait for suites that need a fixture image to themselves).

Pinned dependencies for this release — the first two lines differ from 0.15.2:

  • swift-demangling → 0.6.0 ..< 0.7.0. This line carries NodeStore / SharedNodeStore, the reshaped NodePrinterTarget (@autoclosure parameters, writtenUnitCount), the split node factories, and the removal of Node: Codable — all of which this release adopts.
  • MachOObjCSection → 0.8.105 ..< 0.9.0.
  • MachOKit → 0.52.101 ..< 0.53.0, unchanged and for the reasons given in 0.15.2.
  • MachOKitExtensions → from: "0.1.1", unchanged.
  • swift-semantic-string → from: "0.3.0", unchanged.

SwiftDeclaration picks up FoundationToolbox (FrameworkToolbox) for @Loggable / #log, which supplies the #available fallback this package needs below os.Logger's macOS 11.

No snapshot formatVersion change.

Testing and CI

  • Full suite green at 1,433 tests / 270 suites (--skip IntegrationTests), up from 1,315 in early August.
  • Output parity is the hard constraint of the migration: Scripts/run-rendering-ab-verification.py diffs main against this work byte for byte across three reader paths (MachOFile / DyldCache / MachOImage) and both commands. 78 pairs at the migration checkpoint, 96 pairs after the MetadataReaderCache retirement, 7 pairs per evolution — all identical. The script is new here, self-tested, and fails on zero compared pairs or mismatched skip markers.
  • Regression suites written to fail first in each case: SymbolTableRetentionTests, SymbolTableEquivalenceTests, LegacyDyldInfoBindTests, PerImageCacheEvictionTests, DiffRendererHeaderFailureTests, PrintFailureEventTests, DeclarationModelInstanceSizeTests, OpaqueTypeGenericParameterSubstitutionTests, MultiPayloadEnumDescriptorCacheTests, SubclassMapMaterializationFailureTests, plus NodeStoreMigrationInvariantTests as a source scan.
  • MachOSymbolsTests is re-enabled in Package.swift.

Documentation

Requirements

  • Swift 6.2+
  • Xcode 26.0+ (CI validates the test matrix on Xcode 26.6 / macOS 26)
Assets 3
Loading