Skip to content

StorageScope v0.5.3 — Performance measurement slice

Choose a tag to compare

View all tags
@RasputinKaiser RasputinKaiser released this 22 Jun 05:33
v0.5.3
2f23c22

v0.5.3 — Performance measurement slice

Measurement-only release. No behavior changes, no scan-pipeline optimizations. This slice exists to validate (or invalidate) the bottleneck hypotheses in the new PLAN.md before any optimization commits. Premise: most proposed optimizations are waste unless the underlying bottleneck is real, so we measure first.

Signpost coverage added

Six new os_signpost spans wrap the gaps the v0.5.0 instrument (#81) left open. They show up under Instruments → Points of Interest → subsystem com.rasputinkaiser.StorageScope:

Signpost Wraps What it measures
verify_on_demand FileSystemScanner.verifySizeGroup(_:) Per-Verify-Now-call hash time, with items=N metadata
persist_on_demand OnDemandVerificationStore's background hashCache.persist() The JSON encode + disk write cost the main actor skips
cache_invalidate_derived ScanStore.invalidateDerivedCaches() Event density during cleanup-review interactions (annotated with active view)
cache_invalidate_items ScanStore.invalidateItemsCache() Same pattern, items-only cache (annotated with active view)
lookup_build StorageScan.buildItemLookup Eager [String: StorageItem] build cost at scan-complete; reports counts per contributing array
cleanup_candidates_build ScanAccumulator.cleanupCandidates(...) Sort + filter pass; closes the gap ScanBenchmarkReport.text already admits between totalDuration and duration

ScanBenchmark fixture scaling

StorageScopeBenchmark now accepts scaled-fixture flags so maintainers can capture baselines at 10k / 100k / 500k items:

# Still works (v0.5.0 curated 7-file fixture):
swift run StorageScopeBenchmark --synthetic

# New: scaled synthetic tree:
swift run StorageScopeBenchmark --synthetic --items 100000 --depth 8 --duplicates 0.05
  • --items <n> emits N files distributed across a directory tree of --depth <d> levels (~120 files per leaf directory).
  • --duplicates <0..1> is the fraction of items emitted as content-identical pairs/triples sharing a SHA-256, so the verify path actually fires.
  • --keep-fixture prints the fixture path so Instruments can be attached for a manual re-scan at scale.

Backward compat: --synthetic with no --items still builds the v0.5.0-era curated 7-file fixture, so build_and_run.sh --fixture-scan workflow is unchanged.

v0.5.2 baselines captured

Three baselines committed under docs/perf-baselines/v0.5.2/:

Fixture Items Total size Duration (wall) Phase total Peak memory
curated.txt 24 1.11 GB 0.01s 0.01s 16.5 MB
10k.txt 10,353 687.3 MB 6.90s 6.90s 52.8 MB
100k.txt 105,277 7.16 GB 185.51s 185.51s 271.8 MB

500k skipped on the dev machine: would need ~36 GB disk vs ~33 GB free. See docs/perf-baselines/v0.5.2/README.md for regeneration steps and the validation thresholds under PLAN.md §6.3 that gate which optimizations actually ship.

Validation

  • swift test — 90 tests across 10 suites pass
  • ./script/public_upload_audit.sh — passes
  • bash ./script/build_and_run.sh --verify — bundle launches cleanly
  • Scaled fixtures regenerate and benchmark cleanly at 10k and 100k items locally.
  • DMG: exports/StorageScope-0.5.3.dmg, SHA256 080e092b6222dd152d1958410dd27001071279acb6fc08fcf70121f46b8e3edf — hdiutil verify confirmed

What ships next

Per PLAN.md's decision rules (§6.3): each subsequent proposal's signpost must show ≥20% of totalDuration on the 100k fixture (at least 37s out of v0.5.3's 185.51s measured baseline) to be promoted to v0.5.4. If the suspected hotspots don't reach that threshold, the plan pauses and the next slice will be feature-driven, not perf-driven.

The roll forward to v0.5.4 (scan-pipeline wins) is queued but gated on the next profiling pass — likely candidates per §3.1 are the autoreleasepool in tight enumeration and the file-attribute-fetch batching, but neither ships until the new signposts confirm the cost.

Assets 4
Loading