Skip to content

SIDFlow 0.8.2 - HVSC full export

Latest
Latest

Choose a tag to compare

View all tags
@chrisgleissner chrisgleissner released this 30 Jul 16:17
0.8.2
6b6ed49

Portable SID correlation export of HVSC 85 + Update 85, generated by SIDFlow.
SIDFlow 0.8.2
tracks / files 87,868 / 61,157
profile full
similarity vector 58 dimensions, rank-uniform normalised
similarity metric weighted cosine
precomputed neighbours per track 25
SID emulation sidlite
feature schema 1.5.0
schemas sidcorr-1, sidcorr-lite-1, sidcorr-tiny-1, sidcorr-features-1

What's Changed

Only the tiny bundle changes in this release. The full export, compressed full export, lite bundle, and features sidecar are byte-identical to 0.8.1.

No reclassification was run. Asset names, track ordering, file identities, style masks, ratings, vectors, and full-export neighbour rankings remain unchanged.

Published data

Asset Content in 0.8.2 Version
sidcorr-hvsc-full-sidcorr-1.sqlite Byte-identical to 0.8.1 sidcorr-1, unchanged
sidcorr-hvsc-full-sidcorr-1.sqlite.gz Byte-identical to 0.8.1 compressed sidcorr-1, unchanged
sidcorr-hvsc-full-sidcorr-lite-1.sidcorr Byte-identical to 0.8.1 sidcorr-lite-1, unchanged
sidcorr-hvsc-full-sidcorr-tiny-1.sidcorr Neighbour table and graph_flags changed sidcorr-tiny-1, binary format 2
sidcorr-hvsc-full-features-1.jsonl.gz Byte-identical to 0.8.1 sidcorr-features-1, unchanged

The full and lite exports were rebuilt through the normal CLI and reproduced byte for byte. scripts/reproduce-published-bundles.sh performs this verification.

Tiny neighbour graph

The tiny graph is now a searchable proximity index rather than a directed acyclic graph.

  • graph_flags bit 0 now reads 0: acyclicity is no longer declared or guaranteed.
  • Bit 3 is retired.
  • Neighbour targets are no longer restricted to lower track ordinals.
  • Every track has three outgoing neighbours.
  • Slots 0 and 1 preserve the source export's two closest neighbours.
  • Slot 2 provides a more diverse connection for broader traversal.
  • Rows remain ordered by descending similarity.
Metric 0.8.1 0.8.2
mean out-degree 2.799 of 3 3.000 of 3
tracks with no incoming edge 24,669 (28.08%) 2 (0.002%)
tracks with no outgoing edge 2,786 0
largest undirected component 99.08% 99.995%
greedy routing recall@1 0.30% 0.80%
composer lift 84.536 74.772
nDCG@10 0.1440 0.1507

This is a graph-data and specification change, not a file-format change:

  • binary_format_version remains 2
  • the header layout is unchanged
  • section ordering is unchanged
  • track ordering is unchanged
  • neighbors_per_track remains 3
  • asset filenames are unchanged

Existing releases/latest/download/<name> URLs therefore continue to resolve.

File identities, per-file track counts, style masks, packed ratings, and station membership are unchanged. Only the graph used to move between tracks differs.

Migration

Full, compressed full, lite, and features consumers

No migration is required.

The payloads and schema versions are unchanged, so existing parsers, indexes, caches, and recommendation results remain valid.

Tiny consumers that only parse the format

Update the release tag and expected digest.

No parser change is required because the binary layout and version are unchanged.

Tiny consumers that validate or traverse the graph

Remove any assumption that:

  • the graph is acyclic
  • neighbour ordinals are lower than the source ordinal
  • traversal terminates because the graph eventually runs out of forward edges

Cycles are valid in a similarity graph. Maintain an explicit visited set to prevent repeats and bound traversal.

SID Radio clients

For long-running stations:

  1. Update the pinned release tag and tiny-bundle digest.
  2. Maintain an exclusion set for tracks already played.
  3. Deduplicate by SID file identity when multiple subsongs should count as one tune.
  4. Let the station query progress with recent listening history instead of repeatedly expanding from the original seed.

The new graph provides the connectivity. The progressing-query policy turns that connectivity into a station that can continue for hours.

Digests

Asset SHA256 Change from 0.8.1
sidcorr-hvsc-full-sidcorr-1.sqlite d3d825ae…b176da unchanged
sidcorr-hvsc-full-sidcorr-1.sqlite.gz 7eeddad8…b2fcf0 unchanged
sidcorr-hvsc-full-sidcorr-1.manifest.json 4afe8314…d9a741 unchanged
sidcorr-hvsc-full-features-1.jsonl.gz 006cd388…045f39b unchanged
sidcorr-hvsc-full-features-1.manifest.json 62667463…efc2c6 unchanged
sidcorr-hvsc-full-sidcorr-lite-1.sidcorr fe92bd57…a346cd unchanged
sidcorr-hvsc-full-sidcorr-lite-1.manifest.json 1953a666…b22279 generated_at only
sidcorr-hvsc-full-sidcorr-tiny-1.sidcorr 10db2838831bb2386f6dd19041a7ae8525878c401afa6eaf932d9b1610c1c0bc changed
sidcorr-hvsc-full-sidcorr-tiny-1.manifest.json ca6e8f7c…48e371f changed

Verification

Verify the downloaded assets with:

sha256sum -c SHA256SUMS

To run SIDFlow's release gate against the downloaded files:

bun run verify:exports -- --exports <download-dir> --hvsc <your-hvsc>

The release checks the published bytes directly, including graph flags, neighbour ordering, connectivity, dead ends, unreachable tracks, and traversal behaviour.

See the migration guide and graph-design document in chrisgleissner/sidflow:

  • doc/migration/0.8.0-to-0.8.2.md
  • doc/neighbour-graph-design.md
Assets 12
Loading