SIDFlow 0.8.0 - HVSC full export

Portable SID correlation export of the High Voltage SID Collection, generated by SIDFlow.

SIDFlow	0.8.0
corpus	hvsc
HVSC release	85
profile	full
tracks / files	87,868 / 61,157
similarity vector	58 dimensions, rank-uniform normalised
similarity metric	weighted cosine, weights published in the manifest
precomputed neighbours per track	25
SID emulation	sidlite
feature schema	1.5.0
schemas	sidcorr-1, sidcorr-lite-1, sidcorr-tiny-1, sidcorr-features-1

What's Changed

No reclassification was run. Every artefact is derived deterministically from the
0.7.0 export, whose data is unchanged: 87,868 tracks, 2,196,700 neighbour rows, identical
vectors and ratings. What changed is what the export says about itself, how it assigns
stations, and what else is published beside it.

Which asset do I want?

Start with lite. It reproduces the full export's top-25 neighbours at R@25 = 0.987
and its favourite-seeded stations at 98% overlap, in 8 MB against 982 MB.

Asset	Bytes	For
sidcorr-hvsc-full-sidcorr-lite-1.sidcorr	8,118,973	recommendations, stations, similarity — the right default
sidcorr-hvsc-full-sidcorr-tiny-1.sidcorr	1,834,993	constrained clients; style filtering and neighbour expansion. No vectors; needs a local HVSC
sidcorr-hvsc-full-features-1.jsonl.gz (new)	75,933,721	deriving your own representation from raw feature records. Pair with lite
sidcorr-hvsc-full-sidcorr-1.sqlite.gz (new)	194,351,886	the full export, 5.05× smaller. Prefer this over the plain .sqlite
sidcorr-hvsc-full-sidcorr-1.sqlite	982,155,264	features_json in a database, or SQL access
hvsc-full-sidcorr-1-0.8.0.tar.gz	277,449,597	everything except .sqlite.gz, in one archive

Read vector_weights. It decides whether your results are correct.

Similarity is a weighted cosine. Until this release the 58-entry weight table was not
published anywhere, so a consumer who implemented the lite specification exactly computed
plain cosine and agreed with the authoritative neighbours on roughly half their results:

What the consumer computes	R@1	R@25
weighted cosine	0.983	0.988
plain cosine	0.478	0.505

40% station overlap against 98%. Both answers look like plausible recommendations, so
there is no way to notice from the outside.

The weight applies to the dot product and to both norms — equivalently, it is a plain
cosine over vectors whose components are scaled by √wᵢ. Read similarity_metric and
vector_weights from the manifest.

What changed

The manifest now describes the file it ships beside

file_checksums.sqlite_sha256 had never matched the published file, in any release:
the exporter hashed the database and then wrote that hash into it. SHA256SUMS was always
correct, so nothing broke — but anyone verifying via the manifest rejected every release.
Fixed; both now agree.

neighbor_row_count is measured rather than track_count × k. paths.* are basenames
rather than the build host's absolute layout. New: hvsc_version, similarity_metric,
vector_weights.

Station membership changed for most of the corpus

The tiny profile's nine style masks were computed by taking each track's three
highest-scoring personas unconditionally, which produced:

Persona	0.7.0	0.8.0
slow_ambient	46,652 (53.1%)	17,574 (20.0%)
nostalgic	45,562 (51.9%)	17,574 (20.0%)
melodic	45,496 (51.8%)	17,574 (20.0%)
experimental	42,495 (48.4%)	17,574 (20.0%)
fast_paced	41,648 (47.4%)	17,574 (20.0%)
era_explorer	29,116 (33.1%)	17,574 (20.0%)
deep_discovery	11,962 (13.6%)	17,574 (20.0%)
composer_focus	673 (0.8%)	17,574 (20.0%)
theme_hunter	0 (0.0%)	17,574 (20.0%)

A station that could never play anything, one that looped after 673 tracks, a 69× spread,
and 9,451 tracks carrying both fast_paced and slow_ambient — two stations a
listener experiences as opposites.

Each persona is now the top 20% of the corpus by its own score, with a hard gate that
fails the export rather than publishing a starved or indistinguishable station.
style_populations and style_population_policy are in the tiny manifest.

Bit assignments, style keys, labels and style_count = 9 are unchanged, and
binary_format_version is still 2 — so a client reading masks positionally gets corrected
stations with no code change. Verified: the rebuilt tiny bundle differs from 0.7.0's
only in its style-mask table (104,507 bytes); every md5_48 identity, per-file subsong
count, packed rating and neighbour record is byte-identical.

Digests

Every digest changes; update your pins. Two things worth noting:

• The lite bundle's bytes did not change at all — fe92bd57…a346cd is the same digest
  0.7.0 published. Only its manifest gained fields.
• The full export shrank, 1,013,977,088 → 982,155,264 bytes, as VACUUM reclaimed
  free pages during the manifest repair. Track and neighbour counts are unchanged.

Upgrading

Read the migration guide.
It covers two hops — the 0.5-era → 0.7.0 data change and the 0.7.0 → 0.8.0 contract
change — with dedicated u64deck and c64commander sections.

If you are on anything published before 2026-07-26, the short version is that that
export's similarity data was inert: 15 distinct vectors across 87,073 tracks, 91.4%
sharing the literal [3,3,3,3], ratings that never took the values 1 or 5, and
neighbor_row_count: 0. Any threshold tuned against it needs re-tuning.

Verifying

sha256sum -c SHA256SUMS

SHA256SUMS and the sidecar manifests agree from this release onward. To run SIDFlow's
own release gate against what you downloaded:

bun run verify:exports -- --exports <download-dir> --hvsc <your-hvsc>

Release naming

From this release the tag is the SIDFlow version that produced it. Earlier releases
used sidcorr-<corpus>-<profile>-<timestamp>; they are not retagged, so every existing
download URL still resolves and releases/latest/download/… is unaffected. The verified
mapping is in the sidflow-data README.

Assets: the sidcorr-1 SQLite export and its manifest, a gzipped copy of it, the derived
lite and tiny bundles with their manifests, the features sidecar with its manifest, and
SHA256SUMS. The tarball contains everything except the gzipped SQLite.