Repository navigation
A record's state, in one place, and a way to clear it #48
Closed
robert-dean
started this conversation in
Ideas
Replies: 2 comments
|
Status check, 2026-09-22 (main at 23b983e) All three phases are built (592bafd): |
0 replies
|
Closing: all three phases are built, and the "What is left" list moved to #451. |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Status: built, 2026-08-16, all three phases. What each one turned out to be is below its heading,
and what is left is at the bottom.
Written: 2026-08-15, from the operator's own ask: a way to see all the track data in the system
(cached, analyzed, enriched, and so on) plus the ability to clear those.
State of the tree. Everything a record accumulates already exists as rows. What does not exist is
anywhere to look at them together, and any way to throw one away short of a
psqldelete:deadair.tracksGET /catalog/tracks, andGET /catalog/tracks/{id}, which this file called for and which now existsdeadair.track_sources(playable,missing_at,origin, bitrate, format,last_seen_at)deadair.track_audioper BINDING (checksum,byte_size,fetched_at,attempts,last_error,next_attempt_at)deadair.track_analysis(schema_version,complete,analyzed_at,failed_at,failure_reason)deadair.track_enrichment, one row per providerGET /catalog/tracks/{id}/enrichmenttracks.rating(plus the album's and the artist's)PUT /catalog/tracks/{id}/ratingdeadair.play_historyGET /activity, by time rather than by recordSo of seven axes the console can show two. The gap is worst exactly where a record fails: a track
that will not air is silent about which of the three reasons applies (every binding benched, the
bytes unfetchable and backed off, or nothing wrong at all and it is simply inside the repeat window),
and the operator's only recourse is the database.
Why it earns a page rather than a query
The specific loop this closes is the one already written up twice from the other end.
provider-audio-failures is an hour spent finding out that 13 of 581 records were
measured, and analysis-queue-ordering argues about which records the walk should pick
next — both of those are questions about the DISTRIBUTION of these columns, and neither is answerable
without writing SQL. The station also now benches a copy after four consecutive fetch failures
(
TracksRepository.markBindingMissing) and writes off a record that cannot be served(
DirectorService.thin), so the number of ways a perfectly good-looking catalog row quietly cannotair has grown, while the number of ways to see it has stayed at zero.
The clearing half is the other reason. Three of these rows are caches of something recomputable, and
track-cache-eviction already establishes the pattern: deleting a cached file is a
legitimate operator move, and
TrackAudioService.locatetreats a row whose file is gone as are-fetch and repairs the row on the way through. Doing that from a page is the same action with a
button on it.
What to build
Phase 1: a track detail read.
GET /catalog/tracks/{id}, and aTrackDetailcontract thatcarries the summary plus three arrays/objects the list cannot afford per row: bindings (from
track_sources, each with itstrack_audiostate joined on), the analysis row, and the enrichmentproviders with their
fetched_at.platform.view, like the rest ofcatalog.ck. This is worthhaving on its own: it is the page a console links to from a lineup item, from the activity feed and
from search.
Phase 2: a library-wide state column set.
GET /catalog/tracksgrows an optional projection (ora second operation, decided against how wide
Trackis allowed to get) carrying three booleans andnothing else per row: has local audio, measured at the current schema version, enriched by at least
one provider. Plus a
statefilter onCatalogQueryso "show me everything unmeasured" and "show meeverything benched" are one request. The counts belong beside it, because the aggregate is the thing
an operator actually reads first: N of M cached, N of M measured, N benched, N failing.
Phase 3: clearing, one verb per cache. All
platform.manage, all on the track (or the bindingwhere the row is per binding), and each one is a delete of derived rows and nothing else:
DELETE /catalog/tracks/{id}/audio— drop thetrack_audiorows for this record's bindings andthe files behind them. Must go through
TrackAudioServicerather than the repository, because thein-memory
inFlightmap and the eviction rules in track-cache-eviction live there: neverclear a record inside the committable window, since the director commits on audio being present
and pulling the file out from under a committed item produces the silence the commit gate exists to
prevent.
DELETE /catalog/tracks/{id}/analysis— drop thetrack_analysisrow so the walk picks it upagain. This is the cheapest of the three to get right and the most useful, since a detector that
was wrong about a class of records is exactly what
analyzer_plugin_idwas added to attribute.DELETE /catalog/tracks/{id}/enrichment— drop thetrack_enrichmentrows, optionally for oneprovider. Note this must not touch
tracks.mbid, which is identity rather than enrichment outputeven though enrichment is what resolved it.
next_attempt_at/attemptsreset, plus clearingtrack_sources.missing_at, is "try this againnow" and is what an operator wants after fixing an upstream. Today the only thing that clears
missing_atis the hourly sync re-sighting the copy.Rules it has to hold
Deleting a
tracksrow is a catalog operation with merge semantics behind it and is a differentfeature; if it lands, it lands separately and says so.
station_events.actor_id, so a cleared cache belongs on the activity feed for the same reasonplugin.*andairMode.setare there: it explains a re-fetch or a re-measure that would otherwiseread as the station churning for no reason.
track_audiorow with a null checksum is aremembered failure and the page's job is to show it with its
last_errorand its next attempttime. The same for
track_analysis.failure_reason. A page that showed only successes would be aworse version of the list that already exists.
completeis notanalyzed_at. A measurement of a truncated download is confident and wrong(
0005_music.sqlsays so at length), so the page must show the two separately and the "measured"count must filter on
complete, matching what every reader insrc/already does.are two files with different loudness and different cue points, and collapsing them on the page
would make "clear the audio" ambiguous about which file it took.
What is deliberately not proposed
A page per axis. The whole complaint is that these facts are scattered; four new console pages
would be the same scatter with a nicer font.
A bulk clear over the whole library.
rebuild:dataand the analysis walk already cover "startagain with everything", and a button that drops 581 files is a support call rather than a feature. A
filtered list with a per-row action is the version an operator can undo by waiting.
Storing anything new. Every column this needs exists, with one exception already argued for
elsewhere:
last_served_atontrack_audio, whichtrack-cache-eviction wants for LRU and which this page would happily
show. It belongs to that file, not this one.
What it turned out to be
All three phases landed on 2026-08-16, in the order this file set out, behind the eviction work in
track-cache-eviction — which had to go first because
last_served_atcould not be retrofitted, and which this page then got to show.
Phase 1, the detail read, is
GET /catalog/tracks/{id}and/catalog/tracks/$trackId. Onechange from the plan: enrichment is NOT part of it.
GET /catalog/tracks/{id}/enrichmentalreadyanswers every provider's payload and the station's own sourced claims, and the page calls it
alongside and renders the existing panel — so the facts, with their quotes and their source links,
arrived for free and there is one enrichment shape rather than two.
The rules held. Per binding throughout; failures reported with their last error and next attempt;
completeshown separately fromanalyzedAt. One rule had to be added after a live row broke it:held is decided by the BYTES, not by
fetched_at, because the eviction sweep clears the filecolumns and keeps the row, so a record fetched once and dropped since is not on this machine however
recently it arrived. It reads as
Dropped, which is neither held nor a fault.Phase 2, the library-wide columns, came out as three booleans per row (
hasAudio,measured,enriched), astatefilter and a count strip. Two decisions worth keeping:measuredmeanscomplete AND at a schema version the station still trusts, and
benchedrequires a copy to EXISTfirst — a record nobody has a copy of reads as uncached, because "the lookup never resolved" and
"every copy written off" are different problems. The counts honour the search and deliberately ignore
the state filter, since they are what a filter is chosen from.
Against this install on the day: 919 records, 345 cached, 216 measured, 902 enriched, none benched,
four failing. The four were all
upstream answered 502from one provider, which is the sort ofpattern that was invisible before.
Phase 3, the clears, are the four verbs this file named. The audio clear reuses the sweep's own
deletion path rather than growing a second one, which is what stops the two disagreeing about a file
two bindings share, and it REFUSES with a 409 inside the committable window rather than queueing —
the operator is standing there, and what they do about it is theirs to decide. Every clear is an
operator event with
actor_id.One thing this file did not say and the code now does: clearing enrichment leaves
deadair.factsalone. That is a decision rather than an oversight, and it is the one fact-enrichment records
as not yet made — a claim is the station's own argument with its evidence attached, and deleting one
from the console needs a permission rule and an answer about what re-extraction does to a claim an
operator has touched. The confirmation says so before the button is pressed.
What is left
records is one page each. The filtered list is where an operator would want it, and the reason it
is not there yet is the same reason the bulk clear was refused above: a per-row action over a
filtered set of 703 is a bulk clear wearing a hat.
analyzer_plugin_idis shown and nothing filters by it, so "re-measure everything thatdetector touched" is still a psql query — which is the exact shape of the problem this page was
built to end, one level up.
Moved from
docs/todo/track-state-console.md. The file as it last stood: https://github.com/robert-dean/deadair/blob/71e431d45ce7b35a189c7d226832ec4a6247e9dc/docs/todo/track-state-console.mdAll reactions