Configurable outbound User-Agent, 403 handling, queue hold status, sponsor segment category (2.94.11) - #714
Merged
ttlequals0 merged 14 commits intoSep 3, 2026
Conversation
The two outbound User-Agent strings become settings, editable in Settings > Data & Security > Outbound Requests and seeded by DOWNLOAD_USER_AGENT and FEED_USER_AGENT. They are separate because hosts disagree: bot mitigation on some CDNs refuses browser identifiers below a version floor that moves as new browsers ship, while some feed hosts serve only a declared podcast client. Resolution moves to a leaf module so rss_parser and upstream_chapters, which hold no database handle, can read them without a config-to-Database import cycle. The availability probe now reports a 403 as "CDN refused the request (403)" instead of folding it into "CDN not ready". The old wrapper prefixed every CDN failure with that same text, so the retry classifier could not tell a refusal from a not-yet-ready file and worked the full retry ladder against a host that answers the same way every time. A failed probe also logs the host it could not reach. GET /api/v1/status and the SSE stream gain a hold block, and the status bar now appears when the queue is holding work with nothing running. It keeps the two holds distinct: a rate-limit hold pauses the whole queue, an offline wait parks only the episodes waiting on that service. The offline tick persists the reachability probe it already performs so the status read never probes.
Importing user_agent before storage raised ImportError: utils.ttl_cache pulls in utils/__init__, which eagerly imports utils.audio, which imports storage, which imports user_agent. The cycle stayed hidden only because storage happens to load first today. The module now carries its own two-key TTL cache and imports nothing from the repo but config, with a regression test that imports it in a bare interpreter.
Two were real defects. The resolver took a literal fallback, so a missing settings row ignored DOWNLOAD_USER_AGENT / FEED_USER_AGENT entirely and used the compiled constant; it now goes through resolve_env_backed_default. And the just-in-time HEAD in routes.py probes the episode audio URL, so it belongs on the download UA rather than the feed UA it inherited: a host that 403s the application identifier would have refused the probe while the real download succeeded, which is the failure this release exists to fix. The rest is cleanup. offlineHeld counts every non-rate-limit deferral instead of summing a hardcoded service tuple, so it agrees with /settings/offline-queue for a service the block does not break out. hold_is_active replaces a second read of the same settings row and stops api/status reaching for a private helper. Probe-state decoding moves beside its writer in offline_queue, bool coercion goes through coerce_bool_setting, and the blank-value reset path calls db.reset_setting like every other registry-driven field. The hold block is now built in the SSE reader rather than the pipeline's broadcast thread, which must not block on SQLite, and its cache holds the lock across the rebuild so two readers on the same expiry boundary do not both run the queries. TTL raised to 15s to match state that only moves on the five-minute tick. The label-plus-Reset header shared by PromptField and the new section is extracted, the input recipe moves into fieldStyles, and the refusal wording is a constant shared with the retry classifier that matches on it.
The schema still described processing/episode/stage/progress/queue_depth, none of which the endpoint has returned since the status service moved to currentJob/queueLength/queuedEpisodes/feedRefreshes. This release documents GET /status as the way to check queue holds, so the spec has to match what the endpoint actually sends. Adds the QueueHold schema and points the SSE description at it instead of repeating a second stale list.
Reset no longer freezes the shipped User-Agent into a settings row. Blank now clears the row rather than calling reset_setting, which resolves the default at that moment and stores it; _resolve prefers a stored row, so a later image bumping the default past a CDN version floor, or an operator setting DOWNLOAD_USER_AGENT, would have been silently ignored on that install. The UI offered no way out either, since the field then reported itself as already default and disabled its own Reset. Clearing keeps the env-backed default live with no restart. The keys stay out of the refresh_default set: with no row to re-sync it buys nothing, and that set is guarded to keep upgrades from overwriting user-visible tunables. A service that is reachable at its last probe no longer renders as "not checked yet". Only a null verdict means unchecked, and a service can hold episodes again between a recovery probe and the following tick, which made the label contradict the timestamp beside it. A per-field Reset now drops only the field it sent, instead of clearing the whole draft and discarding an unsaved edit in the other field. The holdUntil example matches ISO_FORMAT, which is Z-suffixed.
Four advisories (GHSA-5jgf-p345-68v8, GHSA-f65p-4m7j-42xc, GHSA-fph4-wmhf-6fwf, GHSA-jqff-g426-hqxp) landed after the last release and fail the CI npm audit gate. Lockfile only, no package.json change: fast-uri is a build-time transitive dependency of workbox through ajv and never reaches the browser.
Owner
Author
Pre-existing test failures, for the recordThese fail identically on Python: 6 failures, all local-environment. The clearest is Frontend: 70 vitest failures. Both are worth fixing, neither belongs in a patch release about User-Agent handling. Happy to take them separately. |
Download and availability logs recorded the host alone, so ten consecutive failures left no record of what was being fetched or where the host was sending it. They now log the requested URL including its path, then each redirect hop with its status code, then the final URL. Query strings stay out by default. On a podcast enclosure that is where a signed CDN token or a per-listener tracking id lives, and a log outlives both, so including them unconditionally would put working credentials into every operator's log store. LOG_DOWNLOAD_QUERY and a matching toggle in Settings > Outbound Requests add them while debugging a refusal that depends on one.
ttlequals0
changed the title
Two detections naming the same sponsor within SHORT_GAP_THRESHOLD merged on that alone, with no look at what sat between them. On shows where the host name-drops a sponsor through the episode that absorbed the conversation. Across 45 merges in one sample, 26% of the merged span was gap rather than ad; a 1.9s and a 2.6s detection 83s apart became one 88s span that was 95% talk. Replaying eight of those merges through the fix returns 622s of show content to the episodes. The gap now has to be filler, measured in speech seconds by the same _content_duration_in_range that merge_ads_across_short_content_gaps already used. That pass ran second and could not undo a span this one had widened. Both now read one min_content_between_ads_seconds setting. The escape hatch that allows merging across real speech, "does the gap still mention the sponsor", could not refuse anything: get_transcript_text_for_range is inclusive at both ends, so it returned the two ads' own boundary segments along with the gap, and both ads name the sponsor. It now reads only segments lying inside the gap. A zero-duration detection is no longer a merge partner. It carries no ad audio, so merging with it only pushed the span end out across the gap. The own-site test pair kept its speech-filled gap on the old unconditional merge; it moves to an untranscribed gap so the shared token stays the only variable it is testing.
…e (2.94.9) A feed was promoted to calibrated on five episodes carrying a valid splice_evidence payload, without asking whether any of them held an event. A feed that never splices accumulated five empty payloads and calibrated, which armed the validator's zero-splice veto against evidence it cannot produce and held every cut of 60 seconds or more from the detector or a learned pattern. Such a feed now stays cold start, which the rule already documents as corroborating but never vetoing. That fix only covers feeds with no splice history at all. A feed with some splices stays calibrated and keeps vetoing, so the check is now overridable per feed: null inherits the global, true forces it on, false lets long cuts through. Follows the existing nullable-bool feed toggle, so the API serialization comes from _NULLABLE_BOOL_FIELDS and the resolver from _resolve_override. Help text across the feed and settings pages is shorter. Several fields explained mechanism, tradeoff and background where one sentence on what the setting does and when to change it was enough. Nine ran past 45 words, now two. The no-password security warning keeps its length deliberately: naming what an unprotected instance exposes is what makes it act. The splice check had no user-facing documentation at all despite being the most active hold rule, so it gains a docs section, a Held for Review bullet, and two glossary entries.
ttlequals0
changed the title
… (2.94.10) 2.94.9 also changed splice calibration so a feed with no splice events in its history stayed cold start. That altered behaviour for every feed at once to solve a problem on a few, and it carried a bug: the check counted only digital_silence and deep_silence events, while the detector also emits loudness_step and spectral_step. A feed that splices with hard cuts and no silence gap would have been demoted and lost a veto it should keep, letting long uncorroborated cuts through. The per-feed override already solves the same problem with no blast radius, so the calibration path returns to exactly what it was on main and the override stands alone. Docs and changelog now describe the check as it actually behaves, with the feed setting as the way out.
ttlequals0
changed the title
2.94.6 added flex-wrap to the status and sort selects to stop them overflowing a 320px screen. It worked, but the pair also wrapped at ordinary phone widths, putting each select on its own line under the Episodes heading. They now share the row and shrink instead: flex-1 with min-w-0 below sm, and natural width from sm up. Checked at 320, 412 and 1440. At 320 both sit on one row at 135px each with both labels readable and no horizontal overflow, which is what the original wrap was avoiding. Desktop is unchanged, with the heading and both selects on one row at content width.
…s (2.94.11) A sponsor can carry a segment category. Detection stamps it on LLM markers before the action-aware merge in both passes, pattern and fingerprint rows read it in place of their stored category, the pass-1 hint names the sponsor with it, and a learned pattern stores it. Re-categorizing a learned pattern never reached the model: auto-learned patterns contribute only their name to the hint, so a host's own product kept coming back as sponsor and each cut learned one more sponsor pattern. The existing category field holds an industry label and is unchanged. A download 403 is probed once more with the feed User-Agent. If accepted, the host gates on the browser identifier and the episode downloads with the feed string under a warning naming both. If both draw a 403 the block does not depend on the identifier, so the episode retries on the normal ladder. Same-sponsor merging rejoins detections when the gap reads as ad copy, and treats zero-length detections consistently. The poller waits for the queue lock to drop instead of backing off 30 s at every handoff. SQLite connections log slow lock waits and long-held write transactions with the opening statement. Redirect logging shows the resolved hop URL. The /status hold block serves the last good value while one thread rebuilds. The status bar lists rate-limited episodes after a pause lifts. The Processing Queue row keeps a gap before its Cancel button on phones.
ttlequals0
changed the title
ttlequals0
deleted the
fix/configurable-user-agent-and-queue-hold-status
branch
September 3, 2026 20:52
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
An episode failed ten times with a 403 that read as "CDN not ready". The file was fine; the host was refusing our pinned Chrome 120 User-Agent, which sat below a moving bot-mitigation version floor. The failure was retried as transient, and nothing logged the URL.
What changed
DOWNLOAD_USER_AGENTandFEED_USER_AGENT. A refusing host is now a settings change, not a release. Values are printable ASCII, single line, 512 characters max.LOG_DOWNLOAD_QUERYis set, since an enclosure query string carries a CDN token or tracking id.GET /api/v1/statusand every SSE frame carry aholdblock: rate-limit pause versus offline wait, reset time, and episodes held. The status bar now appears when the queue is holding work with nothing running.spliceVetoEnabled, Advanced on the feed page): null inherits the global, true and false force it. Feeds whose ads are spoken straight through no longer hold every long cut.segment_categoryonPOSTandPUT /api/v1/sponsors). Every read naming that sponsor or its aliases is filed under it, in both detection passes and in learned patterns. This fixes auto-learnedsponsorpatterns never reaching the model.Also in this release
fast-uri3.1.5 to 3.1.7 andbrowserslist4.28.2 to 4.28.8. Both are build-time transitive dependencies that never reach the browser, and both were failing the CI audit gate.category: "sponsor"(196 of 197);CONTRIBUTING.mddocuments the field, and the validation workflow installs its dependencies.openapi.yamlcorrectsProcessingStatusand addsQueueHold.Verification
Backend and frontend suites, ruff, tsc, eslint and the production build sit at the main baseline; pre-existing failures fail identically on
main. New coverage for User-Agent validation and precedence, the 403/404 split, the hold payload, the merge gates, and the sponsor segment category. The settings section and status bar were checked against the design guide at desktop and 390px.