feat: source releases from Radarr/Sonarr, drop Jellyfin#7
Conversation
Jack now reads media availability directly from Radarr/Sonarr instead of
Jellyfin. Since a source and a destination are now the same kind of server,
the connector hierarchy collapses into a single capability-based connector.
- Config: replace `servers.{sources,destinations,peers}` and the global
`indexer` block with a flat `servers` array (radarr/sonarr, each with
`source`/`destination`/`autoregister:{enable,priority}`) plus a separate
`peers` array. `name` is now required on servers and peers.
- Connectors: `ArrServerConnector` implements both source and destination
roles, gated by `canSource`/`canDestination` and guarded by new
`@requiresSource`/`@requiresDestination` decorators (alongside
`@requireInitialization`). `PeerConnector` (was JackServerConnector) stays
source-only. The sources/ and destinations/ split is gone.
- New normalized `Release` type replaces Jellyfin's BaseItemDto across the
peer/torznab/blackhole flow, mirroring the *arr file metadata (scene/release
name, quality, size, languages, mediaInfo, ids). The absolute file path never
leaves the source jack.
- Torznab output carries the richer release info (imdbid/tmdbid/tvdbid,
season/episode, download/upload volume factors).
- schemas: generate Radarr/Sonarr API types via openapi-ts (specs committed),
drop the Jellyfin client.
- Tests + e2e + docs updated to the new model.
BREAKING CHANGE: the `servers`/`peers`/`indexer` config layout changed; see
README and examples/config.jsonc.
The --build flag was on the radarr/sonarr line (image-based, nothing to build), so the jack containers reused a stale cached image on repeat local runs. Move --build to the jack-alpha/jack-beta up so the e2e always runs the current code.
Radarr's /ping returns 200 before the v3 API finishes migrating, so the first /api/v3/movie call raced and returned 400 on the cold CI runner. Poll the authenticated /api/v3/system/status until it's ready before seeding.
system/status returns 200 while the DB-backed endpoints still 400 during startup. Poll the actual movie endpoint until it's ready, which also fetches the existing list.
Include the HTTP body in fetchJson errors and the Radarr readiness poll so CI logs show what Radarr actually returns (the 400 is reproducible only there).
9809e76 to
161eae4
Compare
The add-movie POST failed in CI with 'Root folder /media/movies does not exist': the fixtures weren't readable by Radarr's uid 1000 (CI runner checks them out as a different uid), so the swallowed rootfolder POST never took. chmod a+rX the fixtures (like the blackhole dirs) and add+verify the root folder before seeding instead of ignoring failures.
Radarr's FolderWritableValidator rejects a root folder that isn't writable by its uid-1000 user; a+rX only granted read. Use a+rwX on the fixtures.
Migration guide — upgrading to the Radarr/Sonarr versionThis release drops Jellyfin and sources media straight from Radarr/Sonarr. Migrating is a config + volumes change, plus coordinating with your peers. Follow the steps below. 1. Coordinate the upgrade with your peersThe 2. Rewrite your
|
jack streams from disk using the *arr's reported file path, which differs from the old Jellyfin library path. Add a callout + sharpen the compose comment so the media mount target matches the *arr container path (not a convenient one).
Wrap every request in an OTel server span via @hono/otel and bridge pino logs to the OTel Logs API in-process, both exported over OTLP/protobuf and enabled whenever an OTLP endpoint is configured. Logs carry the active span's trace/span ids for native correlation. The /ping healthcheck is excluded from both tracing and request logging. Span/log export is immediate (Simple processors), flushed on shutdown. Also: set up eslint at the workspace root and apply lint autofix across the repo.
This comment was marked as outdated.
This comment was marked as outdated.
ffc3196 to
54b978f
Compare
Add debug/trace logging through the torznab -> peer -> source search path (fan-out with per-peer/per-source counts, outgoing HTTP requests, non-2xx and schema-validation failures, capability/init guards) and include the response body in the request-completed trace log, to diagnose empty peer searches. Run with LOG_LEVEL=debug for the flow, trace for HTTP/response bodies. Also fix @requireInitialization: it awaited the isInitialized boolean (which resolves immediately) instead of the initialization promise, so guarded calls could run before init finished.
Connectors that failed to connect at boot were excluded from searches until a restart. init() is now an idempotent, retry-aware state machine: it re-pings only when never tried or the last attempt FAILED (logging that it is retrying the previously-down connector), and is a no-op while an attempt is in flight or once connected. Subclasses provide the check via runInit() instead of overriding init(). The search controllers (torznab fan-out, peer, items) drop the isInitialized pre-filter: every source/peer is attempted and re-initialized lazily by @requireInitialization, so one that comes back online rejoins searches without a restart. Each connector is isolated — a failure is logged and treated as zero results instead of failing the whole search. Add tests for the init retry state machine and the search resilience/recovery.
Add an AbortSignal.timeout to the shared ServerConnector.fetch (which all *arr and peer calls go through), so a hung host can't stall a search — important now that searches lazily re-ping connectors that were down. Default is 30s via the new HTTP_TIMEOUT_MS env, overridable per call with a timeoutMs option. Timeouts and network errors are logged distinctly. Add a test covering the timeout.
…h-otel GitGuardian flagged the base64 basic-auth header (example creds) baked into the compose. Make O2_PASS and OTLP_AUTH required env vars sourced from a gitignored .env (no committed defaults), and add a .env.example with placeholders and the command to generate the header.
httpie-style 'api' command (key==query, key=body, key:=rawjson, Header:value), plus 'peer search' and 'torznab search' subcommands. Reads JACK_URL and JACK_API_KEY from the env, sends X-Api-Key, and colorizes JSON output on a TTY (plain valid JSON when piped). Run via 'mise cli ...' (quiet).
54b978f to
a045f1e
Compare
Radarr (and torznab clients in general) query by the numeric IMDb id without the "tt" prefix (imdbid=0133093), but *arr stores it with the prefix (tt0133093). doSearchByImdbId did an exact string compare, so movie searches never matched and returned no releases. Normalize both sides (strip tt) via a new normalizeImdbId helper.
…n-out Make the torznab search match how Radarr/Sonarr actually query and stop listing peers' whole libraries for text searches: - Radarr: add tmdbid lookup via GET /api/v3/movie?tmdbId= (server-side filter, a targeted one-movie lookup instead of list-all). Prefer tmdbid over imdbid. - Sonarr: tvdbid already filters server-side; tmdbid/imdbid -> empty. - Peer protocol: drop the free-text 'q' param; search by imdbId/tmdbId/tvdbId (+season/ep), and no params = full catalog. No fuzzy title matching. - Torznab: id queries -> by-id; a 'q' term -> empty (not fanned out); no term -> catalog (powers RSS + the indexer self-test). Remove 'q' from the movie-search/tv-search caps so *arr only searches by id. Add doListReleases() for the catalog feed. Update tests; add tmdbid + empty-q coverage.
…mdbId) peer search no longer takes --q (the peer protocol is id-only now); add --tmdbId to both peer and torznab search, with torznab preferring tmdbid.
…peers) A peer on an older version doesn't understand the new tmdbId param, drops it, and its old fallback returns the whole catalog (movies + TV) — so a movie search came back with everything. The requesting jack now filters a peer's response by the id it asked for (imdb/tmdb/tvdb + season/ep), so an old or over-eager peer can't pollute results. Add a regression test.
Honor the torznab 'cat' query param: a release matches when its category's top-level bucket is requested (2xxx -> movies, 5xxx -> TV). Mainly affects the catalog/RSS feed (t=movie&cat=2000 now returns only movies, t=tvsearch&cat=5000 only TV); id searches already return the right category. Add a test.
The search refactor dropped free-text fan-out: t=search&q=<term> now returns empty and the peer protocol no longer takes q. Switch the e2e tests to the catalog (t=search with no term) and the id-only peer search, matching the new behavior. (Unrelated: the last e2e CI failure was an lscr.io image-pull timeout, not these tests.)
What
Drops the Jellyfin integration. Jack now reads media availability directly from Radarr/Sonarr. Because a source and a destination are now the same kind of server, the connector hierarchy collapses into a single capability-based connector.
This PR also grew a few adjacent things on top of the migration: id-based torznab search, OpenTelemetry traces + logs, connector resilience, and a small CLI. See the sections below.
Config (breaking)
servers.{sources,destinations,peers}+ the globalindexerblock are replaced by:servers— flat array of radarr/sonarr, each withsource/destination/autoregister: { enable, priority }(source/destination defaulttrue).peers— separate array of other jacks (sources only).nameis now required on both. SeeREADME.mdandexamples/config.jsonc.New optional env vars:
OTEL_EXPORTER_OTLP_ENDPOINT(+OTEL_SERVICE_NAME) to turn on telemetry, andHTTP_TIMEOUT_MS(default 30s) for outgoing connector requests.Architecture
ArrServerConnectorimplements both roles, gated bycanSource/canDestinationand guarded by new@requiresSource/@requiresDestinationdecorators (mirroring@requireInitialization). Radarr/Sonarr subclasses hold the movie- vs series/episode-specific queries.PeerConnector(wasJackServerConnector) stays source-only. Thesources/+destinations/split is gone.Releasetype replaces Jellyfin'sBaseItemDtoacross the peer → torznab → blackhole flow, mirroring the *arr file metadata (scene/release name, quality, size, languages, mediaInfo, imdb/tmdb/tvdb ids). The absolute file path is re-resolved on the source jack and never serialized.autoregister.enable+autoregister.priority) instead of the globalindexerflag.Search behaviour
Searches match how Radarr/Sonarr actually query, and avoid listing peers' whole libraries:
tmdbid(a targetedGET /api/v3/movie?tmdbId=lookup) then falls back toimdbid(compared with thettprefix normalized — Radarr queries without it). Sonarr usestvdbid(GET /api/v3/series?tvdbId=).t=search&q=<term>returns empty (the *arr always also searches by id). A query with no term returns the catalog, which powers the RSS feed and the indexer self-test. Caps advertises only id params formovie-search/tv-search, so the *arr stops sending text queries.catfiltering: results are filtered by the torznabcatparam (2xxx → movies, 5xxx → TV).tmdbIdand dumps its whole catalog) can't pollute results.Observability
@hono/otel); pino logs are bridged to the OTel Logs API in-process so they carry the active span's trace/span ids (native correlation). Both are exported over OTLP/protobuf, enabled whenever an OTLP endpoint is configured. The/pinghealthcheck is excluded from tracing and request logging.debug/tracelogging across the whole search flow (fan-out, per-peer/per-source counts, outgoing HTTP, non-2xx and schema failures).examples/compose-with-otel.yml— jack + OpenObserve (single image, no external DB) wired together; secrets via a gitignored.env.Resilience
init()is a retry-aware state machine: it re-pings only when never tried or the last attempt failed (logging the retry), and is a no-op while in flight or once connected.isInitialized— a connector that was down at boot is retried lazily and rejoins without a restart, and each connector is isolated (one failure → zero results, not a failed search).HTTP_TIMEOUT_MS).CLI
scripts/cli.ts+ amise clitask to talk to a running jack: an httpie-styleapicommand pluspeer search/torznab search(id-based,--cat). ReadsJACK_URL/JACK_API_KEY; colorizes JSON on a TTY (plain when piped).Schemas
Radarr/Sonarr API types generated via
@hey-api/openapi-ts(specs committed: Radarrmaster, Sonarrmain). Connectors useServerConnector.fetch(existing X-Api-Key auth) typed by the generated types. Jellyfin client removed.Tests
bun test apps packages): config, torznab (release→XML), the MSW integration suite (Radarr/Sonarr + peer releases, id search,catfilter, over-eager-peer filtering), and the connector init/retry/timeout state machine. 56 pass,tsc --noEmitclean.setup.tsseeds Radarr with the fixture movie via its API; configs use the newservers/peersshape; tests updated to the id-based / catalog search protocol.