Skip to content

Database Import

Jump to bottom
Michael edited this page Jul 17, 2026 · 1 revision

Database Import

The import feature loads existing torrent database dumps into your index through the web UI. A DHT crawl or Prowlarr search only sees new torrents going forward; import lets you seed the index with history -- the RARBG archive, Pirate Bay exports, magnetico DHT scrapes, and similar public dumps.

Open the Import page in the web UI (nav bar), choose a file, and follow the analyze -> configure -> import flow.

Supported formats

The format is auto-detected by inspecting the file, not by its extension:

Format Notes
CSV Auto-detects delimiter; hex or base64 info hashes
NDJSON Newline-delimited JSON, one torrent per line
SQL dump INSERT statements, including multi-line; hex and \x bytea hashes
SQLite Reads the .db file directly (the format the RARBG archive ships in)

Files are streamed and parsed in a single pass rather than loaded into memory, so import size is bounded by disk, not RAM.

How it works

  1. Analyze -- the file is uploaded and scanned; the UI reports the detected format and the content categories found.
  2. Configure -- you set a source name and pick which categories to keep.
  3. Import -- runs asynchronously in the background while the page polls for progress (imported / skipped counts). A single-job mutex prevents two imports running at once.

Imported torrents are tagged with the source name you choose (import-<name>) and are trim-protected by default, so DB Size Management will not clean them up as stale.

Configuration

Defaults are safe; override in config.yml only if needed:

db_import:
  max_upload_bytes: 26843545600   # 25 GB default; 0 disables the cap
  classify_delay_seconds: 30      # delay on classification jobs to throttle TMDB/OMDb
  classify_batch_size: 50         # info hashes per classification job

Note on large files: the default upload cap is 25 GB. A full Postgres pg_dump can exceed that and be silently truncated at the limit. Raise max_upload_bytes (or set it to 0) for very large dumps.

Bulk imports throttle their classification jobs (classify_delay_seconds) so a multi-million-row import does not exhaust the TMDB and OMDb rate limits.

API

Method Route Purpose
POST /api/import/analyze Upload + detect format and categories
POST /api/import/execute Start the async import
GET /api/import/status Poll progress

Finding dumps

The Import page links to the magnetico DHT dumps, the largest public source of raw info hashes (tens of millions per release). Other sources include the RARBG SQLite archive and various Pirate Bay CSV/SQL exports.

Clone this wiki locally