Downloads EN

Download Management

中文 · English | Part of the Usage Guide series

This guide covers GalleryVault's download management page, task queue controls, automatic self-healing, and ExHentai official zip archive downloads.

Downloads (#/downloads)

• Batch URL & GID/Token Enqueue: Paste one or more gallery URLs or gid/token lines (extra slashes around the URL are accepted). Enqueue pages can explicitly select original/resample (defaults to the global "Image quality" setting or resample if unspecified); Archive download opens the same GP/tier preview as Favorites (tier selected in popup). Task titles follow the Title display setting (English/Japanese) instead of gid xxx.
• Follow newer versions: if ExHentai marks the listing as replaced, the download switches to the new gid (max 5 hops). Only the replacement link after the banner is followed — Parent links are ignored. 404/deleted galleries fail without retry, with plain-language errors in the list and Telegram. Gallery-detail “download original for this copy” does not follow.
• Global pause: auto-triggers on ExHentai 302 anti-abuse challenge; the page toggle and Telegram /pause are the same switch (see Settings); after pause, no new pages are claimed; the current in-flight page finishes, claiming and scans stop; it survives restart; Bot matches the web pause. The yellow top bar stacks with the Cookie red bar.
• GP & image quota: the page header shows a cached GP balance and Image Limit (~30 min TTL); above ~80% the top banner warns you to pause (avoid 509).
• Lists download tasks with their status (waiting / downloading / success / failed / cancelled), filterable by status.
• A channel badge next to each task title marks how it downloads: archive tasks show "Archive · Original/Resample" (or "Fallback pages" if an archive failure falls back to page-by-page), plain H@H page-by-page downloads show "Page-by-page · Original/Resample" (tasks without quality or legacy tasks without backfill still show "Page-by-page"; retries inherit the existing quality), making both channel and quality instantly recognizable.
• Active tasks show a live progress bar (current/total + percentage, plus speed and ETA while transferring); the list auto-refreshes every 2 seconds. Successful tasks that entered cold archive show archive ok / pending / fail.
• Persisted fallback flag & composite index: When an archive download falls back to page-by-page fetching, the state is persisted directly in the database via the archive_fallback column. This eliminates N+1 filesystem probe overhead (.archive.json checking) when listing download tasks. Additionally, the composite index idx_download_tasks_status_id accelerates paginated queue queries and status filtering under high-volume task queues.
• Retries are resumable & integrity checked: only missing/failed pages are fetched; pages already on disk are skipped. Files are verified against image magic headers upon disk write (supporting JPEG, PNG, WebP, and GIF animations with b"GIF8"), preventing HTML interception pages or corrupt files from entering the library.
• Failures self-heal & smart node rotation:
  • Smart H@H node rotation (skip_hath): during per-page retries or when encountering slow/failing nodes, the engine automatically extracts the failed node key and appends nl=<skip_hath_key> to request the next H@H node; if the page has no key, it falls back to parsing the HTML for a replacement node and syncs task state, preventing a single problematic node from stalling or backing off the entire gallery download;
  • Exponential backoff & 509 circuit break: transient errors retry automatically with an exponential backoff (30s → 2m → 8m → 30m → 1h → … up to 6h), retried up to 10 times before being marked failed; if ExHentai returns HTTP 509 (Image limits exceeded), the engine triggers an immediate task-level circuit breaker to abort retries and preserve quota; a periodic sweep automatically reactivates older failed tasks with remaining retry budget.
• Waiting and downloading tasks (both pending and downloading) can be cancelled (the worker will not write to disk once cancelled); failed/cancelled/successful tasks can be retried (individually or in bulk with checkboxes). Retry all tasks calls POST /api/downloads/retry-all, clearing backoff on pending / failed / cancelled and retrying immediately.
• Clear all successful: one click removes every success task record (the confirm dialog shows the count). This only clears the task list; ingested gallery files are not deleted. Failed, cancelled, and in-progress tasks are left alone.
• A finished download is ingested into the index immediately — the gallery row, pages and tags are written straight from the download result (cover thumbnail generated on first view), with no full library scan. The stored signature matches the scanner, so a later manual scan skips it.
• With Telegram configured, you get notified on download success/failure and scan completion; download notifications default to a summary digest (a bulk run collapses into one message), switchable to immediate / failures-only / off in Settings.

Archive Downloads (ExHentai archive)

The official zip channel packs the whole gallery into a zip file on the server (spending GP) and the client streams it on a single connection — far faster than per-page H@H fetches for large galleries. Three entries share one executor:

• The "Archive download selected" / "Archive update selected" buttons in the #/favorites/<favcat> and #/updates toolbars open a cost preview first (read-only, never charges GP): the current GP balance on top, then a row per gallery with original/resample cost and size, tiers that cost more than the balance marked red. Pick a tier and confirm to enqueue. Original = full-resolution originals; Resample = the server's fixed one-level resample (the default tier — cheaper in GP and bandwidth).
• "Download selected original" / "Update selected original" still download page-by-page but force original quality regardless of the global quality setting.
• Scheduled scan: with "Archive large favorites on scheduled scan" and a page threshold enabled in Settings, automatic favorites checks send galleries over the threshold through the archive channel (using the "Archive quality" tier) and keep the rest page-by-page (following the global "Image quality" setting or resample). Threshold 0 = everything archived.
• Archive tasks occupy a download_concurrency slot and share the same FIFO queue as page-by-page tasks; if the same gid already has a pending task, the archive button reports a skip.
• Reliability: the zip resumes via HTTP Range; quality + zip URL are persisted under .gv-{gid}/.archive.json, so a retry only resumes — it never re-packs or re-charges GP; a corrupt zip is deleted and re-packed; when the archive channel cannot serve the gallery (selected tier unavailable, insufficient GP, corrupt zip) the download falls back to page-by-page by default (no GP cost, H@H carries the traffic; persisted in the archive_fallback field with a "Fallback pages" badge) — disable "Fall back to page-by-page if archive is unavailable" in Settings to fail the task immediately instead, without burning automatic retries. On completion the archive goes through the same finishing pipeline as page-by-page downloads: .ehviewer / .galleryvault.json metadata, Telegram notification, immediate ingest, and old-version cleanup for gallery updates.