Config file, auto-cover default, transcribe auto-detect, download resilience

CHANGELOG.md

@@ -4,6 +4,25 @@ All notable changes to this project will be documented in this file. The format
 
 ## [Unreleased]
 
+### Fixed
+
+- **`--auto-cover` for Dutch (and other) books no longer silently misses.** Open Library tags docs with ISO 639-2/B (e.g. `"dut"` for Dutch), while DAISY 2.02 metadata uses ISO 639-1 (`"nl"`); the previous literal `eq_ignore_ascii_case` dropped every plausible match. `dpub-meta` now treats 639-1, 639-2/B and 639-2/T as equivalent (`nl`/`dut`/`nld`, `fr`/`fre`/`fra`, `de`/`ger`/`deu`, etc.). Real-world miss this surfaced: "Het smelt" by Lize Spit. Regression test added.
+- **ISBN search hits are now trusted unconditionally.** When DAISY's `dc:identifier` is ISBN-shaped, the search-by-ISBN already disambiguates the edition, so the language and author filters on the result are noise — and would (incorrectly) reject the cover when Open Library lists a translator under `author_name`. Title+author search remains filtered.
+- **Open Library HTTP timeout raised from 8 s to 30 s.** `covers.openlibrary.org` redirects through archive.org and can take ~20 s on first hit for less-popular editions; 8 s caused spurious "lookup failed" misses.
+- **Whisper model download no longer times out on slow connections.** The HTTP agent used a 60-second total-request timeout, which was insufficient for the 1.5 GB `ggml-medium.bin` download. Now uses per-read timeouts (60 s idle) so downloads can take as long as needed as long as data keeps flowing. Additionally, downloads now retry up to 3 times on transient failures (CDN stalls, connection resets).
+
+### Changed
+
+- **Cover lookup now runs in the background** during conversion. The Open Library HTTP requests overlap with transcription and audio recompression instead of blocking the pipeline, so conversions with `--auto-cover` (now the default) no longer stall waiting for the network.
+
+### Added
+
+- **`--transcribe` auto-detects language from book metadata.** Passing `--transcribe` without a language code now reads `dc:language` from the DAISY NCC metadata and normalises it to ISO 639-1 for Whisper. Explicit `--transcribe nl` still works. Config file supports `"transcribe": true` for auto-detect or `"transcribe": "nl"` for a fixed default.
+- **Shared ISO 639 normaliser** (`dpub-util/lang`). Maps ISO 639-1, 639-2/B and 639-2/T codes to their canonical two-letter form. Used by both `dpub-meta` (cover lookup language filter) and `dpub-cli` (transcription auto-detect).
+- **Persistent config file** (`~/.config/dpub/config.json` on Unix, `%APPDATA%\dpub\config.json` on Windows). Lets users set defaults for `audio`, `bitrate`, `auto_cover`, `no_word_sync`, `rights`, `whisper_model`, `transcribe`, `validate`, `a11y`, `jobs`, and `log_level`. CLI flags always override config values.
+- **`dpub config` subcommand** — shows the config file path and contents. `--init` creates a starter file with all defaults; `--path` prints just the file path.
+- **`--auto-cover` is now on by default** for both `convert` and `batch`. Pass `--no-auto-cover` to opt out.
+
 ## [0.6.0] - 2026-05-07
 
 Word-level Media Overlay sync (karaoke-style highlight-along-with-audio in compatible reading systems) and a major first-run UX overhaul (`dpub doctor`, `dpub setup --whisper-model <size>`, auto-discovery, `scripts/build.sh`, optional `--install` for missing tools).

crates/dpub-cli/Cargo.toml

@@ -26,6 +26,7 @@ dpub-audio = { path = "../dpub-audio", version = "0.6.0" }
 dpub-core = { path = "../dpub-core", version = "0.6.0" }
 dpub-convert = { path = "../dpub-convert", version = "0.6.0" }
 dpub-meta = { path = "../dpub-meta", version = "0.6.0" }
+dpub-util = { path = "../dpub-util", version = "0.6.0" }
 dpub-validate = { path = "../dpub-validate", version = "0.6.0" }
 sha2 = { workspace = true }
 clap = { workspace = true }

crates/dpub-cli/src/config.rs

@@ -0,0 +1,182 @@
+use std::path::{Path, PathBuf};
+
+use serde::{Deserialize, Serialize};
+
+/// Transcription setting in the config file. Accepts:
+/// - `"nl"` / `"en"` / ... → always transcribe with this language
+/// - `true` → transcribe, auto-detect language from book metadata
+/// - `false` or `null` → do not transcribe
+#[derive(Debug, Clone, Deserialize, Serialize)]
+#[serde(untagged)]
+pub enum TranscribeSetting {
+    /// Auto-detect language from book metadata when `true`.
+    Auto(bool),
+    /// Explicit language code.
+    Language(String),
+}
+
+/// Persistent user defaults for dpub. Every field is optional — a missing
+/// key in the JSON simply falls through to the hard-coded default.
+///
+/// The canonical location is `~/.config/dpub/config.json` on Unix and
+/// `%APPDATA%\dpub\config.json` on Windows. CLI flags always override.
+#[derive(Debug, Default, Deserialize, Serialize)]
+#[serde(default)]
+pub struct DpubConfig {
+    /// Audio handling: `"original"` or `"opus"`.
+    pub audio: Option<String>,
+    /// Opus bitrate in kbit/s (32–96 for speech).
+    pub bitrate: Option<u32>,
+    /// Enable automatic cover lookup via Open Library.
+    pub auto_cover: Option<bool>,
+    /// Skip per-word Media Overlay sync (fall back to per-paragraph).
+    pub no_word_sync: Option<bool>,
+    /// Default rights statement for `<dc:rights>`.
+    pub rights: Option<String>,
+    /// Path to a `ggml-*.bin` Whisper model file.
+    pub whisper_model: Option<PathBuf>,
+    /// Transcription: `true` (auto-detect language), `"nl"` (explicit), or `null` (off).
+    pub transcribe: Option<TranscribeSetting>,
+    /// Run EPUBCheck after conversion.
+    pub validate: Option<bool>,
+    /// Run DAISY ACE after conversion.
+    pub a11y: Option<bool>,
+    /// Parallel batch job count (`0` = let rayon decide).
+    pub jobs: Option<usize>,
+    /// Default log level (`"error"`, `"warn"`, `"info"`, `"debug"`, `"trace"`).
+    pub log_level: Option<String>,
+}
+
+/// Return the platform-appropriate config directory for dpub.
+///
+/// - Unix: `$HOME/.config/dpub/`
+/// - Windows: `%APPDATA%\dpub\`
+pub fn config_dir() -> PathBuf {
+    if cfg!(target_os = "windows") {
+        let base = std::env::var_os("APPDATA")
+            .map_or_else(|| PathBuf::from("."), PathBuf::from);
+        base.join("dpub")
+    } else {
+        let home = std::env::var_os("HOME")
+            .map_or_else(|| PathBuf::from("."), PathBuf::from);
+        home.join(".config").join("dpub")
+    }
+}
+
+/// Full path to the config file.
+pub fn config_path() -> PathBuf {
+    config_dir().join("config.json")
+}
+
+/// Load the config file. Returns `Default` (all `None`) when the file is
+/// absent or unparseable — dpub should never fail to start because of a
+/// broken config.
+pub fn load() -> DpubConfig {
+    load_from(&config_path())
+}
+
+fn load_from(path: &Path) -> DpubConfig {
+    let bytes = match std::fs::read(path) {
+        Ok(b) => b,
+        Err(e) if e.kind() == std::io::ErrorKind::NotFound => return DpubConfig::default(),
+        Err(e) => {
+            tracing::warn!("could not read {}: {e}", path.display());
+            return DpubConfig::default();
+        }
+    };
+    match serde_json::from_slice(&bytes) {
+        Ok(cfg) => cfg,
+        Err(e) => {
+            tracing::warn!("ignoring {}: {e}", path.display());
+            DpubConfig::default()
+        }
+    }
+}
+
+/// Example JSON for `dpub config` output and `--init`.
+pub fn example_json() -> &'static str {
+    r#"{
+  "audio": "original",
+  "bitrate": 64,
+  "auto_cover": true,
+  "no_word_sync": false,
+  "rights": null,
+  "whisper_model": null,
+  "transcribe": null,
+  "validate": false,
+  "a11y": false,
+  "jobs": 0,
+  "log_level": "info"
+}"#
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn missing_file_returns_default() {
+        let cfg = load_from(std::path::Path::new("/tmp/dpub-test-nonexistent/config.json"));
+        assert!(cfg.audio.is_none());
+        assert!(cfg.bitrate.is_none());
+        assert!(cfg.auto_cover.is_none());
+    }
+
+    #[test]
+    fn partial_json_works() {
+        let dir = tempfile::tempdir().unwrap();
+        let path = dir.path().join("config.json");
+        std::fs::write(&path, r#"{"bitrate": 48}"#).unwrap();
+        let cfg = load_from(&path);
+        assert_eq!(cfg.bitrate, Some(48));
+        assert!(cfg.audio.is_none());
+        assert!(cfg.auto_cover.is_none());
+    }
+
+    #[test]
+    fn invalid_json_returns_default() {
+        let dir = tempfile::tempdir().unwrap();
+        let path = dir.path().join("config.json");
+        std::fs::write(&path, "not json at all").unwrap();
+        let cfg = load_from(&path);
+        assert!(cfg.audio.is_none());
+    }
+
+    #[test]
+    fn full_json_round_trips() {
+        let dir = tempfile::tempdir().unwrap();
+        let path = dir.path().join("config.json");
+        std::fs::write(&path, example_json()).unwrap();
+        let cfg = load_from(&path);
+        assert_eq!(cfg.audio.as_deref(), Some("original"));
+        assert_eq!(cfg.bitrate, Some(64));
+        assert_eq!(cfg.auto_cover, Some(true));
+        assert_eq!(cfg.validate, Some(false));
+        assert_eq!(cfg.log_level.as_deref(), Some("info"));
+        assert!(cfg.transcribe.is_none());
+    }
+
+    #[test]
+    fn transcribe_accepts_bool() {
+        let dir = tempfile::tempdir().unwrap();
+        let path = dir.path().join("config.json");
+        std::fs::write(&path, r#"{"transcribe": true}"#).unwrap();
+        let cfg = load_from(&path);
+        assert!(matches!(cfg.transcribe, Some(TranscribeSetting::Auto(true))));
+    }
+
+    #[test]
+    fn transcribe_accepts_string() {
+        let dir = tempfile::tempdir().unwrap();
+        let path = dir.path().join("config.json");
+        std::fs::write(&path, r#"{"transcribe": "nl"}"#).unwrap();
+        let cfg = load_from(&path);
+        assert!(matches!(cfg.transcribe, Some(TranscribeSetting::Language(ref s)) if s == "nl"));
+    }
+
+    #[test]
+    fn config_dir_ends_with_dpub() {
+        let dir = config_dir();
+        assert_eq!(dir.file_name().unwrap(), "dpub");
+    }
+}