Skip to content

Releases: XaviCode1000/webfang

v1.1.1: Blindaje de Fidelidad (D1–D5)

Choose a tag to compare

@XaviCode1000 XaviCode1000 released this 11 Jul 12:53

🔧 Fixed / Observability Hardening (Blindaje de Fidelidad D1–D5)

D4 — Error Source Integrity

  • ScraperError::Download / Network now carry #[source] Box<dyn Error + Send + Sync>.
  • CrawlError::Download and DownloadError::Network preserve #[source].
  • From<DownloadError> for CrawlError keeps the full cause (wreq::Error → Connect/timeout) instead of .to_string().
  • scrape_flow no longer flattens with .to_string(): failures retains the ScraperError and the orchestrator prints the source() chain (← cause).
  • is_transient_error inspects the boxed error Display (broadened keywords) → 5xx/connect/timeout retries preserved.

D5 — Observable Connection Pooling

  • client_id (stable {:p} pointer of the shared Arc<Client>) recorded as a span field in wreq_downloader::fetch and adapters::downloader::Downloader::download_once, and as an event field on the download log.
  • Verified live: 15 asset downloads shared the same client_id → pool reuse confirmed (TLS handshake savings, socket-exhaustion protection).

D1–D3 — Stability

  • D1: --download-concurrency 0 rejected at CLI (parse_download_concurrency) and clamped to 1 in with_download_concurrency → no buffer_unordered(0) deadlock.
  • D2/D3: FileTraceLayer emits parent_id and a logical trace_id (W3C OTel / root-span / seed) → reconstructable trace tree.

Maintenance

  • fix(clippy): parse_set_cookie uses ? instead of match (resolves clippy::question_mark on clippy 1.97 / CI).

Verification

  • cargo clippy --all-targets --all-features -- -D warnings ✅ · unit+integration+behavioral tests ✅ · all-features tests ✅ · MCP handshake ✅ · CI conclusion: success.

v1.1.0 — Obsidian Integration & Critical Bug Fixes

Choose a tag to compare

@XaviCode1000 XaviCode1000 released this 04 Apr 02:23

🎉 New Features

Obsidian Markdown Export

  • Wiki-links conversion — Same-domain [text](url)[[slug|text]] for Obsidian compatibility
  • Relative asset paths — Absolute paths rewritten as relative to .md file location
  • Tags in frontmatter--obsidian-tags "tag1,tag2" adds YAML frontmatter tags
  • CLI flags--obsidian-wiki-links, --obsidian-tags, --obsidian-relative-assets

Vault Auto-Detect & Quick-Save

  • 4-tier vault detection — CLI --vault > env OBSIDIAN_VAULT > config file > auto-scan upward for .obsidian/app.json
  • Quick-save mode--obsidian --quick-save bypasses TUI, saves directly to {vault}/_inbox/YYYY-MM-DD-slug.md
  • Rich metadata — Extended YAML frontmatter with readingTime, language, wordCount, contentType, status for Dataview queries
  • Obsidian URI — Opens saved notes in Obsidian via obsidian://open?vault=...&file=... (Linux, fire-and-forget)

🐛 Bug Fixes

Bug Severity Fix
completions subcommand requires --url Medium required_unless_present=subcommand
Frontmatter closing --- delimiter malformed Medium fm.trim_end() + explicit newline
Wiki-links corrupt embedded images Medium Regex alternation to skip images
Relative paths not converted to wiki-links Medium Handle root-relative paths (/path/to/page)
HttpClient missing redirect policy High .redirect(wreq::redirect::Policy::limited(10))

📦 New Dependencies

Crate Version Purpose
whatlang 0.18 Language detection (pure Rust, ~81KB)
urlencoding 2.1 URL encoding for Obsidian URI
slug 0.1 URL slug generation for filenames
pathdiff 0.2 Cross-platform relative paths

🧪 Testing

  • 20/20 real-world tests passing across 6 different websites
  • 361 unit tests passing (36 new for Obsidian features)
  • 0 clippy warnings
  • Verified against: books.toscrape.com, quotes.toscrape.com, webscraper.io, web-scraping.dev, scrapethissite.com, qwen-code-docs

📝 Documentation

  • New docs/OBSIDIAN.md — Complete Obsidian integration reference
  • Updated README.md — Obsidian features, usage examples, badges
  • Updated CHANGELOG.md — v1.1.0 release notes
  • Updated docs/CLI.md — Obsidian flags reference
  • Updated DEVELOPMENT.md — Latest achievements, test counts

💡 Usage Examples

# Quick-save to detected vault
rust_scraper --url https://example.com --obsidian --quick-save

# Full Obsidian mode
rust_scraper --url https://example.com \
  --vault ~/Obsidian/MyVault \
  --obsidian-wiki-links \
  --obsidian-tags "rust,web,scraping" \
  --obsidian-relative-assets

# Set vault via environment variable
export OBSIDIAN_VAULT=~/Obsidian/MyKnowledge
rust_scraper --url https://example.com --obsidian --quick-save

What's Changed

  • feat(v1.3): SPA detection warning + JsRenderer trait stub by @XaviCode1000 in #20
  • feat(obsidian): vault auto-detect, quick-save and rich metadata by @XaviCode1000 in #24

Full Changelog: v1.0.7...v1.1.0

v1.0.7 — Indestructible & Lean Edition

Choose a tag to compare

@XaviCode1000 XaviCode1000 released this 01 Apr 01:19

What's Changed

  • feat: AI-Powered Semantic Content Extraction with Embedding Preservation Fix (Issue #9) by @XaviCode1000 in #11
  • refactor(core): stabilize v1.0.7 by removing dead code and unstable FFI dependencies by @XaviCode1000 in #13

Full Changelog: v1.0.4...v1.0.7

v1.0.4

Choose a tag to compare

@github-actions github-actions released this 10 Mar 07:01

What's Changed

Full Changelog: v1.0.0...v1.0.4