v0.4.0

Changelog

• 1afe7ac Add "ardvark work" command for cooperative frontier draining
• 74cb07a Add host-affinity sharding and DB-backed per-domain page budget
• ce9771a Add lease-based frontier claiming and global termination
• 1e77f99 Clarify doc comments on driver defaulting, lease clock skew, and worker sharding
• b38edaf Deduplicate lease-clearing updates and crawl/work CLI wiring
• e6def05 Fix page budget blocking re-crawls: gate only new rows, not re-activation
• aeb5dac Guard frontier mutators against lost-lease double processing
• 23062e1 Make Frontier.Counts a single-statement snapshot; document dead-worker stranding
• fc76ead Merge branch 'feat/distributed-crawling'
• ce55e9d Merge branch 'main' into feat/distributed-crawling
• 22ae4ee Merge feat/distributed-crawling: follow catalog/registry pointers in registry search results (d2bc362)
• 3b97a9d Persist frontier item provenance in the DB instead of in-memory maps
• f323a18 Prepare v0.4.0: changelog
• c11529a Shard frontier items on fetch-target host, not attribution host
• 6f8fde4 bump dep
• 35d09e4 feat(ard): recognize media types via mediatype parser, base-aware pointers
• 89a86b1 feat(crawler): follow catalog/registry pointers by Kind, not exact suffix
• d2bc362 feat(crawler): follow catalog/registry pointers found in registry search results
• 8271aac feat(mediatype): classify by semantic Kind with pointer/known helpers
• 393111d feat(mediatype): derive encoding Format hint from suffix/type
• 3e6f17c feat(mediatype): parse media types into type/base/suffix/params
• 7316881 style(mediatype): gofmt test file
• a823c0e test+fix(mediatype): guard profile override on pointers; cover unsuffixed catalog, format-in-message, media_type pass
• e4da3ac tools/smoketest: live 10-worker distributed-crawling smoke test
• b5230c2 tools/smoketest: parameterize frontier backend (DB=mysql|postgres)