v2.0.1

Docs-only patch. Repositions the repo as a portable agent skill (not Claude-specific) and adds project-level contributor
documentation.

Added

• AGENTS.md at the repo root: project-level instructions for agents working inside the repo. YAML frontmatter (name,
  description, repository, license, verified-against), bundle conventions (VERSION pin discipline,
  wait_until=networkidle default, PEP 723 scripts, schema-generation fixture contract), repo conventions
  (main-protected branch discipline, Conventional Commits, dual-license acceptance), testing command, quality bar, and a
  "When NOT to edit" block.
• CONTRIBUTING.md at the repo root: minimal contributor flow covering issue filing, branch naming, testing, PR
  conventions, and scope bounding (upstream Crawl4AI library bugs route to the upstream repo, not this one).

Changed

• README.md overhauled for portable agent-skill framing. Title becomes Crawl4AI Agent Skill. Lead paragraph names
  the agent-host audience explicitly (Claude Code, Codex, Cursor, OpenCode, Cline, and other SKILL.md-format hosts).
• Install section: single host-agnostic git clone <repo> <host-skills-dir>/crawl4ai path, illustrated with the Claude
  Code path. Claude Desktop staging-dir zip flow removed.
• Documentation section now links all nine reference guides.
• Adds a Bundle layout table documenting SKILL.md, references/, scripts/, templates/, evals/, fixtures/,
  tests/, and the VERSION pin.
• Marketplace and Support sections removed.

v2.0.0

Major release. Verified against the Crawl4AI library at version 0.8.9 (recorded in VERSION). The repo is now the skill bundle: SKILL.md and the skill directories live at the repo root and load directly with git clone <repo> ~/.claude/skills/crawl4ai.

Added

• VERSION file pinning the Crawl4AI library version the skill is verified against.
• templates/ directory carrying browser, crawler, content-filter, and extraction YAML configs the scripts and SKILL.md
  reference (browser.yml, crawler.yml, extract_css.yml, extract_llm.yml, filter_bm25.yml,
  filter_pruning.yml, login_crawler.yml, css_schema.json).
• evals/ directory carrying four eval scenarios for verifying skill behavior, with a README.
• fixtures/ directory carrying the schema-generation reference fixture (sample product HTML, expected schema, expected
  JSON output), with a README.
• Reference guides under references/: anti-detection.md, content-filters.md, escalation.md, recipes.md,
  troubleshooting.md, url-discovery.md.
• Focused scripts under scripts/: batch_crawl.py, batch_extract.py, generate_schema.py,
  extract_with_schema.py, extract_with_llm.py.
• Test coverage for fixtures under tests/test_fixtures.py.
• LICENSE-APACHE and LICENSE-MIT carrying the full license texts; LICENSE summarizes the dual-license model (SPDX
  MIT OR Apache-2.0).

Changed

• Layout: the skill bundle lives at the repo root. The previous crawl4ai/ wrapper directory is gone. SKILL.md
  and its companion directories are direct children of the repo root.
• Default JS-render wait switches from wait_for=css:body to wait_until=networkidle across SKILL.md and every
  bundled script. wait_for=css:body was satisfied at t=0 on every HTML response and added no real wait;
  wait_until=networkidle waits for ~500ms of network quiet post-load.
• License: dual Apache-2.0 OR MIT (was MIT-only). Existing MIT consumers retain MIT terms.
• CLI guide drift fixes: the deprecated anthropic/claude-3-sonnet LLM provider reference is replaced with a
  pointer to the LiteLLM provider list, the LLM-extraction example no longer passes a stray -s llm_schema.json (the
  schema lives inside extract_llm.yml), and the filter_pruning.yml snippet drops the query key (pruning filters do
  not take one) and adds the required threshold_type: fixed.
• SKILL.md, the existing reference guides, the scripts, the tests, and the evals all refresh against Crawl4AI 0.8.9.
• README.md install section: single git clone into ~/.claude/skills/crawl4ai for Claude Code; staging-dir zip
  snippet for Claude Desktop that includes templates/, evals/, fixtures/, and VERSION. Helper-scripts list and
  E-commerce Product Monitoring example now match the new script names. License section reflects the dual-license model.
• .gitignore extended with the script-output patterns (output.md, screenshot.png, batch_results.json,
  batch_extracted.json, batch_markdown/, generated_schema.json, extracted_data.json, manual_extracted.json,
  llm_extracted.json) so script invocations from the repo root no longer pollute the working tree.

Removed

• marketplace/metadata.json. No Claude consumer reads it; the data it carried lives on SKILL.md frontmatter, GitHub
  topics, README, and LICENSE.
• The prebuilt crawl4ai.zip artifact. README documents how to build the zip from the tree on demand.
• The monolithic scripts/extraction_pipeline.py. Its responsibilities split across scripts/generate_schema.py and
  scripts/extract_with_schema.py.
• scripts/batch_crawler.py. Replaced by scripts/batch_crawl.py.

Breaking Changes

• Symlinks pointing into the previous crawl4ai-skill/crawl4ai/ no longer resolve; repoint at the repo root.
• Script callers referencing scripts/extraction_pipeline.py switch to scripts/generate_schema.py plus
  scripts/extract_with_schema.py. Callers of scripts/batch_crawler.py switch to scripts/batch_crawl.py.
• License changes from MIT-only to dual Apache-2.0 OR MIT.