Skip to content

fix(#377): migrate url_import to firecrawl-py 2.x API#378

Merged
NoveliaYuki merged 1 commit into
projects/refactor/kbserver-lamb-integrationfrom
fix/#377/firecrawl-py-2x-migration
May 12, 2026
Merged

fix(#377): migrate url_import to firecrawl-py 2.x API#378
NoveliaYuki merged 1 commit into
projects/refactor/kbserver-lamb-integrationfrom
fix/#377/firecrawl-py-2x-migration

Conversation

@NoveliaYuki
Copy link
Copy Markdown
Collaborator

Purpose

URL imports fail at runtime with AttributeError: 'Firecrawl' object has no attribute 'crawl_url'. The plugin was written against firecrawl-py 0.x; the dependency currently in the library-manager container is firecrawl-py 4.25.2, which renamed the entry points and reshaped the response objects.

Changes

  • Replace app.crawl_url(url, params={...}, poll_interval=5) with app.crawl(url, **kwargs, scrape_options=ScrapeOptions(formats=["markdown"]), poll_interval=5) — keyword arguments now carry limit, max_discovery_depth, crawl_entire_domain, allow_external_links directly.
  • Per-page metadata extraction handles both the new Pydantic DocumentMetadata (attribute access on source_url/title) and the legacy dict form (camelCase sourceURL) for forward compatibility.
  • No behavior change to the zero-pages fallback, progress reporting, or surrounding plugin scaffolding.

The migration is dependency-only — no public API of the plugin changes, no other call sites needed updates.

Verification

  • library-manager/tests/ — 52/52 passing against the patched plugin.
  • Live smoke against a self-hosted Firecrawl instance: queue an import of https://example.com, item reaches status ready, content endpoint returns non-empty markdown.

Related

firecrawl-py >= 2.x renamed crawl_url() to crawl() and replaced the
params dict with keyword arguments + a ScrapeOptions value object. The
returned doc.metadata is now a Pydantic model with attribute access;
support both shapes for forward compatibility.
@NoveliaYuki NoveliaYuki added the bug Something isn't working label May 12, 2026
@NoveliaYuki NoveliaYuki self-assigned this May 12, 2026
@NoveliaYuki NoveliaYuki merged commit bbab916 into projects/refactor/kbserver-lamb-integration May 12, 2026
@NoveliaYuki NoveliaYuki deleted the fix/#377/firecrawl-py-2x-migration branch May 12, 2026 05:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant