Skip to content

Octocrawl 0.4.1

Latest
Latest

Choose a tag to compare

View all tags
@77777R7 77777R7 released this 10 Oct 08:19
· 916 commits to main since this release
v0.4.1
da3842f
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Octocrawl 0.4.1, released 2026-10-10: a fix release on top of 0.4.0.

  • No more connection resets while polling a crawl. A client of the local API or the local MCP host could have its next request on a kept-alive connection reset (TypeError: fetch failed, read ECONNRESET). This happened about one run in two when a crawl's status was polled after reading its pages mid-run. The cause was the server's own work holding its event loop past the keep-alive timeout. An idle connection is now closed only when no request has arrived on it. Hosted Octocrawl has had this fix since 2026-10-10.
  • Products listed at the top level of a page's JSON-LD (three names or more, one script each, as on Redfin's search results) are read as a listing's cards.
  • The page's <main> wins over chrome around it. It is preferred when every h1 of the page is in it, as on Sephora's category pages.

Packages: npm octocrawl, @octocrawl/cli, @octocrawl/sdk and @octocrawl/mcp; PyPI octocrawl-client.

npx octocrawl@latest scrape https://example.com --markdown
pip install --upgrade 'octocrawl-client[pandas]'

@octocrawl/mcp 0.4.1 is listed in the official MCP Registry as io.github.77777R7/octocrawl, and https://mcp.octocrawl.dev/mcp serves 0.4.1.

Every change since 0.4.0 is listed in CHANGELOG.md.

Website and docs: https://octocrawl.dev

Assets 2
Loading