Skip to content

v1.8.0

Choose a tag to compare

@rashidazarang rashidazarang released this 05 Aug 19:49

Crawlio Browser is now open source under Apache-2.0 at
Crawlio-app/crawlio-browser
the full MCP server and Chrome extension source, not just distribution artifacts.
The selector kernel in packages/selectors remains MIT (it ports MIT and
BSD-3-Clause code); see NOTICE and THIRD_PARTY_NOTICES.md.

Added

  • detect_sections — live-DOM region perception. Returns a depth-capped tree of page
    regions (semantic tags, ARIA landmarks, data-component/data-testid attributes, and
    PascalCase/BEM class hints) with a verified selector, bounding box, interactive-element
    count, and text density per region. Regions scope perception; pair with
    browser_snapshot({ selector }) to mint interaction refs inside a chosen one
  • Table detection upgrades — a native <table> fast path that promotes <thead> cells
    to real column names (previously the header row was lost entirely), geometric consistency
    filters on the repeated-block path, and normalized confidence / strategy / warnings
    on every candidate so an agent can threshold results instead of reading a raw score
  • Semantic column naming in extract_table — link-aware columns (title, title_url)
    instead of DOM-path keys
  • Structured problem error codes — every failure now carries a machine-readable code
    (disconnected, target_closed, permission_denied, timeout, …) alongside the message,
    so callers branch on cause rather than parsing prose
  • Idle debugger release (opt-in) — after a configurable quiet period the debugger detaches
    so Chrome's "being debugged" banner clears; the next command re-attaches and restores the
    session, including the injected scripts only the capture path installs. Configure with the
    new set_idle_release / get_idle_release tools (minimum 60s; idleMs: 0
    disables). It never fires during a recording, an active agent session, a pending dialog,
    installed request-interception rules, active coverage, or a pinned frame

Fixed

  • pick_element and robot-training were broken in every shipped build — both install
    their in-page kernel via a single browser_evaluate, but the extension hard-rejected any
    expression over 10,000 characters and the kernel prelude is ~17KB, so both failed 100% of
    the time. The cap is now a shared constant (32,768) with a regression test asserting the
    prelude fits
  • CDP disconnect recovery re-enabled nothing — recovery rebuilt commands from stored
    domain names that already included .enable, producing Page.enable.enable, and dropped
    the parameters Performance.enable requires. Every replay failed silently, so after any
    mid-session disconnect network and console capture stopped without an error
  • Stale bridge sockets could permanently unverify their replacement — a zombie socket's
    late onclose cleared the handshake state and identity timer belonging to a newer socket
    on the same port, leaving it refused-but-never-dropped in token mode
  • Domain state was last-writer-winsensureDebugger and startNetworkCapture each
    overwrote the other's record, so recovery could re-enable a subset of the live domains
  • Per-port reconnects had no backoff — a flapping server produced a fixed 3s connect/close
    loop; now exponential to a 30s cap, reset on a successful open
  • Errors thrown inside execute() lost their structure — the sandbox boundary flattened
    them to a bare message, discarding permission and problem details
  • Selectors were embedded into generated page JavaScript with JSON.stringify alone,
    which leaves U+2028 and U+2029 raw. Both are valid JSON but are line terminators in
    JavaScript source, so the programs were at the mercy of the parsing engine; they are now
    escaped, and the injection tests execute a hostile selector rather than grepping for it
  • Offline-queue rejections carried no problem code — the failure a user hits whenever
    the extension is closed was indistinguishable from any other error
  • npm ci && npm run typecheck failed on a fresh clone, because the two workspace
    packages are file: dependencies whose main/types point at uncommitted dist/ output
    and neither had a prepare script. This broke CI and any new contributor's first build
  • The published npm package omitted NOTICE, which Apache-2.0 §4(d) requires
    redistributions to carry — so the tarball claimed Apache-2.0 while dropping the attribution
    notice it depends on, including the Chromium BSD-3 and Selector Forge credits
  • Two declared problem codes were never emitted. opt_out was dead: all twelve
    site-opt-out refusals return a response literal directly, bypassing the only place the
    classifier runs. Four bridge rejections were untagged as well — a cleared queue, a
    drain-side resend timeout, a disconnect inside the reconnect grace window, and a stopped
    bridge

Changed

  • export_session_raw now writes to ~/.crawlio/<domain>-session.json (previously a
    different home directory). Pass outputPath to choose your own location; it must still be
    inside your home directory
  • Minimum @modelcontextprotocol/sdk raised to 1.25.0. The server statically imports
    @modelcontextprotocol/sdk/server/express.js, a subpath that does not exist before 1.25 —
    the previous ^1.8.0 floor allowed an install that crashed at startup with
    ERR_MODULE_NOT_FOUND, in stdio mode as well as portal mode
  • Relicensed from proprietary to Apache-2.0; package.json, server.json, and README
    repoint to the new public repository
  • persistState failures are now logged instead of silently swallowed
  • CI verifies the extension bundle builds and that versions stay in sync across
    package.json, both manifests, and server.json
  • CRAWLIO_RAW_LANE replaces CRAWLIO_RE_LANE as the documented switch for the
    raw-capture lane, and CRAWLIO_*_MODE=raw|traffic replaces =re. The old spellings still
    work and will be removed in the next major, so no configuration breaks today
  • Adds ESLint and oxlint, wired into CI as separate steps. One project-specific rule enforces
    that every debugger attach goes through attachDebugger(), so a tab can never end up
    attached but untracked. Prettier is deliberately absent
  • Adds .editorconfig, .gitattributes (pinning LF so a Windows checkout cannot rewrite
    shell scripts), SUPPORT.md, issue and pull-request templates, Dependabot, and CodeQL
  • CI pins every third-party action to a commit rather than a mutable tag, .npmrc holds new
    dependency versions for a week before they may enter the lockfile, and a version gate fails
    any pull request that changes shipped files without moving the version forward
  • The public repository is produced by a scripted, allowlist-based export whose sanitation
    gate is itself covered by tests that plant each internal directory and require rejection