refactor: replace scraper with lol_html for HTML charset prescanning

[Copilot]

• Identify scraper usage (only in impit/src/response_parsing/mod.rs)
• Replace scraper with lol_html = "2.7.2" in impit/Cargo.toml
• Rewrite prescan_bytestream in response_parsing/mod.rs to use lol_html
• Build and test verified passing
• Regenerate impit-node/yarn.lock to fix CI (updated native packages from v0.10.0 → v0.10.1)

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Make scraper dependency optional or replace with lightweight alternative</issue_title>
<issue_description>I'm using impit as a library for browser fingerprinted HTTP requests. Impit making scraper crate a required dependency causes two problems:

First problem: namespace collision. Scraper is a common crate name. When impit pulls it in as a transitive dependency, any workspace that also has a crate named scraper gets ambiguous resolution, for example I had to specify exact version of my own crate in order for this command to run: cargo run -p scraper@0.1.0 --bin scraper --, otherwise I was getting an error:

error: There are multiple `scraper` packages in your project, and the specification `scraper` is ambiguous.

Since scraper is a generic name, this is likely to affect other consumers too.

Second problem: using scraper as dependency is too heavyweight for what it does. The only usage of scraper crate I found is in response_parsing/mod.rs — prescan_bytestream() parses the first 1024 bytes into a full html5ever DOM tree just to read and attributes. This pulls in html5ever, selectors, ego-tree, cssparser, etc. for a two-selector lookup.

Additionally, this encoding detection isn't even wired into the response pipeline — Impit::send() returns a raw reqwest::Response. The decode() function is only exposed as an opt-in utility via impit::utils, so consumers who don't call it still pay the compile cost.

Suggested fixes (either would work):

• Make scraper optional behind a feature flag (e.g. encoding-detection) so consumers who don't use impit::utils::decode don't pull it in.
• Replace "scraper" with lol_html, a streaming HTML parser that can extract charset from tags without building any node trees. Different name avoids the namespace issue, and it's far lighter for this use case.</issue_description>

<agent_instructions>Try replacing scraper with lol_html (or similar lightweight html parser), as described in the issue. Do not add the additional feature flag. Make minimal changes to achieve this goal. Use conventional commits for the commit messages and PR titles. Be as concise as possible, do not add excessive comments.</agent_instructions>

Comments on the Issue (you are @copilot in this section)

• Fixes Make scraper dependency optional or replace with lightweight alternative #396

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[barjin]

@copilot regenerate the yarn.lock lockfile so the CI passes

[Copilot]

@copilot regenerate the yarn.lock lockfile so the CI passes

Done in eb547e8 — updated impit-node/yarn.lock to bump native packages from v0.10.0 → v0.10.1, matching the current package.json version. yarn install --immutable now passes locally.

[Copilot]

Pull request overview

This PR removes the scraper dependency from the impit Rust crate by rewriting the HTML charset prescanning logic to use lol_html, addressing both the “heavyweight dependency” concern and the common-crate-name workspace collision described in #396. It also updates the Node wrapper lockfile to keep CI green after native package updates.

Changes:

• Replaced DOM-based charset extraction (scraper) with streaming parsing via lol_html in prescan_bytestream.
• Swapped scraper = "0.25.0" for lol_html = "2.7.2" in impit dependencies (and updated Cargo.lock accordingly).
• Regenerated impit-node/yarn.lock and normalized impit-node/package.json optionalDependencies entries for native package versions.

Reviewed changes

Copilot reviewed 3 out of 5 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
impit/src/response_parsing/mod.rs	Reimplements HTML prescan encoding detection using lol_html element handlers instead of scraper selectors/DOM.
impit/Cargo.toml	Replaces scraper dependency with lol_html.
Cargo.lock	Reflects dependency graph changes from swapping scraper → lol_html.
impit-node/yarn.lock	Updates native package resolutions (v0.10.0 → v0.10.1) to match regenerated lockfile state.
impit-node/package.json	Reorders/normalizes optionalDependencies entries (no functional changes observed).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.