Repository navigation
Releases: chendefine/dsh-web-fetch-playwright
Releases · chendefine/dsh-web-fetch-playwright
Release list
v0.2.8
feat: migrate to the dsh 0.1.7 plugin-configuration system (0.2.8) Same design as dsh-web-search-aggregation: the host's settings service derives the settings section from the plugin entry's Config schema, so the plugin stops installing its own section and every field turns .volatile() — a committed edit is pushed into the running plugin's volatile references without a remount. Behavior, defaults, and field semantics are unchanged; only the configuration plumbing moves. - host: apply() now receives the PlaywrightPluginConfig volatile face (snapshotsOf/resolveConfig in src/config.ts); the provider keeps its per-fetch () => config() thunk, so a settings commit still reaches the next fetch. @deepseek-ai/dsh-settings is gone from peerDependencies (installSection no longer exists) and schemastery needs >=3.18.4 for .volatile() - client: the card moves from the removed settings.plugin.item seat (设置 → 插件 → 插件配置 tab, disclosure-card chrome) to the Plugins page's plugins.bundle.config + plugins.row.config seats, served by one shared controller through ctx.configForms (mounts gated by whileServed); CardForm stages over the shared ConfigForm contract (revision-fenced set/unset booleans) and the summary view answers a row's description fallback - deps: devDependencies aligned to the host generation 0.1.7-rc.2, cordis 4.0.4; dsh.client.inject drops dsh-client-runtime - tests: config spec resolves through the volatile face (schema output, snapshotsOf, defaults, commit visibility); client-form spec fakes the ConfigForm boolean contract and covers dispose - verified: typecheck, 131/131 tests incl. real-browser integration, build, host smoke (8 volatile fields, apply registers the provider, per-fetch resolve), client smoke (bundle registers, both seats mount)
v0.2.7
Fixed
- Boot no longer fails against dsh-settings 0.1.2+ (
dsh webdied withplugin tree failed to load … The requested module '@deepseek-ai/dsh-settings' does not provide an export named 'installSettingsSection'). dsh-settings 0.1.2 moved that behavior onto the settings service itself;apply()now installs the section throughctx.inject(['settings'], (settingsCtx) => settingsCtx.settings.installSection(...)), solib/index.jsno longer imports@deepseek-ai/dsh-settingsat runtime at all. Dependencies were aligned to the host generation and the dsh-settings/dsh-web peer ranges tightened to>=0.1.2-alpha.0 <0.3.0. Fixes #3. - Normal pages of Cloudflare-Bot-Management sites no longer misclassify as challenges (badcase:
https://openrouter.ai/openai/gpt-6-astra-pro). Cloudflare injects its passive JavaScript-Detections (JSD) telemetry into every normal page of a protected zone; the URL embeds the/cdn-cgi/challenge-platform/prefix, so the content-level marker scan misread plain-200 real-content pages as interstitials and the fetch failed withWEB_FETCH_CHALLENGE(last status 200). The classifier now strips thescripts/jsd/telemetry directory before the prefix check and the DOM probe excludes it from its script-src scan; real interstitials (orchestrate scripts,window._cf_chl_opt,#challenge-*, the localized title family) still classify.
Removed
dsh.plugin.json, the standalone plugin manifest — fully redundant with what the host reads frompackage.json(dsh.*fields pluscordis.patch.yml). The package manifest is now the single version source.
Full Changelog: v0.2.6...v0.2.7
v0.2.5
Added
- Bounded Cloudflare-challenge wait (issue #2). When a navigation lands on a challenge interstitial, the fetch now keeps the same page and browser context and waits for the browser's own verification to clear it, instead of returning the interstitial as content — with a strict site that meant "Just a moment…" as markdown within ~1s while the real article loaded seconds later.
- Detection is layered behind a suspicion gate: the documented
cf-mitigated: challengeresponse header, then 403/503 HTML from aserver: cloudflareedge, then content-level markers — the localized interstitial title family ("Just a moment...", "请稍候…", "Минутку…", …) plus structural markers (/cdn-cgi/challenge-platform/scripts,#challenge-*elements,cf-chl-widget-frames,window._cf_chl_opt, the.footer .footer-inner .ray-idfooter). Markers are attribute/assignment-shaped so an article that merely quotes them stays clean, and the content tier only runs on challenge-compatible responses (isChallengeCompatibleResponse: 403/429/503, orserver: cloudflare/cf-raypresent) — interstitials never ship a plain 200, so a normal article cannot be misread as a challenge. Hard-block pages ("Attention Required!") classify separately and fail immediately. - During the wait the provider tracks the last main-frame navigation response (the real page reloads in after the clear) and probes the live DOM every 500ms, so SPA-style clears (content swapped with no navigation) are captured too; a settled-DOM recheck catches chained rounds (JS test → Turnstile interstitial).
- Knobs:
challengeWaitMs(0–60000, default 15000; 0 disables the whole path and restores exact pre-0.2.5 behavior) andchallengeRetries(0–3, default 1 — a same-tab re-navigation whose context keeps any clearance cookies the browser earned). Everything stays inside the 45s per-fetch deadline with a finish reserve; on exhaustion the fetch fails with the provider-specificWEB_FETCH_CHALLENGEcode naming the site, budget, and last challenge status. - Cookie lifecycle unchanged by design: isolated fetches' clearance dies with their context (verified), profile-mode clearance stays in the remote browser's own persistent profile (verified: a second profile fetch skips the challenge). No clicking, no CAPTCHA answers, no fingerprint spoofing.
- Settings card gains a Cloudflare challenge wait (ms) number field;
scripts/challenge-demo.mjsprints a before/after against a local simulated strict edge.
- Detection is layered behind a suspicion gate: the documented
- New exports:
classifyChallengeResponse,classifyChallengeHtml,isChallengeCompatibleResponse,CHALLENGE_DOM_PROBE,CHALLENGE_TITLE_RE,CHALLENGE_POLL_INTERVAL_MS,CHALLENGE_FINISH_RESERVE_MS,WEB_FETCH_CHALLENGE_CODE,DEFAULT_CHALLENGE_WAIT_MS,DEFAULT_CHALLENGE_RETRIES,effectiveChallengeWaitMs,effectiveChallengeRetries, and theChallengeVerdicttype.
npm: https://www.npmjs.com/package/dsh-web-fetch-playwright/v/0.2.5
Full changelog: v0.2.4...v0.2.5
v0.2.4
Fixed
- Elide inline
data:image payloads to size placeholders. Build tools (Docusaurus/webpack) inline images above a size cutoff straight into the HTML asdata:image/png;base64,...URIs. They survived the whole denoise pipeline as raw base64 — Readability keeps them (they are content), DOMPurify allowsimg+data:(itsDATA_URI_TAGSallowlist), and Turndown's default image rule emits them verbatim. On the onlyoffice.com docs events page that meant 65% of the returned body was base64 (100k chars, truncated at the cap). Network-level filtering cannot touch them: a data URI is never fetched, so the provider's subrequest abort missed it entirely.- Inline images are now rewritten in the DOM before extraction to
— alt text, MIME type, and approximate decoded size kept. The same page now returns 41.8k chars complete with zero base64 payload. - Covers both the article path and the whole-document fallback; runs under the denoise toggle only (raw-HTML mode untouched).
- Inline images are now rewritten in the DOM before extraction to
- Realigned
dsh.plugin.json's drifted version (still 0.2.2 after 0.2.3) withpackage.json.
Full changelog: v0.2.3...v0.2.4