Skip to content

Releases: chendefine/dsh-web-fetch-playwright

v0.2.8

Choose a tag to compare

@chendefine chendefine released this 27 Sep 13:43
feat: migrate to the dsh 0.1.7 plugin-configuration system (0.2.8)

Same design as dsh-web-search-aggregation: the host's settings service
derives the settings section from the plugin entry's Config schema, so
the plugin stops installing its own section and every field turns
.volatile() — a committed edit is pushed into the running plugin's
volatile references without a remount. Behavior, defaults, and field
semantics are unchanged; only the configuration plumbing moves.

- host: apply() now receives the PlaywrightPluginConfig volatile face
  (snapshotsOf/resolveConfig in src/config.ts); the provider keeps its
  per-fetch () => config() thunk, so a settings commit still reaches
  the next fetch. @deepseek-ai/dsh-settings is gone from
  peerDependencies (installSection no longer exists) and schemastery
  needs >=3.18.4 for .volatile()
- client: the card moves from the removed settings.plugin.item seat
  (设置 → 插件 → 插件配置 tab, disclosure-card chrome) to the Plugins
  page's plugins.bundle.config + plugins.row.config seats, served by one
  shared controller through ctx.configForms (mounts gated by
  whileServed); CardForm stages over the shared ConfigForm contract
  (revision-fenced set/unset booleans) and the summary view answers a
  row's description fallback
- deps: devDependencies aligned to the host generation 0.1.7-rc.2,
  cordis 4.0.4; dsh.client.inject drops dsh-client-runtime
- tests: config spec resolves through the volatile face (schema output,
  snapshotsOf, defaults, commit visibility); client-form spec fakes the
  ConfigForm boolean contract and covers dispose
- verified: typecheck, 131/131 tests incl. real-browser integration,
  build, host smoke (8 volatile fields, apply registers the provider,
  per-fetch resolve), client smoke (bundle registers, both seats mount)

v0.2.7

Choose a tag to compare

@chendefine chendefine released this 06 Sep 13:42

Fixed

  • Boot no longer fails against dsh-settings 0.1.2+ (dsh web died with plugin tree failed to load … The requested module '@deepseek-ai/dsh-settings' does not provide an export named 'installSettingsSection'). dsh-settings 0.1.2 moved that behavior onto the settings service itself; apply() now installs the section through ctx.inject(['settings'], (settingsCtx) => settingsCtx.settings.installSection(...)), so lib/index.js no longer imports @deepseek-ai/dsh-settings at runtime at all. Dependencies were aligned to the host generation and the dsh-settings/dsh-web peer ranges tightened to >=0.1.2-alpha.0 <0.3.0. Fixes #3.
  • Normal pages of Cloudflare-Bot-Management sites no longer misclassify as challenges (badcase: https://openrouter.ai/openai/gpt-6-astra-pro). Cloudflare injects its passive JavaScript-Detections (JSD) telemetry into every normal page of a protected zone; the URL embeds the /cdn-cgi/challenge-platform/ prefix, so the content-level marker scan misread plain-200 real-content pages as interstitials and the fetch failed with WEB_FETCH_CHALLENGE (last status 200). The classifier now strips the scripts/jsd/ telemetry directory before the prefix check and the DOM probe excludes it from its script-src scan; real interstitials (orchestrate scripts, window._cf_chl_opt, #challenge-*, the localized title family) still classify.

Removed

  • dsh.plugin.json, the standalone plugin manifest — fully redundant with what the host reads from package.json (dsh.* fields plus cordis.patch.yml). The package manifest is now the single version source.

Full Changelog: v0.2.6...v0.2.7

v0.2.5

Choose a tag to compare

@chendefine chendefine released this 27 Aug 12:11

Added

  • Bounded Cloudflare-challenge wait (issue #2). When a navigation lands on a challenge interstitial, the fetch now keeps the same page and browser context and waits for the browser's own verification to clear it, instead of returning the interstitial as content — with a strict site that meant "Just a moment…" as markdown within ~1s while the real article loaded seconds later.
    • Detection is layered behind a suspicion gate: the documented cf-mitigated: challenge response header, then 403/503 HTML from a server: cloudflare edge, then content-level markers — the localized interstitial title family ("Just a moment...", "请稍候…", "Минутку…", …) plus structural markers (/cdn-cgi/challenge-platform/ scripts, #challenge-* elements, cf-chl-widget- frames, window._cf_chl_opt, the .footer .footer-inner .ray-id footer). Markers are attribute/assignment-shaped so an article that merely quotes them stays clean, and the content tier only runs on challenge-compatible responses (isChallengeCompatibleResponse: 403/429/503, or server: cloudflare / cf-ray present) — interstitials never ship a plain 200, so a normal article cannot be misread as a challenge. Hard-block pages ("Attention Required!") classify separately and fail immediately.
    • During the wait the provider tracks the last main-frame navigation response (the real page reloads in after the clear) and probes the live DOM every 500ms, so SPA-style clears (content swapped with no navigation) are captured too; a settled-DOM recheck catches chained rounds (JS test → Turnstile interstitial).
    • Knobs: challengeWaitMs (0–60000, default 15000; 0 disables the whole path and restores exact pre-0.2.5 behavior) and challengeRetries (0–3, default 1 — a same-tab re-navigation whose context keeps any clearance cookies the browser earned). Everything stays inside the 45s per-fetch deadline with a finish reserve; on exhaustion the fetch fails with the provider-specific WEB_FETCH_CHALLENGE code naming the site, budget, and last challenge status.
    • Cookie lifecycle unchanged by design: isolated fetches' clearance dies with their context (verified), profile-mode clearance stays in the remote browser's own persistent profile (verified: a second profile fetch skips the challenge). No clicking, no CAPTCHA answers, no fingerprint spoofing.
    • Settings card gains a Cloudflare challenge wait (ms) number field; scripts/challenge-demo.mjs prints a before/after against a local simulated strict edge.
  • New exports: classifyChallengeResponse, classifyChallengeHtml, isChallengeCompatibleResponse, CHALLENGE_DOM_PROBE, CHALLENGE_TITLE_RE, CHALLENGE_POLL_INTERVAL_MS, CHALLENGE_FINISH_RESERVE_MS, WEB_FETCH_CHALLENGE_CODE, DEFAULT_CHALLENGE_WAIT_MS, DEFAULT_CHALLENGE_RETRIES, effectiveChallengeWaitMs, effectiveChallengeRetries, and the ChallengeVerdict type.

npm: https://www.npmjs.com/package/dsh-web-fetch-playwright/v/0.2.5

Full changelog: v0.2.4...v0.2.5

v0.2.4

Choose a tag to compare

@chendefine chendefine released this 24 Aug 18:33

Fixed

  • Elide inline data: image payloads to size placeholders. Build tools (Docusaurus/webpack) inline images above a size cutoff straight into the HTML as data:image/png;base64,... URIs. They survived the whole denoise pipeline as raw base64 — Readability keeps them (they are content), DOMPurify allows img+data: (its DATA_URI_TAGS allowlist), and Turndown's default image rule emits them verbatim. On the onlyoffice.com docs events page that meant 65% of the returned body was base64 (100k chars, truncated at the cap). Network-level filtering cannot touch them: a data URI is never fetched, so the provider's subrequest abort missed it entirely.
    • Inline images are now rewritten in the DOM before extraction to ![alt](data:image/png;base64,...8.9KB) — alt text, MIME type, and approximate decoded size kept. The same page now returns 41.8k chars complete with zero base64 payload.
    • Covers both the article path and the whole-document fallback; runs under the denoise toggle only (raw-HTML mode untouched).
  • Realigned dsh.plugin.json's drifted version (still 0.2.2 after 0.2.3) with package.json.

Full changelog: v0.2.3...v0.2.4