v1.21.0 — HTTP Basic Auth, guide hygiene, and four new demonstration cases

Highlights

• --http-credentials — Diwall now answers HTTP Basic Auth (RFC 7617),
  the network-level challenge a reverse proxy (Caddy, nginx, Traefik) raises
  in front of self-hosted admin interfaces — a common pattern in
  self-hosting that Diwall's form-based vault mechanism didn't cover until
  now. Credentials are resolved from the vault and scoped to the target's
  origin — never sent to a third-party origin loaded in the same browser
  context. Confirmed against a real Basic Auth–protected target, not just a
  synthetic fixture.
• docs/GUIDE_LLM.md back under its own budget — compressed from 413
  to 250 lines, the limit CLAUDE.md had promised but silently exceeded
  across several prior releases. A length check now runs as part of
  scripts/verifier-coherence.sh to prevent the drift from recurring.
• Non-presumption rule — a model must never assert that a Diwall
  capability is absent, and must never presume one exists, without
  checking the documentation first.
• Four new demonstration cases in docs/GUIDE.md — self-hosted
  observability/analytics dashboards, ticketing platform administration,
  local events tracking, and honest e-commerce access data under Citizen
  Navigation. Tools and platforms are described by functional category
  rather than named, so the examples stay usable as prompt templates
  regardless of which specific product a reader is running.

Fixed

• --secrets vault-file resolution: dedicated http_username/
  http_password keys are tried first, falling back to the plain
  username/password pair already used by most existing vault files —
  found during real-target validation.
• A stale claim in the documentation stated --secrets could be passed
  multiple times to combine several vaults; it cannot (single value,
  last one silently wins) — corrected.
• A vault-file naming assumption (filename derived from the target's
  hostname) was clarified: with --secrets, the filename is whatever the
  operator chose, never hostname-derived.
• The .deb package never shipped scenarios/diagnostic_dom.json or
  scenarios/test_stealth.json, even though the operational manual and
  two notices reference them directly — found via a full clean-install
  test, not just an upgrade test.

Also in this release

• scenarios/interoperabilite/ gained a local, deterministic Basic Auth
  fixture (a minimal server issuing a real 401 challenge) as a permanent
  regression fixture.
• docs/RETOUR_EXPERIENCE.md, docs/MANUEL.md, docs/FAQ_LLM.md, and
  README.md updated for completeness alongside the new capability.

v1.20.0 — Observability and human-operator compass demonstration cases

Observability

• journal.py --erreurs — new flag filtering the operations log to entries where resultat != "succes", same pattern as the existing --mutatif filter.
• latences_actions — shot.py now measures wall-clock time per dispatched action and exposes it as an always-present JSON root key ({"index", "type", "latence_ms"} per entry). Complements the existing global citoyennete.duree_totale_ms — useful for spotting which specific step of a scenario is slow.

Human-operator compass — demonstration use cases

• A real, runnable scenario for local CSS/JS troubleshooting: scenarios/exemples/depannage_local.json (fast probe, erreurs_js/erreurs_console, --som, pixel-diff validation pattern).
• Narrative guidance in docs/GUIDE.md for two further composition patterns (hardware component comparison across shops, single-page-app documentation synthesis) — no scenario file committed for either and no third-party brand named, by design: a public scenario pinned to one named commercial target degrades with that site's anti-bot posture rather than staying reproducible.

Packaging

• diwall_1.20.0-1_all.deb (attached below) installed as a real in-place upgrade over an active v1.19.0-1 installation (sudo apt install ./diwall_1.20.0-1_all.deb, never dpkg -i alone) — the upgrade path itself had never been exercised before this release; every prior cycle was a purge followed by a fresh install.

Documentation

• README.md, docs/MANUEL.md, docs/FAQ_LLM.md: corrected stale version references and command examples left over from earlier releases.

Full changelog: v1.19.0...v1.20.0

v1.19.0 — Reliable recommendations, chained-scenario traceability, .deb packaging fixes

Reliability fixes

• mode_conseille (pre-flight configuration advice) is now filtered to
  successful diagnostic runs only — a diagnostic interrupted mid-way (WAF
  block, timeout, crash) can no longer feed a recommendation to a later call.

New capability

• Chained-scenario traceability (chainage): when a scenario uses
  declencher_scenario, the journal now records the full call tree
  (scenario, depth, action_start, action_end) alongside the flattened
  action list. The root CLI reader (journal.py) renders it as an indented
  tree. Absent on any run without chaining — purely additive.

Debian package fixes

• postinst now creates /opt/diwall/references/ — missing since the
  package's introduction, this caused watch.py --sauver-reference to fail
  with a permission error on a fresh install.
• postinst now sets the execute bit on shot.py/watch.py/rpa.py/
  journal.py — watch.py invokes shot.py as a direct subprocess and
  failed with a permission error without it.
• Both caught and fixed via a real install/remove cycle on this release.

Documentation

• Clarified: etat (the deterministic operational verdict) is declarative,
  never a gate — no verb dispatcher checks pret_a_agir before running an
  action.
• New decision rule for --ignorer-waf: only after an independent,
  non-mutating check confirms the page is actually usable.
• Documented the guide-read lock's cooperative-nature limit (a model already
  holding a valid token can pass it without rereading the guide) rather than
  hardening it.
• Reference-safe assertion guidance for --replay-verifier/
  monitor-verifier.sh: assert structure, not volatile values (timestamps,
  counters, session tokens).
• Zero-delay guidance for min_action_delay_ms on local development targets
  — the shipped default (800 ms) protects an unconfigured first run against
  the public internet, not a doctrine of slowness.
• New docs/ACCESS_OBSERVATIONS.md: a neutral, dated log of real access
  outcomes.
• Three previously undocumented watch.py flags now documented:
  --prompt, --heatmap-tile, --sortie-json.
• GUIDE.md title and wording no longer single out "human" operators.

Tooling (git source only, not packaged)

• scripts/verifier-coherence.sh: static check that documentation notice
  versions stay in sync with the index table, and that every CLI flag is
  documented somewhere under docs/.

v1.18.0 — Guide-read lock, mode_conseille, nested iframes, structural monitoring

Mandatory guide-read lock (--guide-version) + --version

shot.py, rpa.py, and watch.py now refuse to run without proof of reading docs/GUIDE_LLM.md — the one deliberate exception to the project's additive-only design. A model that never read the guide gets a hard failure instead of improvising silently.

• --version — prints the installed version and exits immediately, no Playwright launch.
• --guide-version X.Y — token read from the top of docs/GUIDE_LLM.md (<!-- notice-version -->). Once accepted, a local marker (~/.config/diwall/guide_state.json) avoids re-asking — until the guide's next substantial update.

Triggered by a recurring field friction, surfaced directly by a Claude instance on a partner project: documentation alone kept failing to get read before first use.

mode_conseille — deterministic pre-flight advice

New etat.mode_conseille object, never an order, always grounded in a real measurement. Diwall checks the operations journal's history for the current host: if a prior DOM diagnostic (diagnostic_dom.json) detected a JS framework or Shadow Roots, a configuration recommendation (--mode full, --shadow-dom, --som-rafraichir) appears in the output. Absent when no real data exists — never a guess.

Nested iframes (iframe_chemin)

cliquer_iframe and remplir_iframe accept a new iframe_chemin key: an ordered array of CSS selectors to descend through multiple levels of nested iframes. iframe_selecteur (single frame) stays strictly unchanged — the two keys are mutually exclusive, deliberately not a pluralised name, to avoid confusion on a generated action.

Continuous structural monitoring (scripts/monitor-verifier.sh)

New orchestration script: one pass, zero image, zero LLM call, built on the existing --no-capture + --replay-verifier composition. Silent when stable, an ntfy push on regression. Repetition over time is left to cron/systemd-timer — no internal daemon, no long-running memory-leak risk.

Interoperability fixtures

New structural non-regression corpus under scenarios/interoperabilite/: a neutral witness (example.com) and a fully Diwall-controlled local HTML fixture (nested iframe, open Shadow DOM component, form). Reuses the existing --replay-verifier — no new comparison capability.

---

How to test

/opt/diwall/venv/bin/python3 /opt/diwall/shot.py --version
/opt/diwall/venv/bin/python3 /opt/diwall/shot.py --url https://example.com --mode fast --guide-version 3.6

Validation: scenarios/v1.18.0_validation/verifier.py, 5/5. Regression on v1.15.2/v1.16.0/v1.17.0/v1.17.2: 0.

v1.17.2 — Vault write guard and reliability fixes

Four fixes addressing a security gap and three reliability issues.

Security

• The operations journal and mutative-run proof archiving now verify the vault is actually mounted before writing. Previously, if the vault was closed at write time, entries were silently written in clear text to the raw host directory. Both write paths now redirect to the existing local fallback instead.

Reliability

• Set-of-Mark: repeated captures within the same page no longer leave stale element markers behind, preventing a rare identity collision that could resolve --som-rafraichir to the wrong element.
• WAF detection: generic vendor names (Cloudflare, Akamai) are now matched only against the page title instead of the full page HTML, eliminating false positives on pages that simply load an ordinary CDN resource. A new --ignorer-waf flag lets the operator override a residual false positive without disabling the signal entirely.
• --checkpoint: a run stopped by a citizenship cap (max_actions_par_run/max_pages_par_run) no longer loses its progress — it is now treated the same way as a partial failure instead of a completed run.

v1.17.1 — Documentation quality pass

Documentation quality pass

Diwall has moved fast through three release cycles in quick succession
(v1.15.2, v1.16.0, v1.17.0) — deterministic run verdicts, a unified run
identity, passive WAF detection, scenario checkpoints, stable Set-of-Mark
resolution, and cross-origin iframe support. This release brings every
public document up to the same standard: complete, accurate, and genuinely
useful to read.

Documentation

• docs/MANUEL.md — full pass through 1.17.1. New sections cover the
  deterministic etat verdict, operation_id, the passive WAF signal,
  --replay-verifier, --checkpoint, --som-rafraichir, and cross-origin
  iframe actions (cliquer_iframe/remplir_iframe), each with a working
  example. Action and CLI flag tables, and the output JSON reference, updated
  to match. The stealth benchmark walkthrough now uses a quantitative
  fingerprint-count method instead of a visual screenshot comparison.
• README.md — usage example corrected, vault encryption status
  clarified (fully supported since v1.5.0), capabilities and requirements
  tables extended through v1.17.0. The long-superseded "Roadmap (v1.7)"
  section removed — those items (Shadow DOM, cross-origin iframes) have been
  shipped for several release cycles now.
• docs/FAQ_LLM.md — cross-origin iframe support documented, boussole
  field reference and version history extended through v1.17.0.
• docs/GUIDE.md, docs/GUIDE_EXPLORATION.md — Citizen Navigation
  summary, WAF signal guidance, and v1.17.0 exploration checklist items
  added for operators and exploring agents.

No functional change to shot.py or rpa.py — this release is documentation
catching up to capability, not a code fix.

v1.17.0 — Frontier of robustness and qualification

Frontier of robustness and qualification

Four items extending Diwall's reach into longer, more dynamic, and more complex targets.

New

• --replay-verifier / --sauver-verifier-reference: structural
  non-regression check (HTTP status, DOM stats, evaluer results, SoM element
  count) against a saved reference — no pixels, no vision model, CI-friendly.
• --checkpoint: resume a long scenario after a mid-run failure without
  replaying already-completed actions. Session state and action-list position
  persist; DOM state (open modals, partially filled forms) does not.
• --som-rafraichir: stable element identity for cliquer_som /
  remplir_som on highly dynamic pages. By default, Set-of-Mark IDs are
  resolved by re-indexing the live DOM, which can silently retarget to the
  wrong element if the page mutates between capture and click. This flag
  marks each element at capture time and resolves by that marker instead —
  opt-in, no change to default behaviour.
• cliquer_iframe / remplir_iframe: target elements inside same-origin
  or cross-origin iframes via Playwright's native frame API. No Set-of-Mark
  numbering inside frames yet — selector-based targeting only.

v1.16.0 — Deterministic boussole and unified run identity

Deterministic boussole and unified run identity

Six items building toward a boussole an agent can act on directly, plus a unified run identity.

New

• etat: a deterministic verdict object (pret_a_agir, niveau_confiance,
  raisons) at the JSON root, synthesizing authentication, session drift,
  citizenship caps, and console/JS errors into one read.
• operation_id: a single UUID generated once per run, isolating all
  temporary files under /tmp/diwall/<operation_id>/ and shared with the
  operations journal — replaces two independent, inconsistent identities.
• Passive WAF detection: citoyennete.waf_bloquants flags a likely block
  (HTTP 403/429 or known keywords) as a non-fatal signal — never an exception.
  Diwall reports the friction; it does not arbitrate access.
• erreurs_console: captures console.error(...) calls, distinct from
  uncaught JS exceptions (erreurs_js).
• citoyennete.indice_agressivite: ratio of mutating to total actions per
  run, logged in the operations journal.

Fixed

• --stealth was non-functional since v1.15.0. The installed
  playwright-stealth 2.x removed the stealth_sync function this project
  called; the import failed silently and boussole.stealth_actif reported
  true regardless. Fixed to use the current Stealth().apply_stealth_sync()
  API, and stealth_actif now reflects real application, not just the flag.
  Verified: fingerprint-detection failures on a public bot-detection benchmark
  dropped from 12 to 0 with the fix applied.

v1.15.2 — Consolidation, DX and anti-collision patch

Consolidation, DX and anti-collision patch

Eight items from a June multi-model review cycle. Zero new capability — hygiene and defensive hardening only.

Fixed

• Same-second capture collision: chemin_png() now uses nanosecond resolution
  instead of second resolution. Two concurrent runs no longer risk overwriting each
  other's captures under /tmp/diwall/.
• Silent no-op on --auth-indicator-negative: passing this flag without
  --auth-indicator previously had no effect at all (the auth check block was
  skipped entirely). Now rejected early (exit 2) with an explicit message, before
  any browser is launched.

New

• scenarios/exemples/: three canonical, secret-free scenarios covering the
  fast probe pattern, Set-of-Mark navigation, and a secured RPA login pattern
  (vault-backed credentials, initial page assertion).

Documentation

• Exhaustive activation table for every conditional key in boussole.
• data-testid selector priority guidance.
• Perceptual fallback ladder (SoM → accessibility tree → visual click) for
  elements SoM cannot number.
• --stealth + --shadow-dom compatibility clarified.

Tests

• scenarios/v1.15.2_validation/: live proof that --no-evaluer blocks only
  scenario-level evaluer actions (not Diwall's own internal DOM calls), and
  that the auth negative assertion degrades correctly.

v1.15.1 — Security hardening

Security hardening — 11 vectors, 12/12 tests green

Static audit performed in session 46. Zero new features. Defensive hardening only.

Critical

• C1 — evaluer audit trail: every evaluer execution is now logged with its script
  and return value in operations.jsonl. New --no-evaluer flag blocks the action entirely
  — recommended in production on interfaces with sensitive forms.
• C2 — operations.jsonl permissions: file created with explicit 0o640 mode and diwall
  group ownership via os.open. Prevents world-readable journal entries.

Elevated

• E1 — _prendre_capture(): single entry point for all PNG captures. Secret-masking JS
  is guaranteed to run before every screenshot, with restoration in finally.
• E2 — URL scheme validation: javascript:, file:, data: and any non-HTTP scheme
  are rejected before Playwright launches (exit code 2).
• E3 — Journal URL sanitisation: query string and fragment are stripped before writing
  to operations.jsonl. OAuth tokens and reset codes are never logged.
• E4 — --ignore-tls-errors flag: replaces the hardcoded ignore_https_errors=True.
  TLS validation is now strict by default; the flag is opt-in for LAN/internal PKI targets.

Medium

• M1 — Preflight scope: extended from .md + scenarios/*.json to .py, .sh, .yaml.
• M2 — Fallback journal: /tmp/diwall/ directory created with 0o700, file with 0o600.
• M3 — Secret masking: _MASQUER_SECRETS_JS extended from 1 to 9 selectors
  (autocomplete, name patterns for token, secret, otp, totp).

Low

• F1 — Structural credential check: preflight now detects plaintext values in remplir
  actions via Python JSON parsing — no longer relies on a hardcoded password string.
• F2 — Inline action validation: _valider_actions_vault() runs on charger_actions()
  and on the --action inline path. A vault_cle field without depuis_vault raises an
  error before Playwright launches.