feat(graybox): OWASP API Security Top 10 (2023) implementation#406
Merged
Conversation
Locks in the scenario ID prefix for the new graybox OWASP API Top 10 2023 probe families before any catalog or probe code is written. Disambiguates from existing PT-A<NN>-<NN> web-app IDs which differ by only one character in position 5. Implements Subphase 1.0 commit #1 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the permissive `PT-[A-Z0-9]+-\d+` catch-all with explicit alternation over the three valid prefixes documented in the ADR: PT-A<NN>-<NN>, PT-API7-<NN>, and PT-OAPI<N>-<NN>. Adds positive and negative test cases covering the boundary IDs (PT-OAPI10-01) and the visually-ambiguous typo `PT-API1-01` that the new convention prevents. Implements Subphase 1.0 commit #2 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds eight frozen-dataclass endpoint sub-models for the OWASP API Top 10 2023 graybox families: - ApiObjectEndpoint — BOLA (PT-OAPI1-01) - ApiPropertyEndpoint — BOPLA read+write (PT-OAPI3-01/02) - ApiFunctionEndpoint — BFLA read+mutating (PT-OAPI5-01..04) - ApiResourceEndpoint — bounded resource consumption (PT-OAPI4-*) - ApiBusinessFlow — sensitive flow abuse (PT-OAPI6-*) - ApiTokenEndpoint — broken-auth probes (PT-OAPI2-01..03) - ApiInventoryPaths — inventory mismanagement (PT-OAPI9-*) - ApiSecurityConfig — aggregating wrapper ApiOutboundEndpoint is deliberately absent: API10 is deferred to Phase 9 until callback-receiver infrastructure exists. Mirrors the existing IdorEndpoint / JwtEndpoint shape (from_dict ctor, sensible defaults, frozen=True). GrayboxTargetConfig is not yet wired — that lands in Subphase 1.1 commit #2. Implements Subphase 1.1 commit #1 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the api_security field to GrayboxTargetConfig (default-empty ApiSecurityConfig) and routes the new section through from_dict so a launch payload can carry the OWASP API Top 10 endpoint configs end-to-end without any other plumbing. Implements Subphase 1.1 commit #2 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds TestApiSecurityConfig covering all eight new sub-models: - Per-endpoint defaults and full from_dict round-trip - Missing-required-key behaviour (raises KeyError for `path`) - ApiSecurityConfig default lists (ssrf body fields, tampering fields, debug paths, OpenAPI candidates) - GrayboxTargetConfig wiring (default api_security, payload propagation, KeyError on malformed nested payload) 16 new test methods. Existing 18 unchanged. Implements Subphase 1.1 commit #3 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extends the catalog schema with an optional `attack: list[str]` field and registers the 23 OWASP API Top 10 v1 scenarios: API1 (BOLA): 1 entry (PT-OAPI1-01) API2 (auth): 3 entries (PT-OAPI2-01..03) API3 (BOPLA): 2 entries (PT-OAPI3-01, -02) API4 (resource): 3 entries (PT-OAPI4-01..03) API5 (BFLA): 4 entries (PT-OAPI5-01..04) API6 (flows): 2 entries (PT-OAPI6-01, -02) API8 (misconfig): 5 entries (PT-OAPI8-01..05) API9 (inventory): 3 entries (PT-OAPI9-01..03) Per-family attribution (`api_access`/`api_auth`/`api_data`/`api_config`/ `api_abuse`) matches the five-family probe split landing in Subphase 1.3. API7 SSRF keeps its legacy ID `PT-API7-01`. Side fix: legacy `PT-API7-01` `owasp` tag was `A10:2021` in the catalog but the probe code already emits `API7:2023`. Catalog now agrees with probe so the new Navigator §3.3.3 dispatch picks it up. Adds helpers `graybox_scenario(id)` and `attack_for_scenario(id)` so emit helpers (Subphase 1.6) can use the catalog as the runtime source of truth for ATT&CK defaults. API10 (Unsafe Consumption) intentionally absent — Phase 9 follow-up. Implements Subphase 1.2 commit #1 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Promotes the MITRE ATT&CK mapping to a v1 contract: every PT-OAPI* and PT-API7-01 catalog entry MUST declare a non-empty `attack` list. The catalog (not probe code or this markdown plan) is the executable source of truth for the default attack mapping; ProbeBase.emit_vulnerable will read it via `attack_for_scenario(id)` in Subphase 1.6. Also: - Bumps the graybox inventory floor from 80 to 103 (legacy 80 + 23 new PT-OAPI* entries). - Adds a test for `attack_for_scenario` covering known/legacy/unknown ids. - Adds a count + per-category coverage assertion (8 categories ≥1 entry, no PT-OAPI10-* in v1). Note: the "widen regex" commit (#2 in the plan's Subphase 1.2 list) was landed earlier in Subphase 1.0 commit #2 (1d8d07e) so it isn't duplicated here. Implements Subphase 1.2 commit #3 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
First of the five OWASP API Top 10 probe families introduced by the five-family split (amendment #4). Covers API1 (BOLA) and API5 (BFLA). Skeleton only — `run()` returns no findings until the concrete probe methods land in Phases 2.1, 2.3, and 3.4. Implements Subphase 1.3 commit #1 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Covers OWASP API2 — Broken Authentication. Skeleton; concrete probes land in Phase 2.6 (PT-OAPI2-01/02) and Phase 3.x (PT-OAPI2-03 stateful). Implements Subphase 1.3 commit #2 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Covers OWASP API3 — Broken Object Property Level Authorization (BOPLA). Skeleton; concrete probes land in Phase 2.2 (read-side excessive exposure) and Phase 3.1 (stateful mass-assignment write). Implements Subphase 1.3 commit #3 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Covers OWASP API8 (Security Misconfiguration) and API9 (Improper Inventory Management). Skeleton; concrete probes land in Phase 2.4 and Phase 2.5. Implements Subphase 1.3 commit #4 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Final of the five OWASP API Top 10 probe families. Covers API4 (Unrestricted Resource Consumption) and API6 (Unrestricted Access to Sensitive Business Flows). Skeleton; concrete probes land in Phase 3.2 (bounded resource consumption) and Phase 3.3 (stateful flow abuse). All five API probe families are now registered. Implements Subphase 1.3 commit #5 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two new tests on the registry side (test_target_config.py): - test_registry_has_expected_probes asserts every legacy + new API family key is present. - test_api_family_classes_importable resolves each module-relative dotted path, instantiates the class, and verifies capability flags. Two new tests on the worker side (test_worker.py): - test_supported_features_include_api_top10_families confirms the five new keys flow through GrayboxLocalWorker.get_supported_features(). - test_api_family_skeletons_dispatch_cleanly instantiates each new family against a minimal mocked context and asserts run() returns an empty list (skeleton behaviour expected before Phase 2/3 probes are wired). Implements Subphase 1.3 commit #6 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The launch path already deep-copies the `target_config` dict into the persisted JobConfig and forwards it to the worker, which parses it via GrayboxTargetConfig.from_dict (extended in Subphase 1.1 to handle the new `api_security` section). No filter strips unknown keys; the only mutation is `_apply_launch_safety_policy` normalising `discovery`. This commit documents the passthrough contract in the docstring so future contributors do not assume new target_config sections need explicit allowlisting at the launch boundary. Implements Subphase 1.4 commit #1 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Asserts that a launch with a populated `target_config.api_security` section round-trips through `launch_webapp_scan` into the persisted JobConfig: object_endpoints (with tenant_field), function_endpoints (with revert_path), token_endpoints (with logout_path), and inventory_paths (with deprecated_paths) are all preserved verbatim. Implements Subphase 1.4 commit #2 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Operator-facing reference for the OWASP API Top 10 target_config section introduced in Subphase 1.1. Documents every endpoint sub-model, which scenario IDs they drive, which fields are required, stateful gating expectations, and the API10 / auth / budget forward-reference notes. Minimal-config example at the bottom for quick-start. Implements Subphase 1.4 commit #3 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…FormAuth Introduces the strategy pattern that lets graybox auth handle Bearer and API-key targets in addition to form-login. This commit only adds the new infrastructure; the existing AuthManager remains the active code path until Subphase 1.5 commit #3 wires it to the strategy dispatcher. New `graybox/auth_strategies.py`: - `AuthStrategy` ABC with `preflight()`, `authenticate(creds)`, `refresh()`, `cleanup()`, and a shared `make_session()` helper. - `FormAuth(AuthStrategy)` carrying the existing form-login behaviour (CSRF auto-detection, robust success detection, hidden-input fallback). Behaviour is identical to the legacy inline logic; copies are intentional so the orchestrator can switch over in commit #3 without intermediate breakage. Design note: package layout deviates slightly from the plan's `graybox/auth/` package suggestion. Sibling module `auth_strategies.py` preserves all existing import paths (`from .auth import AuthManager`, ~15 callers + ~10 test patches) and is functionally equivalent. Can be re-organised into a package later if it grows. Implements Subphase 1.5 commit #1 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mutable credential bundle handed by AuthManager to each AuthStrategy at authenticate() time. Covers form (username/password), Bearer (bearer_token + optional bearer_refresh_token), and API-key (api_key). Secret-handling contract: - Never serialised — no to_dict(), no JSON. The persisted JobConfig carries only `secret_ref` + non-secret capability flags (Subphase 1.5 commit #8). - __repr__ overridden to expose only boolean has_* flags, never values. - clear() overwrites every field with empty strings; called by AuthManager on cleanup so accidental references see no historical secrets. Implements Subphase 1.5 commit #2 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…trator AuthManager now delegates form-login to FormAuth and preflight checks to the strategy's preflight() method. The orchestrator owns lifecycle (expiry, retry, multi-principal coordination, cleanup) while the strategy owns protocol-level details (CSRF detection, success heuristics). Mechanical changes: - `_try_login_attempt` builds a FormAuth, hands it a Credentials VO, catches `requests.RequestException` to classify retryable failures. - `preflight_check` delegates to strategy.preflight(). - `_is_login_success`, `_extract_csrf`, `_find_csrf_value` removed from AuthManager (now live on FormAuth verbatim). - `extract_csrf_value` static helper delegates to FormAuth (preserves the public probe-facing API surface). - `detected_csrf_field` property unchanged — populated via `strategy.last_detected_csrf_field` after each auth attempt. - FormAuth.authenticate raises `requests.RequestException` on transport errors so the orchestrator can drive the retry path. Test updates (no behaviour change, only refactor accommodation): - `TestCsrfAutoDetect` instantiates FormAuth directly and exercises `_extract_csrf` there; the standalone-helper variant of the legacy `test_csrf_field_property` was reshaped to test `last_detected_csrf_field`. - `TestLoginSuccessDetection._check` calls FormAuth._is_login_success. - All `requests` patches updated from `auth.requests` to `auth_strategies.requests` since that's where the HTTP calls now happen. - `test_authenticate_retries_transient_transport_error` drops the leading-anon-session MagicMock from `Session.side_effect` (the anon session is built via auth.requests, which is unpatched in the test). Implements Subphase 1.5 commit #3 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… to GrayboxTargetConfig `AuthDescriptor` carries the non-secret auth configuration for graybox: auth_type selector, header/scheme/location knobs, optional Bearer refresh URL, and `authenticated_probe_path` used by strategy-aware preflight. Secret values (bearer_token, api_key, bearer_refresh_token) are deliberately absent — they travel as top-level launch parameters and land in the R1FS secret payload (Subphase 1.5 commit #8). Wired into ApiSecurityConfig as the `auth` field with a default-form AuthDescriptor so existing form-login launches continue to work without any config change. Implements Subphase 1.5 commit #4 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`_build_strategy` now resolves `auth_type` from `target_config.api_security.auth` and dispatches to the appropriate AuthStrategy. The default (``form``) continues to route to FormAuth so existing graybox launches behave identically. Non-form auth types raise NotImplementedError until Subphase 1.5 commits #6 (bearer) and #7 (api_key) land — explicit failure is better than silently dispatching to the wrong strategy. `preflight_check` (already strategy-delegating from commit #3) now correctly preflights against whichever strategy `auth_type` selects. Implements Subphase 1.5 commit #5 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`BearerAuth` injects `creds.bearer_token` into every request via the configured header/scheme (default `Authorization: Bearer <token>`). No HTTP traffic during `authenticate`; preflight optionally HEADs `auth.authenticated_probe_path` and rejects 401/403 responses. Header name, scheme, and an optional `authenticated_probe_path` are sourced from `target_config.api_security.auth` (AuthDescriptor). The strategy gracefully degrades to defaults when ApiSecurityConfig is absent, so unit tests with minimal fixtures continue to work. Wired into `AuthManager._build_strategy` so launches with `auth.auth_type='bearer'` automatically route through this strategy. Implements Subphase 1.5 commit #6 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`ApiKeyAuth` places `creds.api_key` either in a header (default, `X-Api-Key` configurable) or a query parameter (`auth.api_key_location='query'`, `auth.api_key_query_param`). Query-parameter placement is supported for legacy interoperability — the Subphase 1.6 evidence scrubber will redact the configured param name from finding evidence; the Navigator launch form shows a warning banner (Subphase 8.5). Header is preferred and is the default. Wired into `AuthManager._build_strategy` — `auth_type='api_key'` now dispatches here; unknown auth types raise ValueError (was NotImplementedError) since the dispatch table is now complete. Implements Subphase 1.5 commit #7 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…e scrubbing
OWASP API Top 10 secrets travel as top-level launch parameters
(mirroring official_password), get persisted into the R1FS secret
payload alongside form credentials, and are blanked from the publicly
archived JobConfig before put_job_config().
Changes:
- `models/archive.py::JobConfig`: add runtime-only secret fields
`bearer_token`, `api_key`, `bearer_refresh_token` plus non-secret
capability flags `has_bearer_token`, `has_api_key`,
`has_bearer_refresh_token`.
- `services/secrets.py`:
- `build_graybox_secret_payload(...)`: accept the three new secret
kwargs.
- `persist_job_config_with_secrets(...)`: extract them from the
config, push into the secret payload, set capability flags on the
persisted config, then `_blank_graybox_secret_fields` strips raw
values before archive write.
- `_blank_graybox_secret_fields(...)`: also blanks the three new
fields.
- `resolve_job_config_secrets(...)`: repopulates the three runtime
fields from the secret payload at worker startup.
- `services/launch_api.py::launch_webapp_scan`: accept top-level
`bearer_token`, `api_key`, `bearer_refresh_token` kwargs. Replace
the unconditional "official credentials required" check with
auth-type-aware validation (form requires user+pass; bearer
requires bearer_token; api_key requires api_key). Pass through
the three new secrets to `_persist_and_announce_pentest_job`.
- `_persist_and_announce_pentest_job`: accept the three new secret
params, pass them to JobConfig.
Implements Subphase 1.5 commit #8 of the API Top 10 plan.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New test classes: - TestBearerAuthStrategy: default + custom header/scheme; empty token rejected; refresh round-trip; preflight skipped without probe_path and returns error on 401. - TestApiKeyAuthStrategy: header vs query placement; unknown location rejected; empty key rejected. - TestAuthManagerStrategyDispatch: AuthManager._build_strategy routes correctly to FormAuth / BearerAuth / ApiKeyAuth based on target_config.api_security.auth.auth_type, and raises ValueError on unknown types. Existing form-login tests unchanged; all 41 pass. Implements Subphase 1.5 commit #9 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…nce, or LLM input New `tests/test_secret_isolation.py` enforces the secret-handling contract from Subphase 1.5: - TestSecretIsolationInBuildPayload: build_graybox_secret_payload carries the three new secrets; _blank_graybox_secret_fields zeroes them. - TestSecretIsolationInPersistedConfig: persist_job_config_with_secrets produces a JobConfig with `bearer_token`/`api_key`/ `bearer_refresh_token` blanked, `has_*` capability flags set, and a populated `secret_ref`. Worker-side `resolve_job_config_secrets` repopulates the runtime fields from the secret payload. - TestSecretIsolationInCredentialsRepr: Credentials.__repr__ shows only capability booleans, never secret values. Note: GrayboxFinding evidence redaction lives in Subphase 1.6 (the centralised scrubber); this test focuses on the persistence boundary. The full LLM-input boundary check is exercised by test_llm_input_isolation.py (extended in Subphase 1.6). Implements Subphase 1.5 commit #10 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…nclusive helpers Introduces three single-call helpers that probe families use instead of constructing GrayboxFinding by hand. Two benefits: - Reduces boilerplate (the typical 8-10-line GrayboxFinding(...) call becomes a single emit_vulnerable(...)). - Provides a single point at which evidence redaction will be enforced once the centralised scrubber lands in Subphase 1.6 commit #2. Helpers: - emit_vulnerable(scenario_id, title, severity, owasp, cwe, evidence, *, attack=None, evidence_artifacts=None, replay_steps=None, remediation=None): default attack mapping resolved from `scenario_catalog.attack_for_scenario(scenario_id)` so probes do not carry per-scenario ATT&CK lists in code. - emit_clean(scenario_id, title, owasp, evidence): not_vulnerable / INFO. - emit_inconclusive(scenario_id, title, owasp, reason): records the reason as `evidence=["reason=<value>"]` for downstream grouping. Existing probe families (PT-A* / PT-API7-01) are unchanged for now — migration to the helpers is a follow-up cleanup. Implements Subphase 1.6 commit #1 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds `scrub_graybox_secrets(value, *, secret_field_names=())` to
graybox/findings.py and wires it into:
- GrayboxFinding.to_flat_finding (final storage-boundary pass)
- ProbeBase.emit_vulnerable / emit_clean / emit_inconclusive
(pre-emission scrub via _scrub_for_emission, with configured names
pulled from target_config.api_security.auth)
Generic patterns redact:
- Authorization: <…> (full header value to next field separator)
- Cookie / Set-Cookie headers (same)
- JWTs (eyJ…, three base64url chunks)
- Bare `Bearer <token>` references
- name=value forms for password / secret / token / api_key / apikey
- JSON `"name": "value"` for the same names + bearer_token + api*key
Per-call extension via `secret_field_names`: ProbeBase passes the
configured API-key header name + query param name + Bearer header name
from AuthDescriptor so custom names (X-Customer-Key, etc.) are also
scrubbed before the finding crosses the storage boundary.
Defense-in-depth: the storage scrubber runs even on findings that were
emitted before Subphase 1.6 helpers existed, so legacy probes that
construct GrayboxFinding directly cannot leak.
Implements Subphase 1.6 commit #2 of the API Top 10 plan.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New tests/test_findings_redaction.py covering scrub_graybox_secrets and to_flat_finding pass-through: - TestScrubGenericPatterns (10 cases): Authorization/Cookie/Set-Cookie headers, bare JWTs, Bearer tokens, password/token/api_key/apikey k=v forms, JSON bearer_token, embedded headers in compound evidence strings. - TestScrubConfiguredNames (2 cases): custom header + custom query parameter names supplied via secret_field_names. - TestScrubRecursive (3 cases): list / dict recursion; non-string passthrough. - TestToFlatFindingScrubs (1 case): an end-to-end GrayboxFinding with three secret patterns and four non-secret fields confirms the storage-boundary scrubber strips secrets while preserving asset identifiers, scenario_id, severity, etc. Implements Subphase 1.6 commit #3 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New TestApiAuthSecretsScrubbed class (4 cases) verifying that API-flavoured secret patterns never reach the LLM input even when carried in fields that the build_llm_input pipeline forwards: - Authorization: Bearer <jwt> in evidence — scrubbed end-to-end. - Cookie: sessionid=<value> — scrubbed. - password=<value> in evidence k=v form — scrubbed. - API-key in URL query param — scrubbed regardless of whether the carrier field is dropped (legacy `evidence`) or forwarded (`evidence_items`/title/description). Each case drives a real GrayboxFinding through to_flat_finding and then through build_llm_input so both the storage-boundary scrubber and the LLM input pipeline are exercised together. The contract: the secret value's exact string must not appear in repr(out.findings). Implements Subphase 1.6 commit #4 of the API Top 10 plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
What changed: - Moved runtime job-config secret resolution into the local launch error boundary. - Added a shared helper for marking worker entries terminal with sanitized errors. - Classified secret-resolution, assignment-validation, and launch failures with terminal reasons. Why: - A bad graybox secret_ref should surface as a terminal worker failure instead of escaping the launcher loop or leaving the job stuck.
What changed: - Added worker-owned rollback journal records for stateful graybox mutations. - Wrote pending records before mutate, updated records after revert, and surfaced manual cleanup on revert failure. - Added attempted-unknown mutation handling and exempted cleanup/revert requests from probe budget exhaustion. Why: - Stateful API probes must leave a durable cleanup trail and attempt rollback even when the mutating request outcome is uncertain.
What changed: - Require PT-OAPI2-03 to mint a disposable token from token_path before calling logout. - Prove rollback by minting a fresh token after the logout test. - Add an opt-in stateful helper mode for probes where verify=false is the clean outcome. Why: - Prevent the scanner from revoking the primary operator credential during logout invalidation checks.
What changed: - Add launch-side positive-integer validation for request budgets and graybox target-config numeric safety fields. - Normalize safe numeric strings before persistence and reject invalid, zero, negative, and oversized payload values. - Make RequestBudget.consume reject non-positive consumption amounts and fail invalid assignment budgets closed. Why: - Scanner safety limits must be explicit launcher decisions, not silent coercions at worker runtime.
What changed:
- Add exact scalar placeholder rendering for API6 flow bodies using test_account, run_id, and job_id.
- Require {test_account} in API6 mutate/revert bodies unless an explicit unsafe static-body override is set.
- Keep API6 runtime probe state in local closures instead of mutating frozen ApiBusinessFlow config objects.
Why:
- Business-flow abuse probes must operate on designated test identities, not accidentally replay static real-user payloads.
What changed: - require Bearer/API-key validation paths unless the launch explicitly opts into unverified auth - gate unverified API scenarios to auth_unverified inconclusive findings and route API7 through scenario assignment checks - carry configured API auth field names through finding storage, reporting, risk flattening, and LLM-boundary tests Why: - avoid misleading API Top 10 results and prevent custom auth secrets from leaking through direct finding/report paths
What changed: - Send host-only target_confirmation in the API Top 10 e2e harness. - Replace the skipped LLM-boundary placeholder with archive-backed redaction assertions. - Add harness unit coverage for both contracts. Why: - Keep the e2e launch path aligned with authorization validation and make the LLM/report boundary check executable.
What changed: - Mint honeypot bearer tokens in the API Top 10 e2e harness and launch through API-native auth. - Retry transient status polling failures instead of aborting the harness. - Align e2e fixture paths and manifest expectations with the API Top 10 honeypot routes. Why: - The honeypot form login is CSRF-protected; the e2e proof should exercise the bearer/API auth path used by the new probes.
Set the live API Top 10 e2e launch payload to request launcher-owned SLICE assignment, matching the multi-worker stateful validation contract. Align the e2e target config with probe contracts by using exact API6 test-account placeholders and authorizing root scope for the honeypot's exposed /openapi.json endpoint. Verification: python -m py_compile extensions/business/cybersec/red_mesh/tests/e2e/api_top10_e2e.py; python -m json.tool extensions/business/cybersec/red_mesh/tests/e2e/fixtures/api_security_target_config.json; python extensions/business/cybersec/red_mesh/tests/e2e/api_top10_e2e.py --rm http://localhost:5082 --honeypot http://172.17.0.1:30001 --scenario vulnerable --timeout 600; python -m pytest extensions/business/cybersec/red_mesh/tests -q
GrayboxHttpClient.request() only changed method to GET on HTTP 303, preserving POST on 301/302/307/308. Django's LoginView redirects post-login with 302, so the wrapper re-POSTed the login form to the LOGIN_REDIRECT_URL target. Django's CSRF middleware rejected it with 403 (csrftoken cookie rotates after successful login), and _is_login_success saw status 403 and returned False. Result: official_login_failed → FATAL abort, every form-auth graybox scan aborted before reaching discovery. Match real-world behavior of the requests library and browsers: convert POST→GET on 301/302/303 (and drop request body); preserve method on 307/308 per RFC 7231 §6.4.7. HEAD stays HEAD on any redirect since it is safe + body-less. Tests: ten new cases in test_http_client.py covering 301/302/303 conversion, 307 preservation, HEAD preservation, missing Location, out-of-scope Location, 5-hop loop cap, and sticky GET after first conversion in a chain. Live verification: full graybox scan against the rm-gb honeypot (job 3061a4a6) completes auth + discovery + all probes in 112s (was aborting at 1s). 32 vulnerable findings vs 26 in baseline 4709a7e7, 7 stateful rollbacks vs 6, 0 regressions across all 24 PT-OAPI scenarios. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Webapp launches were failing with "Failed to store job config in R1FS" whenever no operator-supplied secret-store key was configured. The prior fail-closed gate required REDMESH_ALLOW_UNSAFE_SECRET_STORE_FALLBACK plus a per-node fallback (cfg_comms_host_key or attestation private key) — the per-node fallbacks differ across rm1 and rm2, so even when the gate was opened the launcher's encryption key did not match the worker's decryption key and credentials silently came back as empty strings. Add a built-in default secret-store key that ships with the plugin and is therefore identical on every node running the same image. Resolution order: 1. REDMESH_SECRET_STORE_KEY env var (custom, audit unsafe=False) 2. cfg_redmesh_secret_store_key plugin config (custom, unsafe=False) 3. built-in default (unsafe=True; key_id "redmesh:default_plugin_key") Persisted JobConfig still records secret_store_unsafe_fallback=true when the default is in use, so audit trails reflect that the key is well-known. The cross-node failure mode is gone; deployments that want a real KMS-managed key just set the env var or plugin config. Tests: replaced the obsolete fail-closed gate tests in test_secret_isolation.py and test_api.py with assertions that the default key produces correct metadata. Added TestSecretRoundTripAcrossNodes which encrypts on a launcher FakeNode and decrypts on a separate worker FakeNode through a shared in-memory R1FS — proves credentials survive the persist→resolve round trip with no operator configuration. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ntials Remove silent fallback to the well-known _DEFAULT_SECRET_STORE_KEY so launches abort closed when no deployment-specific key is configured. The fallback now lives behind an explicit dev opt-in (REDMESH_ALLOW_UNSAFE_SECRET_STORE_FALLBACK env or cfg_redmesh_allow_unsafe_secret_store_fallback config), and is rejected unconditionally under REDMESH_ENV=production. Adds SecretStoreKeyMissing typed exception, exported from services __init__. persist_job_config_with_secrets catches it and blanks the returned config secret slots so accidental log exposure is reduced. rm1 devcontainer turns on the unsafe fallback for local dev so plug-and-play scans still work after the change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tighten bearer/API-key validation so an invalid token can't be
mistaken for a valid one via a 302 redirect to a public 200 login
page. The new flow:
* authenticated probe uses allow_redirects=False;
* 3xx, 401/403, and >=400 are all rejected;
* an anonymous control request runs against the same path. If it
also returns 2xx, an explicit success assertion (status allow-
list, marker substring, or identity JSON path) is required;
* the identity JSON path traversal is dotted-keys-only and cannot
evaluate arbitrary expressions.
AuthDescriptor grows three optional fields
(authenticated_probe_success_statuses, _marker, _identity_json_path)
to drive the new assertions. New tests cover the invalid-token-302
case, the public-2xx case, marker/identity-path success and failure,
and the status allow-list filter.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PT-OAPI6-02 sends two duplicate flow requests. Previously a transport exception during either send returned False from the mutate function, so run_stateful() concluded no mutation had happened and skipped the revert path — leaving the first duplicate order/account/voucher in place while reporting clean. Track whether the first request was already issued and, on transport exception after that point, return MUTATION_ATTEMPTED_UNKNOWN so the caller still invokes the configured revert endpoint. The verify path also returns MUTATION_ATTEMPTED_UNKNOWN on transport error so a half- landed mutation never gets converted to "clean" by silent uncertainty. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
run_stateful() previously did `confirmed = bool(verify_fn(...))`, which collapsed the MUTATION_ATTEMPTED_UNKNOWN sentinel (a non-empty string) and any other truthy-but-not-True value into a confirmed vulnerable result. That turned probe uncertainty into a published high-confidence claim. Treat only literal True as confirmation. When verify_fn returns the MUTATION_ATTEMPTED_UNKNOWN sentinel, mark the finding inconclusive with the mutation_attempted_unknown reason and keep the rollback path running. Non-bool truthy values (dicts, strings, etc.) also fall to inconclusive rather than vulnerable. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three coordinated changes so backend assignment defaults and visible metadata match the Navigator expectation: * Default webapp/API launches to SLICE (the Navigator default). MIRROR remains an explicit operator choice. * Multi-worker MIRROR no longer silently multiplies max_total_requests across workers. Without the new allow_mirror_per_worker_budget opt-in, the per-scan budget is divided across workers (budget_scope=per_scan). The opt-in keeps the old per-worker semantics for operators who genuinely want workers × budget total traffic. * Emit a job-level graybox_assignment_summary on CStoreJobRunning / CStoreJobFinalized so the dashboard has a stable source for strategy, budget, budget_scope, total_assigned_scenarios, and per-worker rows without having to derive them from each worker entry. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
assignment_hash includes assignment_revision in its payload, but the reannounce path in _maybe_reannounce_worker_assignments() bumped the revision without recomputing the hash. Workers that overlaid the revision-2 assignment into JobConfig rejected it with assignment_hash_mismatch and went terminal, even though the launcher had issued the reannounce legitimately. Add rehash_worker_assignment_dict() to scenario_runtime and call it right after the revision bump. Network jobs (no graybox fields) are left untouched. New unit test covers the rehash round-trip through GrayboxWorkerAssignment.from_job_config. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A worker upgraded to the new launcher-owned scenario assignment will reject any webapp job announced without assignment fields, marking itself terminal with assignment_validation_failed. That strands valid jobs during rolling upgrades or when a pre-PR launcher announces work. Add synthesize_legacy_mirror_assignment(): when a worker entry carries NONE of the new assignment fields, synthesize an explicit MIRROR assignment for all runtime scenarios with the per-scan budget pulled from target_config.api_security.max_total_requests (or the default). The synthesized entry is marked assignment_compat_mode=legacy_mirror and emits a one-line warning + audit event so operators see it. Partial or corrupt assignment fields (any single new field present alongside others missing) still fail closed — only fully-absent assignments take the compat path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
_mark_worker_terminal_error used to write the launcher's local job_specs snapshot wholesale. Two workers failing concurrently could overwrite each other's terminal state because the second write re-persisted a snapshot that didn't include the first worker's terminal fields. _write_job_record detected the staleness but still persisted the incoming data. Reload the current job record by job_id, overlay the patched worker entry on top, and write that merged record. If the current record is missing entirely we fall back to the incoming snapshot with a warning so we never silently drop a terminal write. New regression test covers the two-stale-snapshots case. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
REDMESH_ALLOW_UNSAFE_SECRET_STORE_FALLBACK (and the cfg equivalent) is the explicit opt-in for the shared well-known secret-store key. Previously REDMESH_ENV=production silently overrode the opt-in and fail-closed, which forced production deployments to ship a dedicated key even when operators had deliberately accepted the trade-off. Drop the production guard and the now-unused _is_production_env helper. The opt-in flag alone decides; absent dedicated key + absent opt-in still fails closed via SecretStoreKeyMissing. Update the module comment and exception message to reflect the new contract, and flip the production-rejection test to assert the opt-in is now honored. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…op10 # Conflicts: # ver.py
What changed: - Restored automatic use of the built-in graybox secret-store key when no deployment key is configured. - Updated secret-store tests to reflect dev fallback behavior. - Added the devcontainer watch helper referenced by the rm1 devcontainer. Why: - The current environment is development-first and needs graybox launches to work without requiring per-deployment secret-store key setup.
What changed: - Restored automatic use of the built-in graybox secret-store key when no deployment key is configured. - Updated secret-store tests to reflect dev fallback behavior. - Removed local rm1 devcontainer files from repo tracking so developer-specific setup stays local. Why: - The current environment is development-first and needs graybox launches to work without requiring per-deployment secret-store key setup.
…t/redmesh-api-top10 # Conflicts: # .devcontainer/rm1/devcontainer.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.