fix: fixed index.html path for weather_app#7
Merged
cristibleotiu merged 1 commit intodevelopfrom Nov 21, 2024
Merged
Conversation
aidamian
pushed a commit
that referenced
this pull request
Feb 11, 2025
aidamian
added a commit
that referenced
this pull request
Feb 11, 2025
toderian
added a commit
that referenced
this pull request
Apr 30, 2026
* feat: add attestation registry submission in redmesh close flow * fix: add execution_id to attestation * Add RedMesh job-start attestation submission flow * fix: set up private key in plugin config * fix: pass history read * fix: add loggign for attestation * feat: user can configure the count of scanning threads on UI * feat: add data models package * feat: keep jo config in r1fs * feat: single aggregation + consolidated pass report (phase 2) * feat: job archive & UI Aggregate (phase 3-4) * feat: fix backend endpoints to work with new cstore structure (phase 5) * fix: use constants everywhere in API (phase 11) * feat: live worker progress endpoints and methods (phase 1) * feat: job deletion & purge (phase 15) * fix: listing endpoint optimization (phase 15) * feat: scan metrics collection (phase 16a) * feat: scan metrics aggregation at node level (phase 16b) * fix: metrics visualization improvements * fix: scan profile simplification * fix: redmesh test * fix: service tests * fix: improve web tests | add cms fingerprinting * feat: add OWASP-10 identification * feat: add erlang_ssh & dns bind to cve db * fix: CVEs for databases * fix: CVEs for CMS & Frameworks * fix: tests CVEs for CMS & Frameworks * fix: Java applications & servers * fix: detected services count calculation * fix: add jetty | fix CVE findings * fix: use running env port for signaling plugin readiness * feat: job hard stop * fix: job stop * fix: PoT * feat: add scanner nodes ips to the report * feat: display thread-level ports info and stats * fix: increase job check timeout * feat: improve per-worker progress loader. Display per-thread status * fix: tests classification * fix: move metrix collector to a separate file * refactor: rename redmesh_utils to pentester_worker * refactor: split the pentester_api_01 * refactor: split code in mixins | split tests * feat: extract BaseLocalWorker for GrayBox integration (phase 0) * feat: add core modules for gray box (phase 1) * feat: graybox core modules safety / auth / discovery (phase 2) * feat: graybox probes (phase ) * feat: graybox worker and API integration (phase 4) * fix(redmesh): preserve graybox job identity in phase 1 contracts * fix(redmesh)(phase 2): correct graybox evidence counting and aggregates * refactor(redmesh)(phase 3): split launch API by scan type * refactor(redmesh)(phase 4): model feature capabilities by scan type * fix(redmesh)(phase 5): harden worker probe metrics and isolation * docs(redmesh)(phase 6): summarize navigator graybox parity * fix(redmesh)(phase 7): harden attestation and audit logging * refactor(redmesh)(phase 8): extract launch strategies and state machine * fix: add llm agent prompts for graybox scans * fix: add scan type to worker progress * fix: add extra scanning probes to graybox * fix: add extra scanning probes to graybox | login rate limit | password reset token predictability | business logic validation * fix: add more graybox tests (path traversal, session fixation...) * use config var for progress publish interval * fix cleanup constants * fix: docs cleanup * fix: normalize live-progres publish interval * fix: enforce cap for continuous jobs * fix: add job_revision to job store model * fix: add tests * refactor: extract redmesh query services * refactor: extract redmesh launch services * refactor: extract redmesh lifecycle services * feat: split redmesh graybox secrets from job config * refactor: add redmesh repository boundaries * refactor: type redmesh repository boundaries * refactor: normalize redmesh running job state * refactor: add explicit redmesh network feature registry * refactor: streamline redmesh worker phase execution * refactor: type redmesh graybox runtime flow * refactor: add redmesh graybox probe context * refactor: harden redmesh graybox auth lifecycle * refactor: type redmesh graybox probe boundaries * feat: harden redmesh secret storage boundary * refactor: add redmesh typed evidence artifacts * refactor: normalize redmesh graybox finding contract * feat: add redmesh finding triage state * feat: add redmesh cvss finding metadata * feat: harden redmesh resilience and launch policy * test: add redmesh regression and contract suites * fix: harden redmesh live progress phase metadata * fix: harden redmesh llm failure handling * fix: preserve pass reports during finalization * fix: llm analysis generation * fix: add redmesh agents.md * feat(redmesh): define distributed reconciliation schema * feat(redmesh): publish startup live state * feat(redmesh): reconcile worker live state * feat(redmesh): reannounce missing worker assignments * feat(redmesh): stop jobs on retry exhaustion * fix(redmesh): align distributed job read paths * fix(redmesh): ignore stale and malformed live rows * test(redmesh): cover worker reconciliation states * feat(redmesh): add worker retry timeline events * refactor(redmesh): group reconciliation config * refactor(redmesh): share nested config resolution * refactor(redmesh): group llm agent config * refactor(redmesh): group attestation config * refactor(redmesh): group graybox budgets config * feat(redmesh): shape llm analysis payloads * feat(redmesh): trim llm findings payloads * feat(redmesh): compact webapp llm payloads * feat(redmesh): track llm payload shaping stats * docs(redmesh): record llm payload shaping rollout * fix(redmesh): normalize llm agent plugin class name * feat(redmesh): add MISP export module — Phase 1 backend Add toggleable MISP threat intelligence export with manual push and JSON download. Uses PyMISP to build MISP 2.5-compliant events from scan data. New files: - services/misp_config.py — config normalization via resolve_config_block - services/misp_export.py — event building, push, JSON export, status tracking - mixins/misp_export.py — _MispExportMixin with 4 endpoint methods - tests/test_misp_export.py — 37 tests (config, severity filter, event building, push with mocked PyMISP, re-export update, error handling) Mapping: findings→vulnerability, ports→ip-port, TLS→x509, tags for OWASP/CWE/ATT&CK. Export metadata stored in CStore (mutable), not PassReport (immutable R1FS). Severity filter (MIN_SEVERITY=LOW default) excludes INFO from export. 4 new endpoints: export_misp, export_misp_json, get_misp_export_status, get_misp_export_config_status. Verified live against MISP v2.5.36. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(redmesh): persist MISP export metadata in CStore finalized stub CStoreJobFinalized was silently dropping the misp_export field during _coerce_job_payload serialization — from_dict/to_dict round-trip only preserved known dataclass fields. Add misp_export: dict = None to the model so export status survives CStore writes for finalized jobs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(graybox): fail-closed aborts + phase-metrics bookkeeping (audit #1, #5) Phase 1 of PR 388 audit remediation. Issue #1: preflight and authorization failures recorded a fatal finding but let the scan continue into auth, discovery, and probe phases. Introduce GrayboxAbort exception + _abort() helper so every safety gate (unauthorized target, preflight, auth, phase-level session refresh) terminates the pipeline immediately. execute_job catches GrayboxAbort, records state["aborted"] / abort_reason / abort_phase, increments a metrics counter, and emits a single [ABORT-ATTESTATION] audit log line. Issue #5: execute_job unconditionally called metrics.phase_end() in finally, double-closing the phase already closed by its own method and corrupting timing data. Each phase method now tracks self._phase_open around its phase_start/phase_end; the execute_job finally only closes when a phase escaped without its own cleanup. Additional hardening in this commit: - state["aborted"] / abort_reason / abort_phase registered in get_worker_specific_result_fields() with OR / first-non-empty merge rules (used by Phase 3 aggregation). - _safe_cleanup wraps auth.cleanup so its errors never mask an abort. auth.cleanup already uses timeout=5 on logout. - Per-probe session refresh keeps its soft-fail contract (failed:auth_refresh) — one flaky re-auth does not kill a loop over N probes. - _abort docstring prohibits passing target-controlled text as the reason (defense in depth; Phase 2 adds the LLM-side sanitizer too). Tests: 11 new TestGrayboxAbortBehavior cases cover every abort path, state surface, aggregation registration, double-close prevention, and the plaintext-credential audit. All 767 existing tests still pass. Not blockchain-attested (deviation from plan): the existing mixins/attestation.py is a blockchain-submission module, and submitting a tx per abort is expensive. Audit trail is via grep-able [ABORT-ATTESTATION] log line. Follow-up ticket can extend the mixin if compliance ever requires blockchain attestation of abort events. * fix(llm): nested service_info traversal + prompt-injection defense (audit #3, #9) Phase 2 of PR 388 audit remediation. Issue #3: _extract_report_findings only iterated service_info.values() once, skipping findings under the nested {port: {probe_method: {findings:[]}}} shape emitted by pentest_worker. Network scans arrived at the LLM with materially incomplete evidence. Issue #9: _build_network_service_summary read fields directly off the per-port entry, which in the nested shape is a map of probe dicts — so banner/server/product/etc. came back empty. Both are fixed via _flatten_network_port_entry which handles the nested shape, the legacy flat shape, and does probe-rank conflict resolution (protocol-match > TLS > web-tests > generic). Every finding gets _source_probe and _source_port stamped at ingest (chain-of-custody across aggregation and downstream rendering). Prompt-injection defense (OWASP LLM01:2025) bundled here because every target-controlled text path touches the same code: - _sanitize_untrusted_text wraps banner/server/title/evidence/etc in <untrusted_target_data>...</> delimiters, scrubs ASCII control bytes, escapes the outer delimiter if attackers embed it, and filters a handful of known LLM-instruction tokens. Belt-and- suspenders — delimiters + the new system-prompt prologue are the real defense; trivial bypass of string-matching is expected. - _LLM_SYSTEM_PROMPT_UNTRUSTED_PROLOGUE is prepended to every analysis-type prompt so the model knows to treat delimited content as opaque data. - abort_reason / abort_phase (Phase 1 additions) are sanitized at the LLM boundary as defense-in-depth even though Phase 1's contract already forbids target-controlled text there. Probe-output validator (_validate_probe_result) quarantines malformed probe dicts into payload["_malformed_probe_results"] instead of crashing or silently dropping. Missing severities default to UNKNOWN; non-list findings fields are coerced with a reason recorded. Shared test fixture at tests/fixtures/multi_probe_report.py exercises: two+ probes per port, metadata conflict, legacy flat shape, malformed probe, prompt-injection banner. Consumed by test_llm_agent_shape (8 tests), test_llm_agent_injection (11 tests), test_llm_agent_validator (9 tests). Existing test_hardening assertion on evidence length updated to account for the wrapper overhead. Full suite: 795 passing (was 767, +28). * fix(finalize): resolve worker_cls per scan_type + source attribution (audit #4) Phase 3 of PR 388 audit remediation. Issue #4: maybe_finalize_pass called owner._get_aggregated_report(node_reports) with no worker_cls, so the aggregation helper fell back to PentestLocalWorker fields even for webapp (graybox) scans. Graybox-specific fields (graybox_results, completed_tests, and the Phase-1-added aborted/ abort_reason/abort_phase) from the second and later graybox workers were dropped from the aggregate — contaminating archive data, risk scoring, UI aggregates, and LLM analysis. Fixed by resolving worker_cls from job_specs["scan_type"] via services.scan_strategy.get_scan_strategy (already exists — no registry pattern needed, existing mapping is sufficient). A structured [FINALIZE] log line records which worker class was used for each pass. Chain-of-custody: _stamp_worker_source stamps _source_worker_id and _source_node_addr on every finding-bearing structure before merging. Handles nested service_info, legacy flat shape, graybox_results, web_tests_info, correlation_findings, and top-level findings. Idempotent via setdefault — Phase 2's _source_probe / _source_port stamps are preserved. Tests: 9 new cases covering nested + flat stamp coverage, idempotency, multi-worker graybox merge, abort-state OR semantics, and a regression test confirming the network aggregation path still works without worker_cls. Full suite: 804 passing (was 795, +9). * fix(live-progress): weak-auth gate + commutative merge (audit #6, #7) Phase 4 of PR 388 audit remediation. Issue #6: _thread_phase returned "done" as soon as graybox_probes landed in completed_tests, even when weak-auth was still pending. The UI/launcher could show a scan as done while the worker was actively running weak-auth attempts. Issue #7: _merge_worker_metrics only treated v == "failed" as a hard failure when picking the worst probe status. Prefixed failures like failed:auth_refresh lost to a neighbor's completed, so the merged metric underreported real failures. Fixes: - GrayboxCredentialSet.weak_auth_enabled(job_config) is the single source of truth for "will weak-auth run?" Used by both _run_weak_auth_phase (worker gate) and _thread_phase (live progress), so the UI and the worker can never disagree. - _thread_phase now takes a required `worker` parameter — no default. Forgotten call sites fail loudly with TypeError. - Aborted scans (state["aborted"] from Phase 1) short-circuit to "done" so live progress doesn't linger in a stuck phase. - _merge_worker_metrics uses a total-order _status_rank: failed > failed:* > skipped > skipped:* > completed > other with suffix tiebreak (alphabetically smallest wins within a rank class). Merge is provably commutative over worker order. Tests: 17 new cases including a 2058-permutation order-independence check across the full status alphabet. Full suite: 821 passing (was 804, +17). * fix(query): return aggregated_report_cid for archived analysis (audit #8) Phase 5 (final) of PR 388 audit remediation. Issue #8: the archived branch of get_job_analysis returned target_pass.get("report_cid"). Archived pass objects (written by services/finalization.py) only carry aggregated_report_cid. The response therefore surfaced None even when a real aggregated report existed, creating an inconsistent API between live-pass and archived-pass analysis lookups. Fix: return aggregated_report_cid in the archived branch. Response key name kept as "report_cid" for API continuity — current consumers don't dereference it (Navigator does not call /get_analysis; MISP uses aggregated_report_cid directly), and renaming the key is gratuitous churn. Inline comment documents the shape divergence between the running and archived branches. Missing aggregated_report_cid is an archive-integrity signal (older buggy path, or a failed aggregation step). A grep-able [ARCHIVE-INTEGRITY] warning is emitted so operators can spot it. Deviation from plan: attestation event for archive-integrity skipped for the same reason Phase 1 skipped the abort attestation — the existing attestation mixin is blockchain-only, and per-warning blockchain submissions are expensive. The log line is the audit trail. Follow-up ticket can extend attestation if compliance requires it. Tests: 3 new cases covering clean archive, missing aggregated CID, and the short-circuit where llm_analysis is missing (no integrity warning emitted in that path). Full suite: 824 passing (was 821, +3). All 9 audit items resolved. reviews/pr-388-audit.md updated to mark each item with its resolving commit. * chore: revert .devcontainer/devcontainer.json to pre-MISP state Out-of-scope devcontainer edits landed on misp-integration via commit 3d18138 (MISP export module Phase 1) leaked into this PR's diff when branched. Restore the file to the state at 3d18138's parent (a24d32d) so the PR 388 remediation PR only contains red_mesh/ changes. No behavior change in backend/runtime — only developer-environment config. --------- Co-authored-by: Alessandro <defranceschi_a@yahoo.com> Co-authored-by: toderian <vitalii.toderian@ratio1.ai> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.