chore: cdxgen/ORT subprocess env scrub + license fetcher follow_redirects (chore PR #6)#10
Merged
Merged
Conversation
Move docs/sessions/_next-session-prompt-chore-pr5.md to docs/sessions/archive/next-session-prompts/. The chore PR #5 prompt was fully consumed by PR #9 (merged in b23871b); keeping the file in docs/sessions/ root would clutter the working-tree status with a stale prompt. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extract the prep-only ``_PREP_ENV_ALLOWLIST`` + ``_scrubbed_env``
helper from ``tasks/scan_source.py`` into a new
``integrations/_subprocess_env.py`` module that exposes three
ecosystem-specific builders:
* ``scrubbed_env_for_prep()`` — Ruby/Cargo/Go/.NET/Java prep
* ``scrubbed_env_for_cdxgen()`` — Node/JVM/Python tooling for
cdxgen, plus the ``npm_config_*`` / ``CDXGEN_*`` prefix bands
* ``scrubbed_env_for_ort()`` — JVM tooling, plus the ``ORT_*``
prefix band
All three share the same base allowlist (PATH/HOME/LANG/LC_ALL/TZ +
corporate CA / proxy hints from chore PR #5 L1) and resolve
``os.environ`` at call time per CLAUDE.md rule #11.
Prefix bands carry a credential heuristic: any allow-listed key whose
name matches ``auth|token|password|passphrase|secret|credential|
apikey|api_key|private(_)?key`` is dropped before forwarding. That
prevents npm's ``_authToken`` / ``_password`` / ``_auth`` and
``ORT_*_TOKEN`` style keys from sneaking through the prefix gates.
``tasks.scan_source._scrubbed_env`` is preserved as an alias so the
existing prep call site and its unit tests keep working unchanged.
The cdxgen / ORT integrations adopt the new helpers in follow-up
commits.
Refs: security-reviewer Medium #1 v2 (chore PR #5 handoff §1.2).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
cdxgen runs as a Node binary under the worker. Hostile clones can load attacker-controlled ``package.json`` plugin hooks / ``cdxgen.config.json`` rules, and the existing ``env = dict(os.environ)`` inheritance hands them ``DT_API_KEY`` / ``SECRET_KEY`` / ``DATABASE_URL`` / ``*_WEBHOOK_URL`` for free — exfiltrable through plugin telemetry, crash reports, or DNS lookups in error paths (security-reviewer Medium #1 v2, chore PR #5 handoff §1.2). ``_build_cdxgen_env`` now starts from ``scrubbed_env_for_cdxgen()`` instead of ``os.environ``. The Gradle 8 compat-shim path from chore PR #5 Part C is preserved verbatim and a unit test is added pinning that operator-set ``CDXGEN_GRADLE_ARGS`` still wins. Three new tests in ``test_cdxgen_gradle_compat.py`` verify the scrubbed env strips worker secrets, forwards ``NODE_EXTRA_CA_CERTS`` / ``HTTPS_PROXY``, and keeps the existing operator-override semantics intact. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The ORT subprocess previously inherited the worker's full env (no ``env=`` argument to ``subprocess.run``). ORT is a JVM tool whose ``--rules-file`` and ``--ort-file`` inputs originate from the worker image / scan workspace, so the inputs themselves are not attacker- controlled — but a malicious ORT plugin or a JVM CVE in the rule evaluator would still see ``DT_API_KEY`` / ``SECRET_KEY`` / ``DATABASE_URL`` and could tunnel them through telemetry, crash reports, or DNS lookups in error paths (security-reviewer Medium #1 v2). ``run_ort`` now passes ``env=scrubbed_env_for_ort()``. The new env forwards JVM toolchain hints (``JAVA_HOME`` / ``JAVA_OPTS`` / ``GRADLE_USER_HOME`` / ``MAVEN_OPTS``) and the operator-override ``ORT_*`` band, while stripping worker secrets and credential-named ``ORT_*`` keys (``ORT_GITHUB_TOKEN`` / ``ORT_NEXUS_PASSWORD`` etc.). A new ``test_ort_env_scrub.py`` exercises the real-binary code path by monkeypatching ``shutil.which`` + ``subprocess.run`` and pinning two regressions: secrets are stripped, and credential-named ``ORT_*`` keys are dropped even though the prefix band is allow-listed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
All four ecosystem fetchers (Maven Central, PyPI, crates.io, pkg.go.dev) now build their default ``httpx.Client`` with ``follow_redirects=False`` (security-reviewer L4, chore PR #5 handoff §1.2). Each registry serves the licence metadata from a single authoritative origin, so a 3xx response is never a legitimate signal — a phishing host inserted via MITM, a misconfigured mirror, or an attacker-poisoned DNS response would otherwise be silently chased. ``base.request_with_retry`` grows an explicit 3xx branch that emits ``license_fetch_unexpected_redirect`` (warning-level, includes the truncated ``Location`` header) and returns ``None`` so the caller registers a negative cache entry. The attacker host is never contacted. Each fetcher gains two unit tests: * a behaviour pin — a 3xx response with a ``Location: attacker.example`` header returns ``None`` and the second URL is never requested; * a contract pin — the production-default ``_client(...)`` factory has ``follow_redirects is False``. Test injections are unaffected: callers can still pass an ``httpx.Client(follow_redirects=True)`` for live integration tests that legitimately need to chase redirects. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…rer/cookie Add ``"session"`` / ``"bearer"`` / ``"cookie"`` to ``_CREDENTIAL_DENY_SUBSTRINGS`` so the prefix-band forwarders drop ``*_SESSION`` / ``*_SESSIONID`` / ``*_BEARER`` / ``*_COOKIE`` env variables that cdxgen / ORT plugins occasionally accept for registry login. Bare ``"key"`` is intentionally not added: it would match ``npm_package_keywords`` (npm script context, benign). The existing ``"private_key"`` / ``"privatekey"`` / ``"apikey"`` / ``"api_key"`` substrings already cover the real threat shapes for crypto material. Refs: security-reviewer L1 (chore PR #6 review). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
6 tasks
haksungjang
added a commit
that referenced
this pull request
May 9, 2026
Combine option A (automated CI verification on Ubuntu 22.04 runner) with option B (operator-facing manual checklist) and add the missing static analyser gate for shell scripts. - scripts/install.sh: --no-prompt mode (INSTALL_HOST / INSTALL_ADMIN_EMAIL / INSTALL_ADMIN_PASSWORD / INSTALL_SECRET_KEY / INSTALL_REUSE_ENV env vars). Interactive flow unchanged. - .github/workflows/install-uat.yml: workflow_dispatch + weekly cron. Pins docker-compose V1 1.29.2 (CLAUDE.md core rule #10), swaps to dev compose for the run, exercises install -> /health -> login + projects smoke -> backup -> restore round-trip -> post-restore /health. - .github/workflows/ci.yml: shellcheck job (--severity=warning, error + warning hard-fail; info advisory). Existing scripts pass. - scripts/upgrade.sh, scripts/release.sh: incidental shellcheck fixes (SC2034 unused loop var, SC2046 intentional word-split with disable). - docs-site: uat-checklist.md (EN + KO) registered in sidebar — 8-step cross-host backup/restore round-trip checklist for operators. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 23, 2026
haksungjang
added a commit
that referenced
this pull request
May 23, 2026
…acy, governance (#129) Prepare the project for open-source publication. Distribution (P0 — install was impossible without this): - add release.yml: multi-arch (amd64+arm64) ghcr.io publish of backend/worker/frontend on vX.Y.Z tags + workflow_dispatch (no :latest) - align docker-compose / .env.example / Helm to ghcr.io/trustedoss, overridable via IMAGE_REGISTRY - install.sh: Compose V2 fallback ($DC shim, V1 preferred — keeps rule #10) - no-clone curl install path + install-uat published-image-pull check Docs accuracy (reconcile with shipped code): - README/landing/intro: GA status, scancode (ORT removed), vulnerability PDF implemented (Excel/compliance PDF on roadmap), Jenkinsfile-example - user/admin guides: NOTICE text/markdown/html, ORT->scancode in workspace sizing, vulnerability PDF section - comparison page (vs commercial / DT-alone / SW360), EN+KO Project meta: - GOVERNANCE / MAINTAINERS / SUPPORT / CODEOWNERS / .editorconfig - ROADMAP.md + docs/post-ga-roadmap.md (post-GA milestones v2.0.1-v2.3) - landing product-screenshot showcase + og:image social card - gitignore agent worktrees + runtime backups Verified: docs-site build (en/ko), actionlint, shellcheck, docker-compose config. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
haksungjang
added a commit
that referenced
this pull request
May 24, 2026
…d seed (B2) (#147) * feat(eval): add lightweight 2 vCPU / 4 GB evaluation compose overlay (B2) Add docker-compose.eval.yml, layered on top of docker-compose.yml via -f, to stand the portal up on a small host for evaluation: - DT-less by design: no Dependency-Track bundle (no 4 GB JVM heap). The portal serves seeded/cached vulnerability data via the DT circuit breaker (OPEN). - Resource budget sized for 2 vCPU / 4 GB: per-service cpu/memory limits whose RAM sum (~3.4 GB) leaves headroom on a 4 GB host; CPU caps oversubscribe the two cores. - Single worker at --concurrency=1 (no parallel scans on eval). - Traefik runs plain HTTP only (no Let's Encrypt / 443) so no public DNS is needed; eval routers bind the `web` entrypoint. Prod security defaults in docker-compose.yml are only superseded for this layer, never weakened. - APP_ENV defaults to demo so seed_demo.py is allowed to run. Document EVAL_URL + DEMO_SUPER_ADMIN_PASSWORD in .env.example. Parses under docker-compose V1 (CLAUDE.md rule #10); images stay pinned (rule #9, no :latest); env read at runtime (rule #11); CORS stays an explicit allow-list (rule #13). No DT, all services restart: unless-stopped, only Traefik publishes host ports. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(eval): add one-command eval bring-up script scripts/eval-up.sh (B2) scripts/eval-up.sh takes the eval stack from clone to a populated dashboard in one command: 1. Pick the Compose binary (V1 docker-compose preferred; V2 plugin fallback for end-user hosts only). 2. Bootstrap .env from .env.example if absent — generate a SECRET_KEY, set APP_ENV=demo, pin CORS to EVAL_URL. Never clobbers an existing .env. 3. Bring up `-f docker-compose.yml -f docker-compose.eval.yml up -d` with --compatibility so the eval resource limits bind on a non-Swarm host. 4. Poll the B1 readiness gate GET /health/ready (200 == schema at Alembic HEAD); on miss, run a one-shot `alembic upgrade head` and re-poll. 5. Seed the idempotent demo dataset (scripts/seed_demo.py, APP_ENV=demo) and surface the generated super-admin password. 6. Print the URL + demo accounts. --no-prompt is CI-friendly (skips the interactive confirm). Separate from install.sh (the prod wizard) — shares only the .env it bootstraps. bash 4+, set -euo pipefail, shellcheck-clean (-e SC1091). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(eval): document the lightweight evaluation install (EN + KO) Add an "Evaluation install (lightweight, one command)" section to the docker-compose install guide and mirror it in the Korean locale: - 2 vCPU / 4 GB requirements vs. the 4 vCPU / 8 GB full stack. - The one command (./scripts/eval-up.sh) and what it does step by step. - Demo accounts table + password sourcing (DEMO_SUPER_ADMIN_PASSWORD). - Per-service resource budget table mapped to 2 vCPU / 4 GB. - DT-less explanation (circuit breaker OPEN + seeded PostgreSQL cache) and how to point at an external DT. - Eval-vs-production warning (scan worker sizing, no TLS / role separation). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
8 tasks
haksungjang
added a commit
that referenced
this pull request
May 26, 2026
Reproducer: open a project, click "Scan" once, click again before the first scan finishes. The DB partial unique index `ix_scans_project_active` already rejects the second INSERT, but the SPA let the user hit it (turning into a 409 toast instead of a no-op disabled button), and the 409 envelope carried no machine-checkable extension so the SPA couldn't render a targeted notice. BE: `_problem_for_scan_error` now adds `scan_already_in_progress=true` to the RFC 7807 envelope on `ScanInProgressConflict`. Test in `test_scans_api.py::test_concurrent_trigger_returns_409_problem` extended to pin the extension. FE: `ProjectDetailHeader` disables the "Scan" button when `activeScan` is non-null and surfaces a localized title attribute pointing the user at the existing "scan running" chip (which already re-opens the in-progress drawer). EN + KO translations added for the new `scan_already_active` key. The chip + click-to-reopen affordance was already in place from #29; this PR closes the obvious second entry point. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
4 tasks
haksungjang
added a commit
that referenced
this pull request
May 26, 2026
Sweep the user and admin guides (EN + KO) so the prose matches what shipped to main between W1 and the current ops-triage batch. The PR is doc-only — no apps/* changes. User guide - new user-guide/dashboard.md (EN + KO) — covers the post-login `/` landing page introduced by W1; sidebar entry as position 0. - projects.md: project-list meta row (`scan_count`/`release_count`/ `last_scan_at` from #30), detail-page tab table refreshed to the post-v2.3 layout (11 tabs incl. Releases/Reports/Source moved between Reports and Remediation per P2 #3), new Project info card section (P2 #1), new "Releases tab" section incl. ?scan= snapshot pin behaviour, new "Reports tab" section (#32 — generate cards + export history table), risk-score rewritten as the two-axis model (#34, Security + License); KO mirror in parallel. - scans.md: "only one scan at a time per project" callout + troubleshooting entry (P1 #10), per-stage log panel description (P2 #8b), Project-name column note on the global queue (P1 #5), Finalizing-spinner regression troubleshooting (P1 #11); KO mirror. - vulnerabilities.md: "Vulnerability data unavailable" banner section (#35), bulk-transition section with the `:bulk-transition` endpoint contract (#33b); KO mirror. - components-and-licenses.md: Components table columns updated to include Type (Direct/Transitive) and Usage (Required/Optional) with the matching toolbar filters and drawer meta (W2 #31); KO mirror. - approvals.md: list-row columns updated to the resolved component / project names + click-through links (P1 #6), batched IN(...) lookups note; KO mirror. Admin guide - users-and-teams.md: new "Recovering a deactivated super-admin" section that documents the ensure-active recovery path in scripts/create_super_admin.py (#14); KO mirror. Docs site - sidebars.ts: added user-guide/dashboard ahead of user-guide/projects. - KO projects.md: explicit anchor `{#build-gate-verdict-overview-tab}` added so the dashboard's KO link to the build-gate section resolves (the auto-derived Korean slug was the previous breakage point). - KO vulnerabilities.md: fixed pre-existing stale anchor target `#epss--악용-가능성` → `#epss--악용-확률`. Stale assets flagged (not removed) - Walkthrough video on projects.md still captures the pre-post-v2.3 four-tab layout. Marked in a `<!-- -->` comment; refresh after merge. - Projects-list and global-scans-queue screenshots predate the scan-meta row and Project-name column. Marked in `<!-- -->` comments; refresh after merge. Not covered (deferred until on main) - P2 #4 (Dashboard KPI deep links), P2 #3/#1/#2 (visible in feature branches but not yet merged to main). Verification - `npm run build` under docs-site exits 0 with no warnings on either locale. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
haksungjang
added a commit
that referenced
this pull request
Jul 3, 2026
…2) (#456) Add a cross-project search endpoint and wire Components + CVEs groups into the command palette. Results are team-scoped server-side through a single chokepoint, so another team's data never appears. Backend: - `GET /v1/search?q=&kinds=components,vulnerabilities` — ILIKE (escaped) over component name/purl and CVE id across the caller's accessible projects, each category deduped + capped at 20 (SQL LIMIT), min query length 2, results self-describing (project id/name/slug) for deep-linking. - New `core.authz.team_scope_filter(actor)` — the mandated list-scope chokepoint every fan-out subquery routes through: super_admin → all, member → `Project.team_id.in_(actor.team_ids)`, no-membership → matches nothing (fail-closed). Gated on `is_superuser` alone (not the derived role string) so no membership-role value can ever open the cross-tenant branch (security-review Low-2, defense-in-depth for all future fan-out surfaces). - Dedicated `SEARCH_RATE_LIMIT` (default 20/minute, per-actor) instead of the shared CI-poll budget, bounding the leading-wildcard scan the debounced palette drives (security-review Low-1). Frontend: - `searchApi.globalSearch` + CommandMenu Components/CVEs groups (debounced, ≥2 chars, TanStack Query). Component hits deep-link to `?tab=components&search=`, CVE hits to `?tab=vulnerabilities&search=`. Severity renders as a token dot + label. EN/KO strings. Security: Producer-Reviewer — security-reviewer PASS, tenant isolation confirmed end-to-end (0 Critical/High/Medium); the two Low findings are addressed above. The FE renders hits as text (React-escaped, no dangerouslySetInnerHTML). Tests: - Backend: team-isolation cross-leak-zero intersection test (actor A never sees actor B's components or CVEs, and vice-versa), super-admin sees all, min length, cap-20, `_escape_like` wildcard containment, kinds filter, 401. 15 tests; changed modules 100% covered. OpenAPI golden updated. - Frontend: CommandMenu vitest (below-threshold no-call, debounced call, component + CVE hits + deep-links). e2e `global_search.spec` drives ⌘K, asserts the Components/CVEs groups and a component deep-link. Docs: dashboard guide "Global search (⌘K)" (EN/KO) + CHANGELOG. Parity #10 → in-progress. Follow-up (non-blocking): a pg_trgm GIN index on the searched columns would make the ILIKE index-backed at very large scale (the rate budget bounds it for now).
haksungjang
added a commit
that referenced
this pull request
Jul 4, 2026
…pletion sweep (#458) Phase H-2 (#456) and Phase I (#457) merged, so flip parity #10 → closed and #12 → closed (code merged; the gate exercises live at the next real tag v0.13.0). Add a completion-sweep review-history line — Phases E–I all landed (#451/#452/#454/#455/#456/#457) plus the jsonata CVE suppression (#453) that unblocked the repo-wide image-scan gate. The only open gap left is #11 (SCANOSS, conditional Phase J).
haksungjang
added a commit
that referenced
this pull request
Jul 4, 2026
…rpolation (#465) The v0.13.0 gate still failed after the dev-mode fix (#464) — but only in CI, not in a local V2 run. Root-caused under the CI-identical docker-compose V1 1.29.2 binary: the production docker-compose.yml resolves the runtime DSN with a NESTED default, DATABASE_URL_APP: ${DATABASE_URL_APP:-${DATABASE_URL}} DATABASE_URL: ${DATABASE_URL_APP:-${DATABASE_URL}} and the gate (unlike install.sh) never sets DATABASE_URL_APP, so the `:-${DATABASE_URL}` fallback fires. docker-compose V1 1.29.2 — the version the gate pins (CLAUDE.md rule #10) — cannot parse the inner `${DATABASE_URL}`: its interpolator stops at the first `}`, so backend/worker/beat get the literal string `${DATABASE_URL}` (a non-DSN). alembic can't connect and the entrypoint emits the masked "database not reachable" crash-loop. V2 expands the nested form, which is why the earlier local V2 run passed while V1 CI failed. (Setting DATABASE_URL_APP in the gate env is NOT enough — V1 then leaves a stray trailing `}`, database `trustedoss}`.) Fix — override ONLY the two DSN keys for the three DB-touching services (backend/worker/beat) in the gate overlay with a single-level `${DATABASE_URL}` V1 resolves cleanly. Per-key environment merge preserves all other env. Gate-only overlay; the production base file and its L1 split are untouched. Verified with the exact CI binary (docker-compose-Linux-x86_64 1.29.2): `config` renders a clean DSN; a `down -v` cold boot of the published :0.13.0 images has the backend connect on attempt 1/10, all 35 migrations to HEAD, /health + /health/ready 200, all six services healthy, super_admin bootstrap, /auth/login (token), and GET /v1/projects all green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
integrations/_subprocess_env.pyand apply it to cdxgen + ORT. Worker secrets (DT_API_KEY/SECRET_KEY/DATABASE_URL/*_WEBHOOK_URL) no longer inherit into either subprocess, closing the hostile-clone exfil channel that chore PR feat: phase 3 pr #11 — vulnerabilities tab (CVE list + drawer + status workflow) #4's prep-only fix left open.follow_redirectson all four license fetcher default clients (Maven Central / PyPI / crates.io / pkg.go.dev). 3xx responses now loglicense_fetch_unexpected_redirect(warning + truncatedLocation) and returnNone(negative cache); the attacker host is never contacted.session/bearer/cookiesubstrings.No new domains / endpoints / schema changes.
Commits
security-reviewer Producer-Reviewer
Verdict: PASS (no conditions). 0 Critical / 0 High / 0 Medium / 1 Low / 3 Info. The Low (L1) was absorbed in
9156bf8. The 3 Info findings are documented invariants / future Phase 8 polish; backlog-only.Highlights:
_CDXGEN_EXTRA_ALLOWLIST+ 4 prefix bands cover every documented env consumer;npm_config__authToken/_auth/_passwordshapes are stripped via the substring deny.JAVA_HOME/JAVA_OPTS/_JAVA_OPTIONS/JDK_JAVA_OPTIONS/GRADLE_USER_HOME/GRADLE_OPTS/MAVEN_OPTS/MAVEN_HOME/M2_HOME) preserved;ORT_*_TOKEN/_PASSWORD/_KEYband stripped.--rules-file/--ort-fileprovenance verified worker-controlled (operator env or worker-image default; cdxgen output under workspace).Locationlog field — log-injection vector closed._subprocess_env.pyimports onlyos).Local regression
pytest tests/unit tests/integration— 881 pass, 8 skipped, modulo two pre-existingtest_licenses_apifixture-isolation flakes (chore PR feat: phase 3 pr #12 — licenses tab #5 §5 backlog, unrelated).ruff check— clean.mypy— clean (146 source files).test_subprocess_env.py, 6 intest_cdxgen_gradle_compat.py(env scrub), 2 intest_ort_env_scrub.py, 8 across the 4 license fetchers (redirect behaviour +follow_redirects is Falsecontract pin) = 53 new tests.Test plan
tests/unit/integrations/test_subprocess_env.py,test_cdxgen_gradle_compat.py::test_build_cdxgen_env_strips_worker_secrets,test_ort_env_scrub.py, license fetcher redirect tests).test_run_prep_passes_only_allowlisted_envstill green via the_scrubbed_envalias.🤖 Generated with Claude Code