Skip to content

Phase 4 PR #14 — Admin Panel: DT/Scans/Disk/Audit/Health (5 screens)#15

Merged
haksungjang merged 24 commits into
mainfrom
feature/phase4-pr14-admin-dt-scans-disk-audit-health
May 7, 2026
Merged

Phase 4 PR #14 — Admin Panel: DT/Scans/Disk/Audit/Health (5 screens)#15
haksungjang merged 24 commits into
mainfrom
feature/phase4-pr14-admin-dt-scans-disk-audit-health

Conversation

@haksungjang

Copy link
Copy Markdown
Contributor

Summary

Phase 4 의 두 번째 PR. docs/v2-execution-plan.md §3.5 의 4.4~4.8. PR #13 (Users/Teams) 의 admin 인프라 (라우터 가드 / AdminLayout / RFC 7807 / audit auto-emit / 하네스) 위에 5 화면 추가.

  • DT Connector (/admin/dt) — health 상태 + circuit-breaker + 고아 탐지/정리 트리거 + 강제 health probe.
  • Scan Queue (/admin/scans) — 4 탭 (queued/running/failed/all) + 강제 종료 + 30s polling + drawer.
  • Disk (/admin/disk) — workspace + DT volume + postgres + redis 사용량 + 임계치 (80/90%) 시각화.
  • Audit Log (/admin/audit) — 인라인 필터 (actor/target_table/action/from-to/q) + CSV streaming export + diff drawer.
  • System Health (/admin/health) — postgres / redis / celery / dt / disk / active_scans / last_24h_errors 종합.

10 신규 backend endpoint + 5 frontend 페이지 + EN/KO i18n + Playwright E2E 5/5 green + 5 신규 하네스.

Backend (9 commits)

  • feat(db): audit_logs target_table/action indexes — migration 0007 (forward-only)
  • feat(schemas): admin_ops Pydantic v2 — DT / scans / disk / audit / health
  • feat(tasks): dt_orphan_cleanup celery task — admin-triggered DT delete
  • feat(admin): DT connector status + orphan list + cleanup endpoints
  • feat(admin): scan queue list + cancel endpoints
  • feat(admin): disk usage telemetry endpoint
  • feat(admin): audit log search + CSV export endpoints
  • feat(admin): system health summary endpoint + router include
  • test(admin): unit + integration tests for PR #14 admin ops (138 신규 테스트, line coverage ≥ 84%)

Frontend (10 commits)

  • feat(i18n): admin DT/scans/disk/audit/health namespaces EN+KO
  • feat(admin-ui): DT connector page + status card + orphan cleanup
  • feat(admin-ui): scan queue page + drawer + cancel hook
  • feat(admin-ui): disk telemetry page + threshold visualization
  • feat(admin-ui): audit log page + CSV export + diff drawer
  • feat(admin-ui): system health dashboard
  • feat(admin-ui): admin layout sidebar + router for 5 new pages
  • test(admin-ui): unit coverage for DT/scans/disk/audit/health
  • feat(harness): AdminDT/Scans/Disk/Audit/Health harnesses + PortalPage entries
  • test(e2e): admin 5-scenario coverage (DT/Scans/Disk/Audit/Health)

CORS fix (1 commit)

  • fix(cors): expose Content-Disposition for admin audit CSV export — frontend agent 발견. CSV 다운로드 파일명이 axios 에서 읽히도록.

RFC 7807 Problem Details (신규)

DoD 체크리스트

  • 모든 endpoint = require_super_admin_or_404 (404 existence-hide).
  • 모든 쓰기 endpoint = audit auto-emit.
  • alembic upgrade head → 0007 (forward-only).
  • backend ruff / mypy clean.
  • backend coverage ≥ 80% (84% ~ 100% per module).
  • frontend npm lint 0 errors / typecheck clean.
  • frontend Vitest 389/389 pass — 91.74% lines / 83.42% branches.
  • Playwright E2E 5/5 green (21.2s) — DT / Scans / Disk / Audit / Health.
  • Playwright 하네스 5개 신규 + PortalPage 5 entry.
  • EN/KO 동시 5 namespace.
  • adversarial input parametrize 충실 (audit q / DT uuids / scan status / disk env).
  • main rebase 흡수 — chore PR chore(pipeline): UAT 패치 정식화 + 다중 언어 worker + pre-cdxgen prep + DT polling retry #8 의 audit PII sha256 hashing + errors.py redact + frontend problem.ts zod schema 가 PR chore: admin security follow-ups (F2~F12 from PR #13 review) #14 endpoint 에 자동 적용.
  • security-reviewer Producer-Reviewer 1라운드 — 본 세션에서 수행 중.

Test plan

  • backend pytest 1158 pass (12 pre-existing flakes orthogonal — test_project_detail_service.py 등 PURL fixture 충돌, main HEAD 에서도 재현).
  • frontend Vitest 389/389 pass.
  • Playwright E2E admin 5 시나리오 + PR Phase 4 PR #13 — Admin Panel: Users & Teams #13 admin_users_teams 회귀 4/4 pass.
  • alembic upgrade 0007 적용 가능.
  • CI 9/9 green — 본 push 후 자동 실행.

후속 / chore PR #8 backlog

chore PR #8 의 security-reviewer 1라운드 결과 (PASS-with-conditions, 0 Critical / 0 High / 3 Medium / 4 Low / 2 Info) 의 follow-up:

  • M1 — _redact_validation_errorsmsg / ctx 도 sanitize.
  • M2 — _seed(super_admin=True) 의 APP_ENV guard 를 _seed 내부로 이동 (defense-in-depth).
  • M3 — _problem_for_admin_team_error 에 F12 PII echo 경고 코멘트 추가.
  • L1~L4 — F4 PII pepper (Phase 8) / OAuth PII 컬럼 (Phase 6) / IntegrityError 의 constraint_name 식별 / validation_error invalid_role_assignment 사용처 정리.

→ chore PR #9 (다음 세션) 로 등재.

🤖 Generated with Claude Code

haksungjang and others added 24 commits May 7, 2026 23:10
Phase 4 PR #14 admin Audit Log search needs cheap lookups by target_table
+ action and an actor-scoped time-ordered compound. Add three indexes via
a forward-only migration; update the SQLAlchemy model so model + DB stay
in lockstep, and bump the alembic head test to 0007.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…alth

Phase 4 PR #14 schemas split out of schemas/admin.py (PR #13 owns the
identity surface). Closed Literal enums for ScanStatus / AuditTargetTable
/ BreakerState / HealthStatus reject out-of-set values at the boundary;
list-bound on OrphanCleanupRequest.dt_project_uuids (≤ 500) caps
DoS surface; AuditSearchQuery.q + .action validators reject NUL / CR / LF.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR #14 admin /v1/admin/dt/orphans/cleanup endpoint dispatches this task
to actually delete DT projects (the existing dt_orphan_cleaner Beat task
is read-only by design). Idempotent: 4xx from DT is treated as
already_gone, 5xx triggers Celery autoretry. The task releases the Redis
SETNX cleanup lock the admin service acquires on dispatch, and emits an
explicit AuditLog row per delete (Celery context has no actor for the
listener to attach to).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR #14 §4.4: four endpoints under /v1/admin/dt/. Status is breaker
snapshot + DT version with a 30s Redis cache (refresh-storm guard).
Orphan list pages an inline classification scan against DT — breaker
OPEN raises 503 dt_unreachable. Cleanup enqueues the Celery task behind
a Redis SETNX lock (409 dt_orphan_cleanup_in_progress on contention).
Force health-check runs run_health_check synchronously and emits an
audit row.

All four endpoints are gated by require_super_admin_or_404 (existence-
hide for non-admin) and use RFC 7807 Problem Details with snake_case
extensions + canonical type URIs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR #14 §4.5: /v1/admin/scans cross-team paginated queue with optional
status filter (closed Literal — out-of-set values 422 at boundary), and
/v1/admin/scans/{id}/cancel that revokes the Celery task (best-effort)
+ stamps status='cancelled'. SELECT FOR UPDATE on the scan row guards
the terminal-state race (two concurrent cancels both pass the guard).

Already-terminal scans return 409 scan_already_cancelled; missing scans
return 404 scan_not_found — both with canonical type URIs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR #14 §4.6: /v1/admin/disk reports four backends in one shot —
workspace + DT volume via shutil.disk_usage(); Postgres via
pg_database_size(current_database()); Redis via INFO memory.

Each probe is independent — a failure populates the per-item ``error``
field with status='down' instead of bubbling a 500. Thresholds default
to 80% warn / 90% critical and are env-tunable. Paths are read at call
time from WORKSPACE_HOST_PATH / DT_VOLUME_HOST_PATH per CLAUDE.md core
rule #11.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR #14 §4.7: /v1/admin/audit paginated JSON search with the closed-set
target_table whitelist + free-text JSONB substring (cast to text +
ILIKE on bound parameters). LEFT JOIN users so a row whose actor was
deleted (FK SET NULL) still renders cleanly with actor_email=null.

/v1/admin/audit/export.csv streams the same filtered slice with a
100k-row hard cap (413 audit_export_too_large). diff column intentionally
omitted from CSV — JSONB shapes vary and the JSON view in the UI is the
forensics surface. Filename embeds from/to so multi-export naming stays
distinguishable.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR #14 §4.8: /v1/admin/health aggregates 7 component probes —
postgres SELECT 1 / redis PING / celery control.ping (workers ≥ 1) /
DT breaker snapshot / disk worst-of-N / active scan count / 24h failed
scan count. Each probe is independent so a partial outage produces a
useful response instead of a 500.

The admin router aggregator now includes all five PR #14 sub-routers
(dt / scans / disk / audit / health) under the parent require_super_
admin_or_404 dependency. 23 admin routes total (was 13 in PR #13).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
138 new tests across 7 files covering:
  - DT service: status (cache + breaker-OPEN + 5xx + cache invalidation),
    list_orphans (empty / mixed / breaker-OPEN), enqueue_orphan_cleanup
    (lock contention + dispatch failure rolls back), force_health_check.
  - Disk service: filesystem probe (happy / OSError), redis probe (with /
    without maxmemory cap / connection error), threshold classification.
  - Audit service: filter coverage (actor / target_table / action /
    time-window / JSONB q), CSV export header + diff omission +
    100k-row cap, adversarial q / action / target_table parametrize.
  - Scan service: list join, status filter, queued -> cancelled,
    running -> revoke + cancelled, broker hiccup non-fatal, terminal-
    state 409, missing scan 404, audit row written.
  - Health service: each per-component probe (postgres / redis /
    celery / dt / disk / active_scans / last_24h_errors) + aggregated
    seven-component shape.
  - dt_orphan_cleanup task: happy path with audit emission, idempotent
    404 = already_gone, malformed UUID -> failed, empty-list = full
    catalog scan + delete.
  - 4-role auth matrix (anonymous=401 / developer=404 / team_admin=404 /
    super_admin=200) parametrized over every PR #14 endpoint.

Coverage on new modules: services 87-99%, routers 88-100%,
schemas 98%, task 84%. All ≥ 80% PR DoD threshold.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 4 PR #14 §4.4-4.8. Extends `apps/frontend/src/locales/{en,ko}/admin.json`
with five new admin namespaces (`admin.dt.*`, `admin.scans.*`, `admin.disk.*`,
`admin.audit.*`, `admin.health.*`) plus six error keys for the new Problem
extensions surfaced by the backend (`dt_unreachable`,
`dt_orphan_cleanup_in_progress`, `scan_already_cancelled`, `scan_not_found`,
`disk_path_unavailable`, `audit_export_too_large`).

Adds the same six extension keys to the strict whitelist in
`lib/problem.ts` so the toast key path stays graceful (RFC 7807 §3.2 +
PR #13's F10 schema-strengthening contract). The new keys also flow
through `adminErrorMessage` so the toast `data-toast-key` carries the
backend-emitted snake_case extension verbatim — locale-agnostic e2e
assertions keep working.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 4 PR #14 §4.4. Two-section page that talks to the four DT admin
endpoints (`/v1/admin/dt/{status,orphans,orphans/cleanup,health-check}`).

Top section is the status card — breaker badge (closed=green, half_open=
yellow, open=red), consecutive fail count, DT version, last check time,
last error, auto-restart flag. The "Force health probe" button drives
`POST /health-check` with audit emission server-side. Status query polls
every 30s (CLAUDE.md "Realtime").

Bottom section is the orphan-projects table — compact 40px row, per-row
checkbox, "Clean up selected" + "Clean up all orphans" actions. Both
actions surface an inline confirm strip (PR #13 pattern) before
dispatching the Celery task. The 503 / 409 Problem extensions
(`dt_unreachable`, `dt_orphan_cleanup_in_progress`) are translated to
the existing toast `data-toast-key` path.

Color is paired with an icon + i18n label so the breaker signal is not
colour-only (CLAUDE.md "Accessibility"). All filters / actions are
inline — no modal.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 4 PR #14 §4.5. Compact 40px-row table fed by `useAdminScans` over
`GET /v1/admin/scans`. Four tabs select the status filter — running,
queued, failed, all. Clicking a row opens `AdminScanDrawer` with the
project / team join, kind, started / finished / duration, error
message, and a "Cancel scan" affordance for queued / running rows.

`POST /v1/admin/scans/{id}/cancel` is wired through `useCancelAdminScan`
with prefix invalidation so every tab refreshes after a successful
cancel. Status-illegal transitions (already cancelled / succeeded /
failed) surface as a toast keyed by the `scan_already_cancelled`
Problem extension; missing scans surface `scan_not_found`. Both keys
flow through the strict zod whitelist in `lib/problem.ts` so the
graceful fallback path still applies.

The list polls every 30s (CLAUDE.md "Realtime") so an operator who
lands on the page sees the queue update without a manual refresh.
Per-scan WebSocket subscription is out of scope for this PR — the
existing `useScanWebSocket` is per-scan and the cross-team queue would
require a fan-out we don't ship yet.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 4 PR #14 §4.6. Four-card grid (workspace / dt_volume / postgres /
redis) wired to `GET /v1/admin/disk`. Each card shows used / total /
free in human-readable bytes plus a horizontal progress bar coloured
against the configured thresholds (80% warning → orange, 90% critical
→ red). The status badge pairs the colour with an i18n label so the
signal is not colour-only (CLAUDE.md "Accessibility").

Per-item `error` strings (mount missing, permission denied) surface as
inline `Alert` cells instead of the gauge so the operator can tell
"telemetry unavailable" apart from "0 bytes used". The page-level 503
`disk_path_unavailable` Problem extension translates through the
existing `adminErrorMessageKey` helper.

Polls every 30s (CLAUDE.md "Realtime").

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 4 PR #14 §4.7. Inline filter toolbar (no modal) over a compact
40px-row table fed by `useAdminAudit`. Filters mirror the Pydantic
validator surface (actor_user_id / target_table / action / from / to /
q with 300ms debounce). The `target_table` select uses the closed enum
from the schema so an attacker can't smuggle SQL fragments via the URL.

CSV export goes through `downloadAdminAuditCsv` — an authenticated axios
fetch with `responseType: blob`, then a programmatic anchor click. This
keeps the bearer token in the Authorization header (out of URL / browser
history / server access logs). The 413 `audit_export_too_large` Problem
extension surfaces as a toast keyed by the snake_case extension.

Diff display: the drawer detects `{"sha256": "<hex>"}` payloads (PII
columns hashed at write time per chore PR #8 F4) and renders a
truncated `sha256:abcd1234…` pill instead of the raw 64-char string.
The toolbar carries a `q_pii_hint` row reminding operators that
plain-text search will not match emails or names.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 4 PR #14 §4.8. Compact card grid (postgres / redis / celery / dt /
disk / active_scans / last_24h_errors) wired to `GET /v1/admin/health`.
Each card pairs the per-component status badge (green / yellow / red)
with an i18n label so the colour signal is not the only cue. Optional
numeric `value` (worker count, active-scan count, error count) renders
as a footer row inside the card.

Polls every 30s. There is no mutation surface — operators use the
per-component pages (DT / Scans / Disk) for actions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 4 PR #14 §4.9. Adds five sidebar entries (DT Connector, Scan
Queue, Disk, Audit Log, System Health) to `AdminLayout`'s NAV_ITEMS
list and registers the matching nested routes (/admin/dt, /admin/scans,
/admin/disk, /admin/audit, /admin/health) in `router.tsx`. The
existence-hide guard at the layout level + the `require_super_admin_or_404`
gate at the router level keep the defense-in-depth intact for the new
surfaces.

Icons use lucide-react's neutral `Network` / `ListChecks` / `HardDrive`
/ `ClipboardList` / `Activity` so the visual hierarchy stays consistent
with PR #13's `Users` / `Building2`. Each NAV_ITEM entry carries a
`data-testid` (`admin-nav-{slug}`) so e2e specs can drive sidebar
navigation without depending on translated labels.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 4 PR #14. Vitest + Testing Library coverage for the five operational
admin pages plus their api glue:

  - tests/unit/features/admin/dt/AdminDTPage.test.tsx — 8 tests including
    breaker tone mapping (closed/half_open/open → ok/degraded/down),
    cleanup confirm-strip flow, force-probe success toast.
  - tests/unit/features/admin/dt/useAdminDT.test.ts — api glue + query keys.
  - tests/unit/features/admin/scans/AdminScansPage.test.tsx — 8 tests
    including tab → status-filter wiring, drawer open, cancel mutation,
    scan_already_cancelled toast key.
  - tests/unit/features/admin/scans/useAdminScans.test.ts — api + key.
  - tests/unit/features/admin/disk/AdminDiskPage.test.tsx — 6 tests
    including threshold→status mapping, per-item error alert, used_pct
    clamp, page-level error alert.
  - tests/unit/features/admin/disk/useAdminDisk.test.ts — api + key.
  - tests/unit/features/admin/audit/AdminAuditPage.test.tsx — 8 tests
    including 300ms q debounce, target_table filter, sha256 fingerprint
    pill rendering, CSV export anchor click, audit_export_too_large
    toast key.
  - tests/unit/features/admin/audit/useAdminAudit.test.ts — api +
    Content-Disposition filename parsing.
  - tests/unit/features/admin/health/AdminHealthPage.test.tsx — 5 tests
    including per-component status data attributes + value rendering.
  - tests/unit/features/admin/health/useAdminHealth.test.ts — api + key.

Extends `tests/unit/lib/problem.test.ts` with whitelist + parser tests
for the six new admin operational extension keys (`dt_unreachable`,
`dt_orphan_cleanup_in_progress`, `scan_already_cancelled`,
`scan_not_found`, `disk_path_unavailable`, `audit_export_too_large`).
Extends `tests/unit/features/admin/adminErrorMessage.test.ts` with the
matching it.each entries.

Suite total: 50 files / 389 tests. Frontend line coverage 91.72%
(branches 83.41%, functions 80.4%) — above the 80%/70% thresholds.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… entries

Phase 4 PR #14. Five new Playwright harness classes — one per operational
admin surface — and matching `gotoAdmin{DT,Scans,Disk,Audit,Health}`
entry points on PortalPage.

Each harness keeps the PR #13 selector philosophy: ``data-testid`` +
``data-*`` attributes only, never translated text. Domain verbs:

  - AdminDTHarness — getBreakerState / getBreakerTone / forceHealthProbe
    / cleanupAll / cleanupSelected / expectErrorAlert(`dt_unreachable`,
    `dt_orphan_cleanup_in_progress`).
  - AdminScansHarness — selectTab / openScanDrawer / openFirstRowDrawer
    / cancelOpenScan / getRowStatus + `scan_already_cancelled` /
    `scan_not_found` error keys.
  - AdminDiskHarness — getCardStatus / getCardUsedPct / refresh.
  - AdminAuditHarness — filterByTargetTable / searchDiff (debounced via
    aria-busy poll) / openFirstRowDrawer / exportCsv (returns Playwright
    Download handle) + `csv_started` / `audit_export_too_large`.
  - AdminHealthHarness — getComponentStatus / getComponentNames / refresh.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 4 PR #14 §6.3. Five scenarios drive the operational admin surfaces
against the live docker-compose dev stack:

  1) DT page — status badge mounts with one of the three breaker states.
     Force-probe button drives `POST /v1/admin/dt/health-check`; the
     post-condition tolerates either the `health_checked` success toast
     (DT reachable) or the `dt_unreachable` error toast (breaker OPEN).
     Then drives the audit page to confirm the route is reachable.
  2) Scan Queue — pivots to the "all" tab so the seeded `succeeded` scan
     appears, opens the first row's drawer, asserts the drawer's status
     badge carries one of the five legal `data-status` values (scoped to
     the drawer locator to avoid the row-level badge collision).
  3) Disk page — workspace + postgres cards render with one of the three
     legal `data-status` values; dt_volume / redis are tolerated as
     stack-dependent.
  4) Audit log — filters by target_table=users, then triggers the CSV
     export. Captures the Playwright `download` event, asserts the file
     is a .csv (CORS in the dev stack does not expose Content-Disposition
     to the SPA so axios falls back to the default filename), and the
     success toast is posted.
  5) System Health — every emitted component card carries one of the
     three legal status values; postgres + redis + active_scans are
     load-bearing and asserted by name.

Suite: 5/5 green in 20.8s. Selectors live in the per-page harnesses
so EN/KO renders pass the same scenarios.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The admin audit page streams CSV via a browser download with a
filename-bearing `Content-Disposition: attachment; filename=audit_export_<from>_<to>.csv`
header. CORSMiddleware did not list `content-disposition` in
`expose_headers`, so axios on the SPA could not read the header — the
browser fell back to a synthetic `audit_export.csv` name.

Discovered during PR #14 frontend agent's E2E setup; the test accepts
the fallback (`*.csv`) but the operator-friendly filename only reaches
the disk dialog after this change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
security-reviewer G1 (High). _csv_cell now prepends an apostrophe to any
cell whose first character is `= + - @ \t \r`, per OWASP CSV-injection
cheat-sheet. Without this, an audit row whose action / target_id /
request_id (operator-controlled columns) starts with `=` was executed
as a formula when the export was opened in Excel / LibreOffice / Sheets,
giving the attacker DDE / shell-escape against the super-admin's
workstation.

The producer's previous comment in _csv_cell deferred this hardening to
a follow-up because the consumer is super-admin gated. The reviewer
correctly observed that operator-controlled inputs (X-Request-ID
header, future webhook actions) flow into these columns and the
super-admin's workstation is exactly the high-value target an attacker
would chase.

Adds 12 regression cases:
  - test_csv_cell_escapes_dangerous_prefix_with_apostrophe (parametrize 7)
  - test_csv_cell_leaves_safe_values_unchanged (parametrize 4)
  - test_stream_audit_csv_escapes_formula_in_request_id (e2e via stream)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…s + redis close untyped

CI typecheck (backend) caught 9 mypy errors that the dev container's
looser config did not flag:

  - tasks.dt_orphan_cleanup imports redis at module scope; tests
    monkey-patch task_module.redis.Redis.from_url, but mypy strict
    requires the attribute be explicitly exported. Added redis to __all__.
  - test_dt_orphan_cleanup.py:210 _ListingClient.list_projects had a
    narrower keyword-only signature than _FakeDTClient.list_projects's
    **_kw. Realigned the override and silenced the bridging case with a
    type: ignore[override] consistent with the existing fake-client
    pattern.
  - test_admin_audit_service.py:86 _seed_audit_row diff: dict | None
    needed dict[str, Any] | None for mypy --strict.
  - test_admin_scan_service.py:89 _FakeControl.calls: list[dict] needed
    list[dict[str, Any]] for the same reason.
  - test_admin_ops_api.py:230,262 redis.Redis.close() is unannotated;
    flagged with no-untyped-call. Added type: ignore at both sites.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
본 세션 (2026-05-08) 두 PR 완료 핸드오프:
  - 2026-05-08-phase4-pr14-admin-dt-scans-disk-audit-health.md
  - 2026-05-08-chore-pr8-admin-security-followups.md

다음 세션 prompt:
  - _next-session-prompt-phase4-pr15-component-approval-workflow.md
    Phase 4 의 마지막 PR. v2-execution-plan §3.5 의 4.9 + 4.10.

archive (본 세션 진입 prompt 2개):
  - _next-session-prompt-phase4-pr14-admin-dt-scans-disk-audit-health.md
  - _next-session-prompt-phase4-pr14-plus-chore-pr8.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@haksungjang haksungjang merged commit 688bfed into main May 7, 2026
@haksungjang haksungjang deleted the feature/phase4-pr14-admin-dt-scans-disk-audit-health branch May 7, 2026 15:58
haksungjang added a commit that referenced this pull request May 8, 2026
* chore: admin security follow-ups (M1~G10 from PR #14/#15 review)

M1: sanitize msg+ctx fields in validation error redaction (CWE-209)
M2: refuse super-admin seed outside safe env (defence-in-depth)
M3: PII echo warning in admin team error helper
L1: remove dead problem extension keys (invalid_role_assignment, validation_error, disk_path_unavailable)
L2: use diag.constraint_name for FK detection, text-match as fallback
G2: skip lock release on Celery Retry to prevent CWE-362 race
G3: populate request_id/ip/user_agent in DT audit rows
G4: strip credentials from exception messages in disk/health/DT services
G5: escape ILIKE wildcards (%, _, \) in audit log search
G6: reject empty dt_project_uuids with 400 to prevent mass-delete footgun
G10: extend orphan cleanup lock TTL to 3600s for large DT catalogs
Unit tests: parametrized strip-credentials, ILIKE escape, error-handler sanitization

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address security-reviewer findings (G2/G4/G6/L1)

G2: fix lock-release race — check DTUnavailable (not Retry) in finally
    block. With autoretry_for, sys.exc_info()[1] is DTUnavailable when
    the finally runs; Retry is raised by the wrapper after finally. The
    previous isinstance(Retry) check never fired, leaving the lock window
    open for concurrent cleanups.

G4: fix _strip_credentials regex for empty-username Redis URLs
    (redis://:pass@host). Changed [^:@/\s]+ → [^:@/\s]* so the pattern
    matches the zero-char username case. Adds rediss:// TLS test case.

G6: remove reference to nonexistent cleanup-all endpoint from 400 detail.
    The error now directs callers to GET /v1/admin/dt/orphans first.

L1: remove disk_path_unavailable and invalid_role_assignment dead mappings
    from adminErrorMessage.ts (backend never emits them). Remove
    disk_path_unavailable from en/ko locale files and clean up the JSDoc
    comment in adminDiskApi.ts.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(test): update KNOWN_PROBLEM_EXTENSION_KEYS pin for L1 removals

invalid_role_assignment and disk_path_unavailable were removed from
problem.ts whitelist in the G2/G4/G6/L1 fix commit; update the module-
export pin tests to reflect the current whitelist.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(test): remove invalid_role_assignment + disk_path_unavailable from admin error tests

L1 fix removed these two dead extension keys from adminErrorMessage.ts and
problem.ts; the adminErrorMessage test suite was not updated at the same
time. Remove the corresponding it.each rows so CI passes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(test): update orphan cleanup tests for G6 non-empty UUID validation

G6 fix rejects dt_project_uuids=[] with 400. Two integration tests were
sending empty lists expecting 409 (lock conflict) and 202 (happy path).
Updated both to send a single valid UUID so the test exercises the
intended code path (lock check / task dispatch) correctly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(lint): sort imports in admin_dt_service.py (ruff I001)

_strip_credentials import from services.admin_disk_service was inserted
between two integrations.dt imports; moved to end of local-import block
in alphabetical order.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
haksungjang added a commit that referenced this pull request May 8, 2026
## Summary
- 신규 component_approvals 테이블 + ENUM (approval_status) + Alembic 0008
- ComponentApproval 모델 (ETag 패턴: version 컬럼)
- 5개 엔드포인트: GET /v1/approvals (list), GET /:id (detail+ETag), POST (create), PATCH /:id/transition (If-Match), DELETE
- 상태 전이 매트릭스: pending → under_review/rejected → approved/rejected (terminal)
- 권한 매트릭스: developer (자신 팀 보기/생성), team_admin (자신 팀 전이), super_admin (cross-team)
- Existence-hide 패턴 (비-팀원 = 404)
- Optimistic concurrency (SELECT FOR UPDATE + ETag echo)
- Domain Problem extensions: approval_already_open / invalid_transition / etag_mismatch / terminal_state
- Audit emit (생성/전이/삭제 모두)
- 프론트: ApprovalsPage (필터/테이블/페이지네이션) + ApprovalsDrawer (조건부 액션 버튼)
- EN/KO i18n approvals namespace 동시 추가

## Test plan
- [x] backend ruff clean
- [x] frontend npx tsc --noEmit clean
- [x] frontend vitest 419 tests pass (54 files), coverage 91%
- [x] 백엔드 16+13 unit tests
- [ ] CI 9/9 green (push 후 확인)

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
haksungjang added a commit that referenced this pull request May 8, 2026
* Phase 4 PR #15 — 컴포넌트 승인 워크플로우 (/approvals)

## Summary
- 신규 component_approvals 테이블 + ENUM (approval_status) + Alembic 0008
- ComponentApproval 모델 (ETag 패턴: version 컬럼)
- 5개 엔드포인트: GET /v1/approvals (list), GET /:id (detail+ETag), POST (create), PATCH /:id/transition (If-Match), DELETE
- 상태 전이 매트릭스: pending → under_review/rejected → approved/rejected (terminal)
- 권한 매트릭스: developer (자신 팀 보기/생성), team_admin (자신 팀 전이), super_admin (cross-team)
- Existence-hide 패턴 (비-팀원 = 404)
- Optimistic concurrency (SELECT FOR UPDATE + ETag echo)
- Domain Problem extensions: approval_already_open / invalid_transition / etag_mismatch / terminal_state
- Audit emit (생성/전이/삭제 모두)
- 프론트: ApprovalsPage (필터/테이블/페이지네이션) + ApprovalsDrawer (조건부 액션 버튼)
- EN/KO i18n approvals namespace 동시 추가

## Test plan
- [x] backend ruff clean
- [x] frontend npx tsc --noEmit clean
- [x] frontend vitest 419 tests pass (54 files), coverage 91%
- [x] 백엔드 16+13 unit tests
- [ ] CI 9/9 green (push 후 확인)

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(test): correct CurrentUser import + Component fixture + per-test engine

- CurrentUser annotation referenced before import (mypy name-defined error);
  hoisted import to module level.
- Component model has no `version` or `kind` columns; updated fixture to use
  the actual schema (purl, name, package_type).
- Module-scoped async db_session_factory caused asyncpg "another operation
  in progress" cross-loop errors under pytest-asyncio's function-scoped loop;
  changed to function-scope with a session-once alembic gate.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(test): bump alembic head assertion + add Home/ScansPage stub tests

- test_alembic_current_reports_head_revision: 0007 → 0008 after the
  component_approvals migration landed.
- Frontend functions coverage dropped to 79.86% because Step 1 introduced
  Home.tsx (Navigate) and ScansPage.tsx (ComingSoon stub) without tests.
  Added minimal smoke tests for both → coverage back to 80.33%.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
haksungjang added a commit that referenced this pull request May 26, 2026
Sweep the user and admin guides (EN + KO) so the prose matches what
shipped to main between W1 and the current ops-triage batch. The PR
is doc-only — no apps/* changes.

User guide
- new user-guide/dashboard.md (EN + KO) — covers the post-login `/`
  landing page introduced by W1; sidebar entry as position 0.
- projects.md: project-list meta row (`scan_count`/`release_count`/
  `last_scan_at` from #30), detail-page tab table refreshed to the
  post-v2.3 layout (11 tabs incl. Releases/Reports/Source moved
  between Reports and Remediation per P2 #3), new Project info card
  section (P2 #1), new "Releases tab" section incl. ?scan= snapshot
  pin behaviour, new "Reports tab" section (#32 — generate cards +
  export history table), risk-score rewritten as the two-axis model
  (#34, Security + License); KO mirror in parallel.
- scans.md: "only one scan at a time per project" callout +
  troubleshooting entry (P1 #10), per-stage log panel description
  (P2 #8b), Project-name column note on the global queue (P1 #5),
  Finalizing-spinner regression troubleshooting (P1 #11); KO mirror.
- vulnerabilities.md: "Vulnerability data unavailable" banner
  section (#35), bulk-transition section with the `:bulk-transition`
  endpoint contract (#33b); KO mirror.
- components-and-licenses.md: Components table columns updated to
  include Type (Direct/Transitive) and Usage (Required/Optional)
  with the matching toolbar filters and drawer meta (W2 #31); KO
  mirror.
- approvals.md: list-row columns updated to the resolved component /
  project names + click-through links (P1 #6), batched IN(...)
  lookups note; KO mirror.

Admin guide
- users-and-teams.md: new "Recovering a deactivated super-admin"
  section that documents the ensure-active recovery path in
  scripts/create_super_admin.py (#14); KO mirror.

Docs site
- sidebars.ts: added user-guide/dashboard ahead of user-guide/projects.
- KO projects.md: explicit anchor `{#build-gate-verdict-overview-tab}`
  added so the dashboard's KO link to the build-gate section resolves
  (the auto-derived Korean slug was the previous breakage point).
- KO vulnerabilities.md: fixed pre-existing stale anchor target
  `#epss--악용-가능성` → `#epss--악용-확률`.

Stale assets flagged (not removed)
- Walkthrough video on projects.md still captures the pre-post-v2.3
  four-tab layout. Marked in a `<!-- -->` comment; refresh after
  merge.
- Projects-list and global-scans-queue screenshots predate the
  scan-meta row and Project-name column. Marked in `<!-- -->`
  comments; refresh after merge.

Not covered (deferred until on main)
- P2 #4 (Dashboard KPI deep links), P2 #3/#1/#2 (visible in feature
  branches but not yet merged to main).

Verification
- `npm run build` under docs-site exits 0 with no warnings on either
  locale.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
haksungjang added a commit that referenced this pull request May 28, 2026
W10-E: mirror the W10-A→W10-D vulnerabilities dual-surface pattern to the
component domain. Same 5-step refactor — body extraction, full-page route,
bi-directional affordance. NEXT STEPS sidebar deferred (see PR body).

Mirror stages
=============
1. (W10-A mirror) ComponentDetailBody — surface-agnostic body extracted
   from ComponentDrawer. Drawer now owns only the Sheet shell + load
   state + affordance; the body renders identically in either surface.
2. (W10-B mirror) ComponentDetailPage — new full-page route at
   `/projects/:projectId/components/:componentId`. Breadcrumb +
   "Back to Components" link + shared body. Backward-compat: the
   `?drawer=` deep-link still opens the drawer.
3. (W10-C mirror) Drawer → page affordance. Drawer header gains an
   "Open in full view" button gated on `projectId + componentId`. Click
   closes the drawer, navigates to the page, and stashes the originating
   URL in `location.state.from` so "Back to Components" returns the user
   to the exact filter/page they came from. Cross-project + protocol-
   relative state.from values are silently rejected (open-redirect guard).
4. (W10-D mirror) NEXT STEPS sidebar — *deferred this PR*. See below.
5. Tests + gates — all green.

NEXT STEPS sidebar — deferred
=============================
Investigated both natural candidates:

  - Approval workflow: `models/component_approval.py` + `/v1/approvals`
    exist (Phase 4 PR #15). However the list endpoint accepts only
    `status / team_id / requested_by_user_id / from / to`. There is no
    `?component_id=&project_id=` filter to look up the open approval for
    the current component. Adding it requires a service-layer change.
  - Remediation: no component-level upgrade recommendation exists in the
    backend (Trivy upgrades are per-finding, not per-component).

Per the W10-E scope rule "no new backend models" + "single PR", neither
half can be wired up without touching backend. The sidebar lands in a
follow-up that exposes the approval lookup filter. This PR ships the
page in single-column layout; adding the sidebar later is purely
additive.

Backward-compat
===============
- `?tab=components&drawer=<id>` continues to open the drawer.
- ComponentDrawer.test.tsx — 8 existing cases unchanged, 4 W10-E cases added.
- No changes to vulnerabilities-domain components.

Files
=====
- ComponentDetailBody.tsx (+223): meta panel + vulns + raw_data accordion.
- ComponentDrawer.tsx (now 159 lines): Sheet shell + affordance only.
- ComponentDetailPage.tsx (+273): full-page surface.
- ComponentsTab.tsx (+1): pass `projectId` prop to drawer for affordance.
- router.tsx (+11): new route declaration.
- en/ko project_detail.json (+16 each): components.detail_page.* +
  drawer.open_full_view.
- ComponentDetailBody.test.tsx (+156): 5 cases.
- ComponentDetailPage.test.tsx (+303): 9 cases incl. cross-project +
  protocol-relative open-redirect guards.
- ComponentDrawer.test.tsx (+115): 4 W10-E affordance cases.

i18n keys (EN + KO mirrored same PR)
====================================
- components.detail_page.breadcrumb_aria
- components.detail_page.breadcrumb.projects
- components.detail_page.breadcrumb.components
- components.detail_page.back_link
- components.detail_page.title_loading
- components.detail_page.not_found
- components.detail_page.errors.{not_found,forbidden,unknown}
- drawer.open_full_view

Pre-merge gates
===============
- typecheck   green
- lint        green (no new warnings)
- i18n:check  OK — locales in sync
- vitest      991 / 991 pass (116 files)

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
haksungjang added a commit that referenced this pull request Jul 11, 2026
…agging (BomLens parity #15-#17) (#482)

* feat(scan): filter cdxgen SBOMs to the runtime scope before persist and matching

cdxgen keeps every resolved node, so a deployed app's SBOM also carried its
test/provided/dev toolchain (junit, lombok, devDependencies) as if it shipped
— inflating CVE counts, license obligations and NOTICE content. Mirror the
BomLens build-prep post-filters (#331/#335/#341) as a worker-side stage:

- integrations/sbom_scope_filter.py: pure post-filter. Maven drops cdxgen
  scope optional/excluded behind a hasScopes guard (an unscoped SBOM is left
  whole so recall never regresses). Node diverges from BomLens deliberately:
  instead of an npm re-resolve subprocess, drop pkg:npm components the
  already-parsed package-lock.json positively classifies dev (hasDev guard,
  keep-if-unknown for purls the root lockfile does not cover). Shared tail
  prunes dependencies[] to kept refs and always preserves the root ref.
- Pipeline wiring runs copy-then-commit BEFORE artifact persist, cosign
  signing, SCANOSS and Trivy so every consumer sees one consistent filtered
  document; a failed disk rewrite degrades to filter-skipped with memory and
  disk still identical. Telemetry lands in scan_metadata.scope_filter and a
  trusca:scope_filter SBOM property; no new pipeline stage (sub-second step).
- Ingest path deliberately unfiltered: an uploaded SBOM is the supplier's
  declared truth (pinned by an AST regression test).
- Config: SCAN_SCOPE_FILTER_ENABLED / _MAVEN_ENABLED / _NODE_ENABLED, all
  default ON; only exact falsy tokens disable, so a typo fails open to the
  correct-by-default behaviour (inverse of SCANOSS_ENABLED, no egress here).
- Fixtures are real captured cdxgen 12.3.3 output (Spring Boot 3.2 Maven app:
  108 components with required/optional/excluded; Express app: 487 components
  + its 502-package lockfile, 377 dev-flagged).
- Docs EN/KO: runtime-scope filtering section, env vars, stage description;
  also refresh the stale WebSocket slug list (dt_upload/dt_findings era).

* feat(fe): surface the runtime-scope filter's excluded-components count

The scope filter (previous commit) drops dev/test components server-side —
without a visible trace, a re-scan that loses 300 components reads as data
loss. Surface the worker telemetry in the Components tab:

- useScanScopeFilter reads scan_metadata.scope_filter off the ordinary scan
  detail response (ScanPublic.metadata already carries the JSONB verbatim —
  no backend change). Scan resolution mirrors the tab's anchoring: pinned
  ?scan= wins, else the latest succeeded scan from the overview. The parser
  treats the blob as untrusted at every level and collapses all nothing-to-
  show shapes to null.
- ComponentsTab summary band gains an additive 'excluded N' note
  (data-testid components-scope-filter-note, per-ecosystem breakdown on the
  title attribute); EN/KO keys in the same PR.
- Seeds (seed_demo + seed_e2e_user) stamp deterministic scope_filter
  telemetry on their succeeded scans so the e2e spec asserts without a live
  worker run.

* feat(scan): survive Podfile repos and fill pods in from Podfile.lock

With a Podfile present and no pod CLI (the worker ships no Ruby/CocoaPods
toolchain), cdxgen's cocoapods cataloger throws a TypeError on the undefined
pod stdout and aborts the whole SBOM stage — a terminal CdxgenFailed for any
iOS repo. Mirror BomLens #347 in two halves:

- Crash guard: pass --exclude-type cocoapods when a Podfile exists within
  depth 3 (Pods/ copies ignored; bounded glob, not BomLens' unbounded find).
  Applied in both the in-process adapter (integrations/cdxgen.py) and the
  sidecar script (build_prep_source.sh), cross-referenced to stay in sync.
- Fill-in: integrations/cocoapods_lockfile.py hand-parses Podfile.lock's
  PODS: block (npm_lockfile discipline: never raises, MAX_PODS cap,
  name-to-ref guard so edges never point at phantom refs, self-edges
  dropped) and merges components + dependency graph into the SBOM before
  the scope filter, copy-then-commit like the filter so disk and memory
  never diverge. Deliberate divergence from BomLens: no syft — the PODS:
  block already carries names, pinned versions and edges. Subspecs
  (Moya/Core) percent-encode into the purl; dependency_scope stays NULL
  (Podfile.lock has no runtime/test signal).
- SPM: repos committing only Package.resolved now detect as swift, and the
  sidecar skips swift package resolve over a committed lockfile (it is the
  resolved truth; re-resolving hits the network). Verified offline: cdxgen
  12.3.3 without a swift toolchain parses Package.resolved directly (the
  captured real_cyclonedx_swift.json fixture is that verification run).
- Fixtures: BomLens' real pod-install Podfile.lock (Moya -> Moya/Core ->
  Alamofire — subspecs, trailing-colon parents, constraint-only entries).

* feat(scan): flag end-of-life components from a vendored endoflife.date snapshot

An EOL runtime or framework gets no upstream fixes — a later Critical has no
patch. That is a supply-chain risk axis distinct from CVEs, and TRUSCA had no
signal for it. Mirror BomLens #368 (enrich-eol.sh) as a catalog-layer
enrichment:

- EOL lives on component_versions (0038: six nullable columns + a partial
  index WHERE eol_state='eol') — the KEV shape: a fact about product+cycle
  shared across scans/projects, re-stampable when the snapshot moves. NULL =
  not a tracked product; the whitelist philosophy never guesses.
- services/eol: the purl->product map vendored VERBATIM from BomLens
  (byte-identity contract test against a captured copy; the %40 npm-scope
  encoding divergence is absorbed in the matcher, keeping the map exact) +
  a compact endoflife.date snapshot vendored into the repo (16 KB, 10
  products) so Docker builds stay network-free and air-gapped installs work
  out of the box. scripts/refresh_eol_snapshot.py (port of BomLens
  build-eol-index.py: 15s/request, 60s budget, failed products skipped)
  refreshes it per release; a map<->snapshot product-set contract test trips
  when the map changes without a refresh.
- Evaluation is the BomLens semantics table-tested: cycle derivation
  (leading numeric segments, major vs major.minor), bool eol as-is, ISO
  date vs today, everything else unknown. express-session must not match
  pkg:npm/express@ (over-match guard test).
- Stamping happens inside persist_sbom_components right after the catalog
  row upsert — one evaluator per persist call, changed-value-guarded
  (idempotent re-scan), covering the source-scan AND sbom-ingest paths for
  free. Never fatal: corrupt/missing dataset degrades to no stamping with
  one aggregate log line.
- Config: EOL_ENABLED default ON (offline, zero egress — contrast SCANOSS),
  EOL_SNAPSHOT_PATH for air-gapped snapshot mounts.
- Verified: alembic upgrade head applies 0038 on live PostgreSQL 17
  (columns + partial index present); full unit suite green.

API/FE surfacing and the refresh beat land in the next two changes.

* feat(fe,api): surface end-of-life verdicts — column, badge, filter, KPI

Expose the catalog-layer EOL data (previous commit) end to end:

- API: ComponentSummary carries eol_state/eol_date and the detail response
  the full verdict (product/cycle/date/source) — free columns on the
  existing ComponentVersion joins, no extra query. New ?eol=true|false
  facet rides the partial index; the overview gains eol_count (one
  count query gathered with the existing aggregation). OpenAPI snapshot
  regenerated for the new param.
- FE: EolBadge (KevBadge structural mirror — High tone, renders only for
  state 'eol', absence is the signal), EOL column in the components table,
  an 'EOL only' toolbar toggle mirrored to ?eol=true, an always-rendered
  drawer row that distinguishes untracked (blank) from unknown (tracked,
  cycle undecidable), and an Overview chip deep-linking to the filtered
  list. EN/KO keys in the same change.
- Contracts: the closed eol_state vocabulary now lives in three places
  (catalog tuple, wire Literals, FE mirror) — set-equality tests on both
  sides (test_catalog_contracts.py + catalogMirrors.test.ts) plus EN/KO
  label coverage.
- e2e: components_eol.spec.ts (badge / filter+URL / drawer / overview
  chip); the seed stamps a far-past EOL verdict on exactly the first
  seeded component so assertions never depend on the run date or the
  vendored snapshot's live contents.
- Docs EN/KO: end-of-life section in the components guide (blank-vs-unknown
  semantics), endoflife.date row in data-sources, EOL_* env vars.

* feat(ops): weekly EOL re-stamp beat + admin snapshot health panel

Close out the EOL feature's ops layer (KEV subsystem structural mirror):

- eol_sync_state (0039): the kev_sync_state single-row pattern plus one
  addition — the fetched dataset itself rides the row (JSONB, a few KB) so
  the re-stamp pass can prefer the freshest data without network.
- tasks/eol_catalog_refresh: two-half tick. The fetch half is env-gated
  (EOL_REFRESH_ENABLED, DEFAULT OFF — new third-party egress follows the
  SCANOSS fail-closed posture, not the KEV one; EOL dates churn quarterly
  and the per-release vendored snapshot bounds staleness) with a
  proportional sanity floor (at least half the mapped products) so a gutted
  sweep never displaces a good dataset. The re-stamp half ALWAYS runs —
  pure local: effective dataset = newer of (vendored, fetched), bounded
  whitelist-prefix SELECT, changed-value-guarded stamps, and a clear pass
  for rows the whitelist no longer covers (the per-scan hook enriches but
  never reconciles). Weekly beat Sunday 02:15 UTC, between the KEV and
  Trivy ticks.
- integrations/eol_feed: kev_feed discipline scaled to many small
  documents — per-product byte ceiling + skip-and-count, 60s whole-run
  budget, EolFeedUnavailable only when NOTHING fetched, host-only logging.
- Admin surface: GET /v1/admin/eol/health (super-admin, existence-hide)
  assembled by eol_health_service (kev_health mirror + effective-snapshot
  resolution); FE EolPanel with a 180-day stale escalation, stamped/cleared
  KPIs and an origin/refresh footer; EN/KO keys; panel unit tests.
- Docs EN/KO: admin health guide section (docs-uat marker on the endpoint),
  EOL_REFRESH_* env rows. OpenAPI snapshot regenerated.
- Also: mypy-strict cleanups in the Phase K/L test files surfaced by the
  full 'mypy .' run.

* docs(parity): close gaps #15-#17 (Phase K/L/M) and move the baseline to 5a62094

7th review cycle over BomLens 9cc37e0..5a62094 (v1.7.0 + 27 commits, 80
total): the runtime-scope filter series, EOL flagging and the iOS lockfile
work were judged as gaps #15-#17 and implemented in this same branch — all
closed. Baseline, local-path correction (~/projects/bomlens) and the review
history row updated. Follow-up backlog recorded: Android release-classpath
scope filter (needs an Android SDK in the worker image), surfacing Trivy's
image-level eosl flag for container scans.

Also carries two doc changes that were finished earlier but stranded
(docs-only PRs are BLOCKED by the ci.yml paths-ignore required checks):
the reachability-analysis coordination row and
docs/commercial-parity-candidates.md.

* fix(security): harden the K/L/M parsers per security review

Producer-Reviewer findings (Medium 3 / Low 3 / Info 2), all addressed:

- CocoaPods ReDoS (M): the PODS-block regexes backtrack quadratically on a
  crafted line (measured 16s at 80k chars). Guards, in order: 2 MiB file
  cap before reading, 4 KiB per-line cap before the regex, and the work
  bound now counts EXAMINED lines (a never-matching block cannot bypass
  MAX_PODS). Regression test pins the pathological line under 1s.
- eol_cycle VARCHAR(32) overflow (M): a 33+-digit crafted version segment
  used to overflow the column at flush time — OUTSIDE the per-component
  best-effort guard — aborting the whole persist. derive_cycle now treats
  over-long numeric segments (>15 chars) as underivable -> 'unknown'.
- eol_sync_state.snapshot bloat (M): per-field 256-char bound in the feed
  compactor (drop, never truncate — a truncated version would match the
  wrong cycle) plus a 256 KiB size gate on the assembled dataset before
  the JSONB UPSERT (skip as 'snapshot_too_large', last-good survives).
- EOL feed redirects (L): max_redirects=3 with the trust-extension
  rationale documented (endoflife.date fronts a redirecting CDN, so
  redirects can't be off entirely).
- Scope-filter audit trail (L): the purls of dropped components are now
  recorded (bounded sample of 200 + exact totals) in ScopeFilterResult and
  scan_metadata.scope_filter.dropped_refs — counts alone are not
  reviewable when the npm predicate trusts an attacker-controlled lockfile
  classification.
- Symlink guards (L): Podfile.lock and the Podfile-detection glob reject
  symlinks (cosign-module precedent).
- LIKE metacharacter escape + assert->explicit guard (Info).

* chore(security): apply the two non-blocking re-review residuals

- Cap each recorded dropped ref at 512 chars so 200 hostile mega-purls
  cannot inflate the scan_metadata JSONB row.
- Correct the _podfile_present comment: the symlink guard covers the leaf
  only (glob traverses directory symlinks) — acceptable because the flag
  decides a CLI argument, not a read path; the read guard lives in
  read_podfile_lock.

* fix(scan,api): findings from full-stack verification of Phases K/L/M

Live-stack verification (docker-compose dev + Playwright + real worker
scans of public repos) surfaced three defects the unit suites structurally
could not see:

- Hand-built responses dropped the new fields (third instance of this
  class): the component drawer built ComponentDetailResponse without the
  eol_* block and the overview built ProjectOverviewResponse without
  eol_count — Pydantic silently defaulted both, so the DB and the list
  carried the verdict while the drawer/overview read null (caught by
  components_eol e2e E3/E4). Fixed, and a new AST completeness contract
  (test_handbuilt_response_completeness.py) requires a keyword for every
  model field in both constructions so the fourth instance fails in CI.
- Root-keyed consumers missed the git-clone subdirectory: _fetch_source
  returns workspace/source while the git path clones into source/repo, so
  the prep lockfile generators, the scope filter's and CocoaPods fill-in's
  lockfile reads, persist's W4-D npm scope enrichment and language
  detection all silently no-opped on every git scan (zip extractions,
  which land at source/ itself, were fine — which is why the fixture e2e
  never saw it). _resolve_project_root now hands them the repo root.
  Verified live: a chalk.git scan now drops 648 lockfile-dev components
  (47 kept) with the audit refs recorded; before the fix the same scan
  persisted all 695 with no telemetry.
- components_scope_filter.spec.ts seeded with the default component
  prefix, which collides with earlier runs' catalog purls (component-mode
  purls are not run-suffixed) — the seed failed on every run after the
  first. Now mints a per-run prefix like the sibling EOL spec.

Also verified live: maven-simple (all test-scope, no required) is left
whole by the hasScopes guard exactly as designed, with Usage=Optional
visible; the iOS guard survives a real cdxgen 12.3.3 run against a
Podfile tree in the worker and fills 3 pods + the subspec edge back in.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant