diff --git a/TRACKER.md b/TRACKER.md index f9bad4c..bee08fb 100644 --- a/TRACKER.md +++ b/TRACKER.md @@ -5,8 +5,8 @@ ## Status - **Current phase:** 6 — Lifecycle, scale, hardening (6F field-hardening done bar the data-blocked 6F.6 detection half) -- **Next step:** 6.5 generated/vendored classification & PII policy, then 6.4 version skew, 6.1 incremental re-scan, 6.2 scale (bench repo). 6.3 determinism landed first as it de-risks the rest (eval now hard-gates two-run byte-identity). Publish 0.4.1 to the tester still pending (needs npm creds). -- **Done:** 0.1–0.4, 1.1–1.6, 2.1–2.5, 3.1–3.6, 4.1–4.6, 5.1–5.7, 6F.1–6F.5, 6F.7–6F.10 + 6F.6 (defensive half), **6.3** · **0.4.1 prepared** (all packages bumped, changelog + version strings, `npm pack` → ui-lineage-0.4.1.tgz verified: both bins, no @coderadar leak; publish to npm pending). 0.4.0 tagged/merged earlier. **Self-validated on Grafana frontend** (6,461 files, 15,334 nodes / 18,367 edges in 72 s): 55 RTK-query data sources, 32 routes, 1,009 coverage edges, gibberish declines — all previously 0/1 in the field run. +- **Next step:** 6.4 version skew, then 6.1 incremental re-scan, 6.2 scale (bench repo). 6.3 determinism + 6.5 generated-classification/PII landed first (6.3 de-risks the rest via a two-run byte-identity gate; 6.5 closes the D5/G7 gate criteria). Publish 0.4.1 to the tester still pending (needs npm creds). +- **Done:** 0.1–0.4, 1.1–1.6, 2.1–2.5, 3.1–3.6, 4.1–4.6, 5.1–5.7, 6F.1–6F.5, 6F.7–6F.10 + 6F.6 (defensive half), **6.3, 6.5** · **0.4.1 prepared** (all packages bumped, changelog + version strings, `npm pack` → ui-lineage-0.4.1.tgz verified: both bins, no @coderadar leak; publish to npm pending). 0.4.0 tagged/merged earlier. **Self-validated on Grafana frontend** (6,461 files, 15,334 nodes / 18,367 edges in 72 s): 55 RTK-query data sources, 32 routes, 1,009 coverage edges, gibberish declines — all previously 0/1 in the field run. - **Gates passed:** Gate 0 (CI + red-path, #5/#6) · Gate 1 (precision 1.000, recall 0.895, zero poison) · Gate 2 (C1 instance attribution 1.000 · B1 4-level handler chains · C6 store writers↔readers · A9 portals — scorecard 137/0/0, precision & recall 1.000) · Gate 3 (B3 action effects · B4 routers · B6 cyclic journeys terminate · B7/B8 form & non-JSX events · G5 flag/role conditions — precision & recall 1.000) · Gate 4 (A4 rarity · A10 fuzzy/OCR · A1 structural · A6 subtree · E3 vision annotations · E2 aliases · G4 corrections — high-conf correct 1.000, ambiguity honesty 1.000, poison rate 0.000) · Gate 5 (F1 context bundle · F2 blast radius · F3 test coverage · F4 response schema · F5 git history · MCP server over stdio — scorecard 265/0/0, all honesty metrics 1.000; **M5 reached** — ticket in → budgeted context bundle out, over MCP) ## What CodeRadar is @@ -352,10 +352,30 @@ eval path exercises no live OCR, so it isn't a determinism risk today; revisit i **Build:** graph store keyed by SHA (`.coderadar/graphs/.json` + `latest` pointer); `resolveContext` accepts `graphVersion`; cross-version diff maps renamed/moved definitions (same structure+text signature, different name/path) → bundle warning: "matched `InvoiceCard` in prod graph; renamed `BillingCard` on main". **Accept:** fixture pair (pre/post rename): query against old graph + current code yields the rename warning with the new name. -### [ ] 6.5 Generated/vendored classification & PII policy +### [x] 6.5 Generated/vendored classification & PII policy **Failure modes:** D5, G7 **Build:** classify generated code (headers like `@generated`, codegen paths, sourcemap-less minified files): excluded from matching, retained as API metadata. PII policy doc (`docs/security.md`): screenshots ephemeral, never persisted/embedded/logged; corrections store holds terms only, never images; enforced by a lint test grepping vision-package writes. **Accept:** generated-code fixture excluded from match candidates but present in lineage; security doc + lint test in CI. **Gate 6 passes.** +**Done:** new `isGeneratedFile` in the scanner classifies a file as machine- +generated by (a) a `__generated__/` or `generated/` directory segment or a +`.generated.`/`.gen.` filename infix, (b) an `@generated` / `DO NOT EDIT` / +`AUTO-GENERATED` banner in the file head, or (c) a sourcemap-less minified line +(≥ 3000 chars). A post-pass tags component and hook nodes in those files with the +`generated` flag — uniform across function components, class components, and +hooks. `matchComponents` drops `generated` components from the candidate pool +(and therefore the IDF corpus), so a screenshot or ticket never resolves to +codegen output, while their nodes/edges — including data sources — stay in the +graph for tracing. New `docs/security.md` states the G7 PII policy (screenshots +ephemeral, graph/corrections hold no image data) and is **enforced** by +`packages/vision/src/policy.test.ts`, which greps the vision source for +filesystem-write and image-persistence APIs and asserts the package never +imports `fs`. New fixture `d5-generated-code` (path-classified `GlyphCatalog` + +banner/filename-classified `SchemaViewer` + hand-written `RevenuePanel`): both +generated components trace to their endpoints yet decline on their rendered text, +`RevenuePanel` matches and isn't poisoned by generated text mixed into the query. +6 parser-react unit tests + 7 vision policy tests. eval 314/0/0/0, determinism +1.000, all metrics 1.000; typecheck + lint clean. **Gate 6 criteria met** +(6.1/6.2/6.4 still to land for the full lifecycle phase). --- diff --git a/docs/security.md b/docs/security.md new file mode 100644 index 0000000..de61cc2 --- /dev/null +++ b/docs/security.md @@ -0,0 +1,66 @@ +# Security & PII policy + +CodeRadar reads source code and, optionally, screenshots attached to a ticket. +Source code is already in the repository the operator controls; **screenshots are +the sensitive surface** — a bug report may show real user data, PII, or internal +UI. This document states how that data is handled and how the policy is enforced +in CI. It maps to failure mode **G7** (privacy) and is part of the **Phase 6** +gate. + +## Principles + +1. **Screenshots are ephemeral.** An image is processed in memory and discarded. + CodeRadar never writes a screenshot to disk, never embeds one in the lineage + graph, and never logs its bytes. The only place image bytes leave the process + is the vision adapter's call to the model API the operator has configured + (`packages/vision/src/claude.ts`), and only for the duration of that request. + +2. **The graph holds no image data.** `scan` emits `ui-lineage.graph.json` from + source code alone. No node or edge carries a screenshot, a crop, a base64 + blob, or a data URI. The graph is safe to commit and share. + +3. **The corrections store holds terms only.** When a human confirms that a set + of terms resolves to a component, `corrections.jsonl` + (`packages/core/src/corrections.ts`) records the **terms and the component + id** — never the screenshot the terms were read from. Corrections are safe to + check in. + +4. **No persistence of derived crops or OCR images.** Vision adapters return + *text and structure* (`VisionExtraction`) — the signals the matcher already + understands — not pixels. Any future OCR/vision cache keys on an image **hash**, + never the image itself. + +5. **Least logging.** Diagnostic output names components, files, and terms. It + does not include screenshot bytes or the raw model image payload. + +## What may be persisted + +| Artifact | Contents | Safe to share | +| --- | --- | --- | +| `ui-lineage.graph.json` | Source-derived nodes/edges | Yes | +| `corrections.jsonl` | Confirmed terms → component id | Yes | +| `ui-lineage.corrections.jsonl` | (same) | Yes | +| Screenshots | — | **Never written** | + +## Enforcement + +The policy is not just documentation — it is asserted by a test that fails CI if +the vision package gains a filesystem write or a code path that persists image +bytes: + +- `packages/vision/src/policy.test.ts` greps the vision source for filesystem + write APIs (`writeFile`, `writeFileSync`, `appendFile*`, `createWriteStream`, + `fs.write`, `mkdir*` used for output) and for image persistence patterns + (writing `.png`/`.jpg`, `toDataURL`, embedding base64 into a written file). Any + hit fails the test. The single legitimate base64 conversion — encoding an image + for the model request in `claude.ts` — is in-memory and is asserted to flow only + into the API `messages.create` call, not into any sink. + +Run it with `pnpm --filter @coderadar/vision test`, or as part of `pnpm -r test` +in CI. + +## Reporting + +If you find a path that writes, embeds, or logs screenshot data, treat it as a +privacy defect (G7): open an issue and add the offending pattern to +`policy.test.ts` so the regression is locked out. diff --git a/eval/fixtures/d5-generated-code/app/RevenuePanel.tsx b/eval/fixtures/d5-generated-code/app/RevenuePanel.tsx new file mode 100644 index 0000000..be39dee --- /dev/null +++ b/eval/fixtures/d5-generated-code/app/RevenuePanel.tsx @@ -0,0 +1,20 @@ +import { useEffect, useState } from "react"; + +// A hand-authored component: a screenshot/ticket for its text must resolve here, +// and it must not be crowded out by the generated components alongside it. +export function RevenuePanel() { + const [rows, setRows] = useState[]>([]); + + useEffect(() => { + fetch("/api/revenue") + .then((res) => res.json()) + .then(setRows); + }, []); + + return ( +
+

Quarterly revenue breakdown

+

{rows.length} periods

+
+ ); +} diff --git a/eval/fixtures/d5-generated-code/app/SchemaViewer.generated.tsx b/eval/fixtures/d5-generated-code/app/SchemaViewer.generated.tsx new file mode 100644 index 0000000..11bac6e --- /dev/null +++ b/eval/fixtures/d5-generated-code/app/SchemaViewer.generated.tsx @@ -0,0 +1,25 @@ +/** + * @generated by coderadar-fixture-codegen — DO NOT EDIT. + * + * Generated by BANNER (the @generated docblock tag) as well as the + * `.generated.` filename infix. Exercises banner-based classification even when + * the path is otherwise ordinary. + */ +import { useEffect, useState } from "react"; + +export function SchemaViewer() { + const [schema, setSchema] = useState(null); + + useEffect(() => { + fetch("/api/schema") + .then((res) => res.json()) + .then(setSchema); + }, []); + + return ( +
+

Generated schema viewer

+ 
{JSON.stringify(schema)}
+
+ ); +} diff --git a/eval/fixtures/d5-generated-code/app/__generated__/GlyphCatalog.tsx b/eval/fixtures/d5-generated-code/app/__generated__/GlyphCatalog.tsx new file mode 100644 index 0000000..40ec498 --- /dev/null +++ b/eval/fixtures/d5-generated-code/app/__generated__/GlyphCatalog.tsx @@ -0,0 +1,23 @@ +import { useEffect, useState } from "react"; + +// Generated by DIRECTORY shape (`__generated__/`). No banner — the path alone +// classifies it. Its data source stays in the graph as API metadata, but its +// rendered text ("Autogenerated glyph catalog") must never win a match. +export function GlyphCatalog() { + const [icons, setIcons] = useState([]); + + useEffect(() => { + fetch("/api/icons") + .then((res) => res.json()) + .then(setIcons); + }, []); + + return ( +
    +
  • Autogenerated glyph catalog
    • + {icons.map((i) => ( +
  • {i}
    • + ))} +
+ ); +} diff --git a/eval/fixtures/d5-generated-code/golden.json b/eval/fixtures/d5-generated-code/golden.json new file mode 100644 index 0000000..546135b --- /dev/null +++ b/eval/fixtures/d5-generated-code/golden.json @@ -0,0 +1,22 @@ +{ + "failureMode": "D5", + "note": "Generated/vendored classification (6.5): machine-generated code — recognized by a __generated__/ directory segment, a .generated. filename infix, or an @generated / DO NOT EDIT banner — is retained in the graph as lineage/API metadata but excluded from match candidates, because it is not authored UI. GlyphCatalog (path-classified) and SchemaViewer (banner + filename) both still trace to their endpoints, yet a query for their rendered text declines, while the hand-written RevenuePanel matches and is not poisoned by a generated component's text mixed into the query.", + "expect": { + "components": [ + { "name": "RevenuePanel", "instances": 0 }, + { "name": "GlyphCatalog", "instances": 0 }, + { "name": "SchemaViewer", "instances": 0 } + ], + "attributions": [ + { "component": "RevenuePanel", "endpoints": ["/api/revenue"] }, + { "component": "GlyphCatalog", "endpoints": ["/api/icons"] }, + { "component": "SchemaViewer", "endpoints": ["/api/schema"] } + ], + "queries": [ + { "terms": ["Quarterly revenue breakdown"], "status": "ok", "top": "RevenuePanel" }, + { "terms": ["Autogenerated glyph catalog"], "status": "declined" }, + { "terms": ["Generated schema viewer"], "status": "declined" }, + { "terms": ["Autogenerated glyph catalog", "Quarterly revenue breakdown"], "status": "ok", "top": "RevenuePanel" } + ] + } +} diff --git a/packages/core/src/query.ts b/packages/core/src/query.ts index c77eaeb..5af1d4e 100644 --- a/packages/core/src/query.ts +++ b/packages/core/src/query.ts @@ -135,7 +135,12 @@ export function matchComponents( if (queryTerms.length === 0 && descriptor === undefined) return declined("no-signal"); const instancesByDefinition = groupInstances(graph); - const components = graph.nodes.filter((n): n is ComponentNode => n.kind === "component"); + // Machine-generated components (6.5, D5) are excluded from the candidate pool + // — and therefore from the IDF corpus below — so a screenshot or ticket never + // resolves to codegen output. Their nodes/edges stay in the graph for tracing. + const components = graph.nodes.filter( + (n): n is ComponentNode => n.kind === "component" && !(n.flags?.includes("generated") ?? false), + ); // Document frequency per token, for rarity (IDF) weighting. const documentFrequency = new Map(); diff --git a/packages/parser-react/src/generated.test.ts b/packages/parser-react/src/generated.test.ts new file mode 100644 index 0000000..5120f6b --- /dev/null +++ b/packages/parser-react/src/generated.test.ts @@ -0,0 +1,53 @@ +import path from "node:path"; +import { fileURLToPath } from "node:url"; + +import { matchComponents } from "@coderadar/core"; +import { describe, expect, it } from "vitest"; + +import { resolveHookEdges, scanReact } from "./scan.js"; + +const fixture = path.resolve( + path.dirname(fileURLToPath(import.meta.url)), + "../../../eval/fixtures/d5-generated-code/app", +); + +const graph = resolveHookEdges(scanReact({ root: fixture })); + +const component = (name: string) => + graph.nodes.find((n) => n.kind === "component" && n.name === name); + +describe("generated-code classification (6.5, D5)", () => { + it("flags components generated by directory shape (__generated__/)", () => { + expect(component("GlyphCatalog")?.flags).toContain("generated"); + }); + + it("flags components generated by banner + filename infix (.generated.)", () => { + expect(component("SchemaViewer")?.flags).toContain("generated"); + }); + + it("leaves hand-authored components unflagged", () => { + expect(component("RevenuePanel")?.flags ?? []).not.toContain("generated"); + }); + + it("retains generated data sources in the graph (API metadata)", () => { + const endpoints = graph.nodes.flatMap((n) => + n.kind === "data-source" ? [n.endpoint] : [], + ); + expect(endpoints).toContain("/api/icons"); + expect(endpoints).toContain("/api/schema"); + expect(endpoints).toContain("/api/revenue"); + }); + + it("excludes generated components from match candidates", () => { + const glyph = matchComponents(graph, { terms: ["Autogenerated glyph catalog"] }); + expect(glyph.status).toBe("declined"); + const schema = matchComponents(graph, { terms: ["Generated schema viewer"] }); + expect(schema.status).toBe("declined"); + }); + + it("still resolves hand-authored components", () => { + const result = matchComponents(graph, { terms: ["Quarterly revenue breakdown"] }); + expect(result.status).toBe("ok"); + expect(result.candidates[0]?.value.component.name).toBe("RevenuePanel"); + }); +}); diff --git a/packages/parser-react/src/scan.ts b/packages/parser-react/src/scan.ts index 8c2a714..e1c8932 100644 --- a/packages/parser-react/src/scan.ts +++ b/packages/parser-react/src/scan.ts @@ -187,9 +187,12 @@ export function scanReact(options: ScanOptions): LineageGraph { const stores = detectStores(project, root, nodes, addEdge, baseUrls, wrappers); /** Event node id → the handler expressions it wires, mined for effects (3.2). */ const handlerExprs = new Map(); + /** Files classified machine-generated (6.5, D5) — kept in lineage, excluded from matching. */ + const generatedFiles = new Set(); for (const sourceFile of sortedSourceFiles(project)) { const file = toPosix(path.relative(root, sourceFile.getFilePath())); + if (isGeneratedFile(file, sourceFile.getFullText())) generatedFiles.add(file); // Test files are swept separately (5.4) — they exercise components, they // don't define the app's UI, so they must not produce component/hook nodes. if (isTestFile(file)) continue; @@ -267,6 +270,19 @@ export function scanReact(options: ScanOptions): LineageGraph { if (openApi !== null) linkOpenApiResponses(nodes, openApi); } + // Mark nodes defined in machine-generated files (6.5, D5). Done as a post-pass + // so it covers function components, class components, and hooks uniformly + // regardless of which extractor emitted them. The `generated` flag is what the + // matcher reads to keep these out of candidate ranking while leaving their + // data-source lineage intact. + if (generatedFiles.size > 0) { + for (const node of nodes.values()) { + if ((node.kind === "component" || node.kind === "hook") && generatedFiles.has(node.loc.file)) { + node.flags = [...(node.flags ?? []), "generated"]; + } + } + } + return { version: 2, root, @@ -311,6 +327,44 @@ function sortedSourceFiles(project: Project): SourceFile[] { }); } +/** + * Codegen path shapes (6.5, D5): a `__generated__/` or `generated/` directory + * segment, or a `.generated.` / `.gen.` filename infix. Case-insensitive. + */ +const GENERATED_PATH = /(^|\/)(__generated__|generated)(\/|$)|\.(generated|gen)\.[jt]sx?$/i; + +/** + * Banner comments codegen tools emit at the top of a file: the `@generated` + * docblock tag, the Go-style "Code generated … DO NOT EDIT" line, or a bare + * "DO NOT EDIT" / "AUTO-GENERATED" marker. + */ +const GENERATED_BANNER = /@generated\b|DO NOT EDIT|AUTO-?GENERATED/i; + +/** A single line this long is a sourcemap-less minified bundle, not authored source. */ +const MINIFIED_LINE = 3000; + +/** + * True when a file is machine-generated (6.5, D5) — by path shape, a top-of-file + * banner, or minification. Generated code is retained in the graph as lineage / + * API metadata but excluded from match candidates: it is not authored UI, so a + * screenshot or ticket should never resolve to it. + */ +function isGeneratedFile(relPath: string, text: string): boolean { + if (GENERATED_PATH.test(relPath)) return true; + // Banners live in the first lines; only scan the head so a stray "DO NOT EDIT" + // deep in a hand-written file doesn't misclassify it. + if (GENERATED_BANNER.test(text.slice(0, 2000))) return true; + // Minified: any single line past the threshold. Real source wraps long before. + let lineStart = 0; + for (let i = 0; i <= text.length; i += 1) { + if (i === text.length || text.charCodeAt(i) === 10 /* \n */) { + if (i - lineStart >= MINIFIED_LINE) return true; + lineStart = i + 1; + } + } + return false; +} + /** Spread helper: emit a `responseType` field only when one was recovered (5.5). */ function responseTypeProp(rt: ResponseType | null): { responseType?: ResponseType } { return rt !== null ? { responseType: rt } : {}; diff --git a/packages/vision/src/policy.test.ts b/packages/vision/src/policy.test.ts new file mode 100644 index 0000000..89bc836 --- /dev/null +++ b/packages/vision/src/policy.test.ts @@ -0,0 +1,66 @@ +import fs from "node:fs"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; + +import { describe, expect, it } from "vitest"; + +/** + * PII policy enforcement (6.5, G7) — see docs/security.md. + * + * Screenshots are ephemeral: the vision package must never write image bytes to + * disk, embed them in the graph, or persist a derived crop. This test greps the + * vision source for filesystem-write and image-persistence patterns so the + * regression is locked out of CI. The one legitimate base64 conversion (encoding + * an image for the model request in claude.ts) is in-memory and allowed. + */ + +const srcDir = path.dirname(fileURLToPath(import.meta.url)); + +/** Vision source files, excluding tests. */ +function sourceFiles(): string[] { + return fs + .readdirSync(srcDir) + .filter((f) => f.endsWith(".ts") && !f.endsWith(".test.ts")) + .map((f) => path.join(srcDir, f)); +} + +// Filesystem-write / persistence APIs. A vision adapter has no reason to touch +// any of these — its output is text + structure, returned to the caller. +const FORBIDDEN_WRITE = [ + "writeFileSync", + "writeFile", + "appendFileSync", + "appendFile", + "createWriteStream", + "fs.write", + "toDataURL", +]; + +describe("vision PII policy (6.5, G7)", () => { + const files = sourceFiles(); + + it("has vision source to check", () => { + expect(files.length).toBeGreaterThan(0); + }); + + for (const file of sourceFiles()) { + const rel = path.basename(file); + it(`${rel} never persists image bytes`, () => { + const text = fs.readFileSync(file, "utf-8"); + const hits = FORBIDDEN_WRITE.filter((api) => text.includes(api)); + expect(hits, `${rel} uses a persistence API — screenshots must stay ephemeral`).toStrictEqual( + [], + ); + }); + } + + it("never imports the node fs module (no disk access at all)", () => { + for (const file of files) { + const text = fs.readFileSync(file, "utf-8"); + expect( + /from\s+["']node:fs["']|require\(\s*["']fs["']\s*\)/.test(text), + `${path.basename(file)} imports fs — the vision package must not access disk`, + ).toBe(false); + } + }); +});