Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 23 additions & 3 deletions TRACKER.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
## Status

- **Current phase:** 6 — Lifecycle, scale, hardening (6F field-hardening done bar the data-blocked 6F.6 detection half)
- **Next step:** 6.5 generated/vendored classification & PII policy, then 6.4 version skew, 6.1 incremental re-scan, 6.2 scale (bench repo). 6.3 determinism landed first as it de-risks the rest (eval now hard-gates two-run byte-identity). Publish 0.4.1 to the tester still pending (needs npm creds).
- **Done:** 0.1–0.4, 1.1–1.6, 2.1–2.5, 3.1–3.6, 4.1–4.6, 5.1–5.7, 6F.1–6F.5, 6F.7–6F.10 + 6F.6 (defensive half), **6.3** · **0.4.1 prepared** (all packages bumped, changelog + version strings, `npm pack` → ui-lineage-0.4.1.tgz verified: both bins, no @coderadar leak; publish to npm pending). 0.4.0 tagged/merged earlier. **Self-validated on Grafana frontend** (6,461 files, 15,334 nodes / 18,367 edges in 72 s): 55 RTK-query data sources, 32 routes, 1,009 coverage edges, gibberish declines — all previously 0/1 in the field run.
- **Next step:** 6.4 version skew, then 6.1 incremental re-scan, 6.2 scale (bench repo). 6.3 determinism + 6.5 generated-classification/PII landed first (6.3 de-risks the rest via a two-run byte-identity gate; 6.5 closes the D5/G7 gate criteria). Publish 0.4.1 to the tester still pending (needs npm creds).
- **Done:** 0.1–0.4, 1.1–1.6, 2.1–2.5, 3.1–3.6, 4.1–4.6, 5.1–5.7, 6F.1–6F.5, 6F.7–6F.10 + 6F.6 (defensive half), **6.3, 6.5** · **0.4.1 prepared** (all packages bumped, changelog + version strings, `npm pack` → ui-lineage-0.4.1.tgz verified: both bins, no @coderadar leak; publish to npm pending). 0.4.0 tagged/merged earlier. **Self-validated on Grafana frontend** (6,461 files, 15,334 nodes / 18,367 edges in 72 s): 55 RTK-query data sources, 32 routes, 1,009 coverage edges, gibberish declines — all previously 0/1 in the field run.
- **Gates passed:** Gate 0 (CI + red-path, #5/#6) · Gate 1 (precision 1.000, recall 0.895, zero poison) · Gate 2 (C1 instance attribution 1.000 · B1 4-level handler chains · C6 store writers↔readers · A9 portals — scorecard 137/0/0, precision & recall 1.000) · Gate 3 (B3 action effects · B4 routers · B6 cyclic journeys terminate · B7/B8 form & non-JSX events · G5 flag/role conditions — precision & recall 1.000) · Gate 4 (A4 rarity · A10 fuzzy/OCR · A1 structural · A6 subtree · E3 vision annotations · E2 aliases · G4 corrections — high-conf correct 1.000, ambiguity honesty 1.000, poison rate 0.000) · Gate 5 (F1 context bundle · F2 blast radius · F3 test coverage · F4 response schema · F5 git history · MCP server over stdio — scorecard 265/0/0, all honesty metrics 1.000; **M5 reached** — ticket in → budgeted context bundle out, over MCP)

## What CodeRadar is
Expand Down Expand Up @@ -352,10 +352,30 @@ eval path exercises no live OCR, so it isn't a determinism risk today; revisit i
**Build:** graph store keyed by SHA (`.coderadar/graphs/<sha>.json` + `latest` pointer); `resolveContext` accepts `graphVersion`; cross-version diff maps renamed/moved definitions (same structure+text signature, different name/path) → bundle warning: "matched `InvoiceCard` in prod graph; renamed `BillingCard` on main".
**Accept:** fixture pair (pre/post rename): query against old graph + current code yields the rename warning with the new name.

### [ ] 6.5 Generated/vendored classification & PII policy
### [x] 6.5 Generated/vendored classification & PII policy
**Failure modes:** D5, G7
**Build:** classify generated code (headers like `@generated`, codegen paths, sourcemap-less minified files): excluded from matching, retained as API metadata. PII policy doc (`docs/security.md`): screenshots ephemeral, never persisted/embedded/logged; corrections store holds terms only, never images; enforced by a lint test grepping vision-package writes.
**Accept:** generated-code fixture excluded from match candidates but present in lineage; security doc + lint test in CI. **Gate 6 passes.**
**Done:** new `isGeneratedFile` in the scanner classifies a file as machine-
generated by (a) a `__generated__/` or `generated/` directory segment or a
`.generated.`/`.gen.` filename infix, (b) an `@generated` / `DO NOT EDIT` /
`AUTO-GENERATED` banner in the file head, or (c) a sourcemap-less minified line
(≥ 3000 chars). A post-pass tags component and hook nodes in those files with the
`generated` flag — uniform across function components, class components, and
hooks. `matchComponents` drops `generated` components from the candidate pool
(and therefore the IDF corpus), so a screenshot or ticket never resolves to
codegen output, while their nodes/edges — including data sources — stay in the
graph for tracing. New `docs/security.md` states the G7 PII policy (screenshots
ephemeral, graph/corrections hold no image data) and is **enforced** by
`packages/vision/src/policy.test.ts`, which greps the vision source for
filesystem-write and image-persistence APIs and asserts the package never
imports `fs`. New fixture `d5-generated-code` (path-classified `GlyphCatalog` +
banner/filename-classified `SchemaViewer` + hand-written `RevenuePanel`): both
generated components trace to their endpoints yet decline on their rendered text,
`RevenuePanel` matches and isn't poisoned by generated text mixed into the query.
6 parser-react unit tests + 7 vision policy tests. eval 314/0/0/0, determinism
1.000, all metrics 1.000; typecheck + lint clean. **Gate 6 criteria met**
(6.1/6.2/6.4 still to land for the full lifecycle phase).

---

Expand Down
66 changes: 66 additions & 0 deletions docs/security.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Security & PII policy

CodeRadar reads source code and, optionally, screenshots attached to a ticket.
Source code is already in the repository the operator controls; **screenshots are
the sensitive surface** — a bug report may show real user data, PII, or internal
UI. This document states how that data is handled and how the policy is enforced
in CI. It maps to failure mode **G7** (privacy) and is part of the **Phase 6**
gate.

## Principles

1. **Screenshots are ephemeral.** An image is processed in memory and discarded.
CodeRadar never writes a screenshot to disk, never embeds one in the lineage
graph, and never logs its bytes. The only place image bytes leave the process
is the vision adapter's call to the model API the operator has configured
(`packages/vision/src/claude.ts`), and only for the duration of that request.

2. **The graph holds no image data.** `scan` emits `ui-lineage.graph.json` from
source code alone. No node or edge carries a screenshot, a crop, a base64
blob, or a data URI. The graph is safe to commit and share.

3. **The corrections store holds terms only.** When a human confirms that a set
of terms resolves to a component, `corrections.jsonl`
(`packages/core/src/corrections.ts`) records the **terms and the component
id** — never the screenshot the terms were read from. Corrections are safe to
check in.

4. **No persistence of derived crops or OCR images.** Vision adapters return
*text and structure* (`VisionExtraction`) — the signals the matcher already
understands — not pixels. Any future OCR/vision cache keys on an image **hash**,
never the image itself.

5. **Least logging.** Diagnostic output names components, files, and terms. It
does not include screenshot bytes or the raw model image payload.

## What may be persisted

| Artifact | Contents | Safe to share |
| --- | --- | --- |
| `ui-lineage.graph.json` | Source-derived nodes/edges | Yes |
| `corrections.jsonl` | Confirmed terms → component id | Yes |
| `ui-lineage.corrections.jsonl` | (same) | Yes |
| Screenshots | — | **Never written** |

## Enforcement

The policy is not just documentation — it is asserted by a test that fails CI if
the vision package gains a filesystem write or a code path that persists image
bytes:

- `packages/vision/src/policy.test.ts` greps the vision source for filesystem
write APIs (`writeFile`, `writeFileSync`, `appendFile*`, `createWriteStream`,
`fs.write`, `mkdir*` used for output) and for image persistence patterns
(writing `.png`/`.jpg`, `toDataURL`, embedding base64 into a written file). Any
hit fails the test. The single legitimate base64 conversion — encoding an image
for the model request in `claude.ts` — is in-memory and is asserted to flow only
into the API `messages.create` call, not into any sink.

Run it with `pnpm --filter @coderadar/vision test`, or as part of `pnpm -r test`
in CI.

## Reporting

If you find a path that writes, embeds, or logs screenshot data, treat it as a
privacy defect (G7): open an issue and add the offending pattern to
`policy.test.ts` so the regression is locked out.
20 changes: 20 additions & 0 deletions eval/fixtures/d5-generated-code/app/RevenuePanel.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
import { useEffect, useState } from "react";

// A hand-authored component: a screenshot/ticket for its text must resolve here,
// and it must not be crowded out by the generated components alongside it.
export function RevenuePanel() {
const [rows, setRows] = useState<Record<string, number>[]>([]);

useEffect(() => {
fetch("/api/revenue")
.then((res) => res.json())
.then(setRows);
}, []);

return (
<section>
<h2>Quarterly revenue breakdown</h2>
<p>{rows.length} periods</p>
</section>
);
}
25 changes: 25 additions & 0 deletions eval/fixtures/d5-generated-code/app/SchemaViewer.generated.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
/**
* @generated by coderadar-fixture-codegen — DO NOT EDIT.
*
* Generated by BANNER (the @generated docblock tag) as well as the
* `.generated.` filename infix. Exercises banner-based classification even when
* the path is otherwise ordinary.
*/
import { useEffect, useState } from "react";

export function SchemaViewer() {
const [schema, setSchema] = useState<unknown>(null);

useEffect(() => {
fetch("/api/schema")
.then((res) => res.json())
.then(setSchema);
}, []);

return (
<div>
<h3>Generated schema viewer</h3>
<pre>{JSON.stringify(schema)}</pre>
</div>
);
}
23 changes: 23 additions & 0 deletions eval/fixtures/d5-generated-code/app/__generated__/GlyphCatalog.tsx

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

22 changes: 22 additions & 0 deletions eval/fixtures/d5-generated-code/golden.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{
"failureMode": "D5",
"note": "Generated/vendored classification (6.5): machine-generated code — recognized by a __generated__/ directory segment, a .generated. filename infix, or an @generated / DO NOT EDIT banner — is retained in the graph as lineage/API metadata but excluded from match candidates, because it is not authored UI. GlyphCatalog (path-classified) and SchemaViewer (banner + filename) both still trace to their endpoints, yet a query for their rendered text declines, while the hand-written RevenuePanel matches and is not poisoned by a generated component's text mixed into the query.",
"expect": {
"components": [
{ "name": "RevenuePanel", "instances": 0 },
{ "name": "GlyphCatalog", "instances": 0 },
{ "name": "SchemaViewer", "instances": 0 }
],
"attributions": [
{ "component": "RevenuePanel", "endpoints": ["/api/revenue"] },
{ "component": "GlyphCatalog", "endpoints": ["/api/icons"] },
{ "component": "SchemaViewer", "endpoints": ["/api/schema"] }
],
"queries": [
{ "terms": ["Quarterly revenue breakdown"], "status": "ok", "top": "RevenuePanel" },
{ "terms": ["Autogenerated glyph catalog"], "status": "declined" },
{ "terms": ["Generated schema viewer"], "status": "declined" },
{ "terms": ["Autogenerated glyph catalog", "Quarterly revenue breakdown"], "status": "ok", "top": "RevenuePanel" }
]
}
}
7 changes: 6 additions & 1 deletion packages/core/src/query.ts
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,12 @@ export function matchComponents(
if (queryTerms.length === 0 && descriptor === undefined) return declined("no-signal");

const instancesByDefinition = groupInstances(graph);
const components = graph.nodes.filter((n): n is ComponentNode => n.kind === "component");
// Machine-generated components (6.5, D5) are excluded from the candidate pool
// — and therefore from the IDF corpus below — so a screenshot or ticket never
// resolves to codegen output. Their nodes/edges stay in the graph for tracing.
const components = graph.nodes.filter(
(n): n is ComponentNode => n.kind === "component" && !(n.flags?.includes("generated") ?? false),
);

// Document frequency per token, for rarity (IDF) weighting.
const documentFrequency = new Map<string, number>();
Expand Down
53 changes: 53 additions & 0 deletions packages/parser-react/src/generated.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
import path from "node:path";
import { fileURLToPath } from "node:url";

import { matchComponents } from "@coderadar/core";
import { describe, expect, it } from "vitest";

import { resolveHookEdges, scanReact } from "./scan.js";

const fixture = path.resolve(
path.dirname(fileURLToPath(import.meta.url)),
"../../../eval/fixtures/d5-generated-code/app",
);

const graph = resolveHookEdges(scanReact({ root: fixture }));

const component = (name: string) =>
graph.nodes.find((n) => n.kind === "component" && n.name === name);

describe("generated-code classification (6.5, D5)", () => {
it("flags components generated by directory shape (__generated__/)", () => {
expect(component("GlyphCatalog")?.flags).toContain("generated");
});

it("flags components generated by banner + filename infix (.generated.)", () => {
expect(component("SchemaViewer")?.flags).toContain("generated");
});

it("leaves hand-authored components unflagged", () => {
expect(component("RevenuePanel")?.flags ?? []).not.toContain("generated");
});

it("retains generated data sources in the graph (API metadata)", () => {
const endpoints = graph.nodes.flatMap((n) =>
n.kind === "data-source" ? [n.endpoint] : [],
);
expect(endpoints).toContain("/api/icons");
expect(endpoints).toContain("/api/schema");
expect(endpoints).toContain("/api/revenue");
});

it("excludes generated components from match candidates", () => {
const glyph = matchComponents(graph, { terms: ["Autogenerated glyph catalog"] });
expect(glyph.status).toBe("declined");
const schema = matchComponents(graph, { terms: ["Generated schema viewer"] });
expect(schema.status).toBe("declined");
});

it("still resolves hand-authored components", () => {
const result = matchComponents(graph, { terms: ["Quarterly revenue breakdown"] });
expect(result.status).toBe("ok");
expect(result.candidates[0]?.value.component.name).toBe("RevenuePanel");
});
});
54 changes: 54 additions & 0 deletions packages/parser-react/src/scan.ts
Original file line number Diff line number Diff line change
Expand Up @@ -187,9 +187,12 @@ export function scanReact(options: ScanOptions): LineageGraph {
const stores = detectStores(project, root, nodes, addEdge, baseUrls, wrappers);
/** Event node id → the handler expressions it wires, mined for effects (3.2). */
const handlerExprs = new Map<string, Node[]>();
/** Files classified machine-generated (6.5, D5) — kept in lineage, excluded from matching. */
const generatedFiles = new Set<string>();

for (const sourceFile of sortedSourceFiles(project)) {
const file = toPosix(path.relative(root, sourceFile.getFilePath()));
if (isGeneratedFile(file, sourceFile.getFullText())) generatedFiles.add(file);
// Test files are swept separately (5.4) — they exercise components, they
// don't define the app's UI, so they must not produce component/hook nodes.
if (isTestFile(file)) continue;
Expand Down Expand Up @@ -267,6 +270,19 @@ export function scanReact(options: ScanOptions): LineageGraph {
if (openApi !== null) linkOpenApiResponses(nodes, openApi);
}

// Mark nodes defined in machine-generated files (6.5, D5). Done as a post-pass
// so it covers function components, class components, and hooks uniformly
// regardless of which extractor emitted them. The `generated` flag is what the
// matcher reads to keep these out of candidate ranking while leaving their
// data-source lineage intact.
if (generatedFiles.size > 0) {
for (const node of nodes.values()) {
if ((node.kind === "component" || node.kind === "hook") && generatedFiles.has(node.loc.file)) {
node.flags = [...(node.flags ?? []), "generated"];
}
}
}

return {
version: 2,
root,
Expand Down Expand Up @@ -311,6 +327,44 @@ function sortedSourceFiles(project: Project): SourceFile[] {
});
}

/**
* Codegen path shapes (6.5, D5): a `__generated__/` or `generated/` directory
* segment, or a `.generated.` / `.gen.` filename infix. Case-insensitive.
*/
const GENERATED_PATH = /(^|\/)(__generated__|generated)(\/|$)|\.(generated|gen)\.[jt]sx?$/i;

/**
* Banner comments codegen tools emit at the top of a file: the `@generated`
* docblock tag, the Go-style "Code generated … DO NOT EDIT" line, or a bare
* "DO NOT EDIT" / "AUTO-GENERATED" marker.
*/
const GENERATED_BANNER = /@generated\b|DO NOT EDIT|AUTO-?GENERATED/i;

/** A single line this long is a sourcemap-less minified bundle, not authored source. */
const MINIFIED_LINE = 3000;

/**
* True when a file is machine-generated (6.5, D5) — by path shape, a top-of-file
* banner, or minification. Generated code is retained in the graph as lineage /
* API metadata but excluded from match candidates: it is not authored UI, so a
* screenshot or ticket should never resolve to it.
*/
function isGeneratedFile(relPath: string, text: string): boolean {
if (GENERATED_PATH.test(relPath)) return true;
// Banners live in the first lines; only scan the head so a stray "DO NOT EDIT"
// deep in a hand-written file doesn't misclassify it.
if (GENERATED_BANNER.test(text.slice(0, 2000))) return true;
// Minified: any single line past the threshold. Real source wraps long before.
let lineStart = 0;
for (let i = 0; i <= text.length; i += 1) {
if (i === text.length || text.charCodeAt(i) === 10 /* \n */) {
if (i - lineStart >= MINIFIED_LINE) return true;
lineStart = i + 1;
}
}
return false;
}

/** Spread helper: emit a `responseType` field only when one was recovered (5.5). */
function responseTypeProp(rt: ResponseType | null): { responseType?: ResponseType } {
return rt !== null ? { responseType: rt } : {};
Expand Down
Loading
Loading