feat: terraform infrastructure plugin + customSections plugin API#1
feat: terraform infrastructure plugin + customSections plugin API#1neilwashere merged 4 commits intomainfrom
Conversation
Plugins could previously only return routes, schemas, components, and middleware — they had no way to contribute new types of content to CODESIGHT.md. This meant plugin-generated insights were invisible to agents unless they knew to look for a separate file. Add customSections to PluginDetectorResult and ScanResult so plugins can return arbitrary markdown sections that get rendered into CODESIGHT.md alongside built-in sections, written as individual .md files, and referenced in AI config files (CLAUDE.md, .cursorrules, etc). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add a plugin that scans Terraform/HCL files and generates infrastructure.md with deployment context for AI agents — where a service runs, what env vars and SSM secrets it receives, whether it's public-facing, what it depends on, and per-environment overrides. Supports two modes: in-project (terraform/ subdir alongside code) and external path (separate infrastructure repo, default ../infrastructure). Uses regex + brace-counting for zero-dependency HCL parsing, following the same approach as the Go extractor. The HCL parser and service matcher are provider-agnostic, but the infrastructure extractor currently targets AWS patterns (ECS, SSM Parameter Store, ALB, Route53, CloudWatch). Azure and GCP extraction would require additional provider-specific logic in the extractor — the parser and matcher layers would not need changes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds a first-class customSections extension point to the Codesight plugin API and introduces a Terraform infrastructure plugin that extracts deployment context from Terraform/HCL into a generated infrastructure.md section for AI tooling.
Changes:
- Extend plugin/scan result types and output pipeline to support plugin-contributed markdown sections (
customSections) rendered intoCODESIGHT.mdand written as standalone.mdfiles. - Add a Terraform plugin (collector, HCL parser, service matcher, extractor, formatter) that generates infrastructure context and optional per-environment overrides from
.tfvars. - Add comprehensive Terraform plugin fixtures and tests.
Reviewed changes
Copilot reviewed 22 out of 22 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/terraform-plugin.test.ts | New Terraform plugin test suite (HCL parser, matcher, extractor, collector, formatter). |
| tests/fixtures/terraform/simple-ecs-service/environments/staging.tfvars | Fixture tfvars for staging overrides. |
| tests/fixtures/terraform/simple-ecs-service/environments/production.tfvars | Fixture tfvars for production overrides. |
| tests/fixtures/terraform/simple-ecs-service/app-service.tf | Fixture Terraform service definition with env/secrets/DNS. |
| tests/fixtures/terraform/multi-service/notifications.tf | Multi-service fixture for matcher isolation tests. |
| tests/fixtures/terraform/multi-service/billing.tf | Multi-service fixture for matcher/infra extraction tests. |
| tests/fixtures/terraform/multi-service/auth.tf | Multi-service fixture for matcher isolation tests. |
| tests/fixtures/terraform/in-project/terraform/main.tf | In-project terraform discovery fixture. |
| tests/fixtures/terraform/in-project/src/index.ts | Placeholder source fixture for mixed code + terraform repo. |
| tests/fixtures/terraform/edge-cases/complex.tf | Parser edge-cases fixture (comments, heredocs, dynamic blocks). |
| src/types.ts | Extend PluginDetectorResult and ScanResult with customSections. |
| src/plugins/terraform/types.ts | Terraform plugin configuration + extracted infra data model. |
| src/plugins/terraform/service-matcher.ts | Service-to-block matching via multi-signal scoring. |
| src/plugins/terraform/index.ts | Terraform plugin entrypoint producing an infrastructure custom section. |
| src/plugins/terraform/hcl-parser.ts | Zero-dependency HCL/tfvars parsing utilities. |
| src/plugins/terraform/formatter.ts | Markdown generator for infrastructure.md content. |
| src/plugins/terraform/file-collector.ts | .tf / .tfvars discovery (in-project and external paths). |
| src/plugins/terraform/extractor.ts | Structured infra extraction (components, env, secrets, DNS, deps, IAM, observability, env overrides). |
| src/index.ts | Aggregate plugin customSections into scan output. |
| src/generators/ai-config.ts | Reference generated custom section files from AI config output. |
| src/formatter.ts | Write plugin custom sections to disk + include in combined output. |
| package.json | Add package exports (root + terraform plugin entrypoint). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Plugin-contributed custom sections | ||
| if (result.customSections) { | ||
| for (const cs of result.customSections) { | ||
| sections.push({ name: cs.name, content: cs.content }); | ||
| await writeFile(join(outputDir, `${cs.name}.md`), cs.content); | ||
| } |
There was a problem hiding this comment.
customSections names are used directly as output filenames (${cs.name}.md). This allows path traversal (e.g. ../...) and can also overwrite built-in section files like routes.md/schema.md if a plugin reuses a reserved name. Validate/sanitize cs.name (e.g., restrict to a safe [a-z0-9_-] basename), and either reject or namespace/resolve collisions before writing.
There was a problem hiding this comment.
Fixed in b0a7fda — section names are now sanitised to [a-z0-9_-] basenames, and a reserved set (routes, schema, components, libs, config, middleware, graph, events, coverage, CODESIGHT) is rejected to prevent overwriting built-in sections.
| // Line comment: # or // | ||
| if (ch === "#" || (ch === "/" && content[i + 1] === "/")) { | ||
| // Skip to end of line | ||
| const lineEnd = content.indexOf("\n", i); | ||
| if (lineEnd === -1) break; | ||
| result.push("\n"); | ||
| i = lineEnd + 1; | ||
| continue; | ||
| } |
There was a problem hiding this comment.
stripComments doesn’t account for heredocs (<<EOF / <<-EOF). As written, # or // sequences inside heredoc bodies will be stripped even though heredoc content is literal in HCL, which can corrupt JSON/user-data and also change brace/bracket balancing for later parsing. Consider detecting heredoc starts and skipping comment stripping until the closing delimiter line.
There was a problem hiding this comment.
Fixed in b0a7fda — stripComments now detects <<EOF / <<-EOF heredoc markers and passes through heredoc bodies verbatim until the closing delimiter, skipping all comment stripping within.
| const actions = stmt.attributes["actions"] ?? stmt.attributes["effect"]; | ||
| const resources = stmt.attributes["resources"]; | ||
| if (actions) { | ||
| permissions.push(`${actions}${resources ? ` on ${resources}` : ""}`); |
There was a problem hiding this comment.
In inline IAM statement extraction, actions falls back to effect (Allow/Deny) when actions is missing, which will produce incorrect “permissions” strings. This should likely read from action/actions (and possibly not_actions) and keep effect separate rather than treating it as an action list.
| const actions = stmt.attributes["actions"] ?? stmt.attributes["effect"]; | |
| const resources = stmt.attributes["resources"]; | |
| if (actions) { | |
| permissions.push(`${actions}${resources ? ` on ${resources}` : ""}`); | |
| const actions = stmt.attributes["actions"] ?? stmt.attributes["action"]; | |
| const notActions = stmt.attributes["not_actions"]; | |
| const effect = stmt.attributes["effect"]; | |
| const resources = stmt.attributes["resources"]; | |
| const permissionTarget = actions ?? notActions; | |
| if (permissionTarget) { | |
| const qualifier = actions ? "" : "NOT "; | |
| const effectPrefix = effect ? `${effect}: ` : ""; | |
| permissions.push(`${effectPrefix}${qualifier}${permissionTarget}${resources ? ` on ${resources}` : ""}`); |
There was a problem hiding this comment.
Fixed in b0a7fda — now reads actions/action (not effect as fallback), includes effect as a prefix when present, and checks resource as an alternative key to resources.
| }, | ||
| "exports": { | ||
| ".": "./dist/index.js", | ||
| "./plugins/terraform": "./dist/plugins/terraform/index.js" |
There was a problem hiding this comment.
Adding an exports map changes Node/TypeScript module resolution and prevents consumers from importing previously-working deep paths (e.g. codesight/dist/...) unless explicitly exported. If this package has any existing programmatic consumers, consider whether this is a breaking change and either (a) add explicit subpath exports for supported internals or (b) bump major version / document the change.
| "./plugins/terraform": "./dist/plugins/terraform/index.js" | |
| "./plugins/terraform": "./dist/plugins/terraform/index.js", | |
| "./dist/*": "./dist/*" |
There was a problem hiding this comment.
Fixed in b0a7fda — added "./dist/*": "./dist/*" wildcard export to preserve deep import paths for existing consumers.
| import { describe, it } from "node:test"; | ||
| import assert from "node:assert/strict"; | ||
| import { readFile } from "node:fs/promises"; | ||
| import { join } from "node:path"; | ||
|
|
||
| const FIXTURES = join(import.meta.dirname, "fixtures", "terraform"); | ||
|
|
||
| // Import from dist — matches codesight's test convention | ||
| const { parseHclFile, parseTfvars, stripComments, extractBraceBlock } = await import( | ||
| "../dist/plugins/terraform/hcl-parser.js" | ||
| ); | ||
| const { matchServiceBlocks, normaliseServiceName } = await import( |
There was a problem hiding this comment.
This new test file won’t run under the current pnpm test script (tsx --test tests/detectors.test.ts), so the new Terraform plugin coverage may be skipped in CI. Update the test command to include this file (e.g. tsx --test tests/*.test.ts) or import/aggregate it from the existing test entrypoint.
There was a problem hiding this comment.
Fixed in b0a7fda — test script updated from tests/detectors.test.ts to tests/*.test.ts so all test files run in CI.
- formatter.ts: sanitise customSections names to safe basenames and reject collisions with built-in section names - hcl-parser.ts: skip comment stripping inside heredoc bodies to avoid corrupting literal content - extractor.ts: fix IAM statement extraction to read action/actions instead of falling back to effect - package.json: add dist/* wildcard export to preserve deep imports, update test script to run all test files Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…egex safety - Fix reserved name "CODESIGHT" casing mismatch in formatter.ts (was uppercase, safeName is always lowercased so the guard never matched) - Sanitize section names in ai-config.ts to match filenames written by formatter.ts - Move BLOCK_HEADER regex inside function body to prevent shared mutable state under concurrent use - Add string-aware bracket counting in multi-line list parser - Remove dead TF_EXTENSIONS constant - Add informational TODOs for parallel file reads, parseTfvars multiline limitation, and custom section name collision Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add customSections to plugin API for first-class plugin output Plugins could previously only return routes, schemas, components, and middleware — they had no way to contribute new types of content to CODESIGHT.md. This meant plugin-generated insights were invisible to agents unless they knew to look for a separate file. Add customSections to PluginDetectorResult and ScanResult so plugins can return arbitrary markdown sections that get rendered into CODESIGHT.md alongside built-in sections, written as individual .md files, and referenced in AI config files (CLAUDE.md, .cursorrules, etc). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add terraform infrastructure plugin (AWS-focused) Add a plugin that scans Terraform/HCL files and generates infrastructure.md with deployment context for AI agents — where a service runs, what env vars and SSM secrets it receives, whether it's public-facing, what it depends on, and per-environment overrides. Supports two modes: in-project (terraform/ subdir alongside code) and external path (separate infrastructure repo, default ../infrastructure). Uses regex + brace-counting for zero-dependency HCL parsing, following the same approach as the Go extractor. The HCL parser and service matcher are provider-agnostic, but the infrastructure extractor currently targets AWS patterns (ECS, SSM Parameter Store, ALB, Route53, CloudWatch). Azure and GCP extraction would require additional provider-specific logic in the extractor — the parser and matcher layers would not need changes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address PR review comments - formatter.ts: sanitise customSections names to safe basenames and reject collisions with built-in section names - hcl-parser.ts: skip comment stripping inside heredoc bodies to avoid corrupting literal content - extractor.ts: fix IAM statement extraction to read action/actions instead of falling back to effect - package.json: add dist/* wildcard export to preserve deep imports, update test script to run all test files Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review findings — reserved name bypass, name mismatch, regex safety - Fix reserved name "CODESIGHT" casing mismatch in formatter.ts (was uppercase, safeName is always lowercased so the guard never matched) - Sanitize section names in ai-config.ts to match filenames written by formatter.ts - Move BLOCK_HEADER regex inside function body to prevent shared mutable state under concurrent use - Add string-aware bracket counting in multi-line list parser - Remove dead TF_EXTENSIONS constant - Add informational TODOs for parallel file reads, parseTfvars multiline limitation, and custom section name collision Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Summary
Add
customSectionsto the plugin API so plugins can contribute first-class sections to CODESIGHT.md, individual.mdfiles, and AI config references (CLAUDE.md, .cursorrules, etc). Previously plugins could only return routes/schemas/components/middleware — any other type of insight was invisible to agents. This is a ~20-line additive change acrosstypes.ts,index.ts,formatter.ts, andai-config.ts.Add a Terraform infrastructure plugin that scans HCL files and generates
infrastructure.mdwith deployment context: where a service runs, what env vars and SSM secrets it receives, whether it's public-facing (DNS/ALB), what it depends on, IAM permissions, observability config, and per-environment overrides from.tfvarsfiles.Infrastructure plugin details
Two modes of operation:
terraform/,infra/, etc. subdirectories within the project../infrastructure)Service matching: scored multi-signal algorithm maps the current project name to its Terraform resources via label prefix, image URI, enable flags, filenames, and user-configured aliases.
HCL parsing: zero-dependency regex + brace-counting parser, following the same approach as the existing Go extractor. Handles comments, heredocs, nested blocks, interpolation,
jsonencode,dynamicblocks, andfor_each.Scope and limitations
The HCL parser and service matcher are provider-agnostic — they work on any Terraform regardless of cloud provider. The infrastructure extractor currently targets AWS patterns (ECS, SSM Parameter Store, ALB, Route53, CloudWatch). Azure and GCP would require additional provider-specific extraction logic; the parser and matcher layers would not need changes. We have no Azure/GCP infrastructure repos to test against, so we're being honest about the boundary rather than guessing.
Test plan
🤖 Generated with Claude Code