feat: generate terraform-plugin-framework providers from a blueprint - #7
Merged
Merged
Conversation
Bootstraps the toolkit and lands a walking skeleton that takes a blueprint
through to a compiling, plan-able provider.
The problem: an OpenAPI document records what an API's fields are called, not
which are writable, which are immutable, what the server normalises, or what it
defaults. Providers built on the specification alone therefore carry those facts
as hand-maintained special cases discovered one production bug at a time --
terraform-provider-thousandeyes does exactly this, in 1,165 lines of runtime
reflection. This toolkit generates that mapping instead, and later phases derive
the missing behaviour by probing a live API and committing the transcripts.
What works end to end:
tfprovidergen emit -blueprint blueprints/thousandeyes -out pilot/thousandeyes
cd pilot/thousandeyes && go build ./... && go test ./... && terraform plan
# + resource "thousandeyes_tag" "example"
# Plan: 1 to add, 0 to change, 0 to destroy.
Structure:
- internal/blueprint the IR. A superset of HashiCorp's Provider Code
Specification, which cannot express CRUD wiring, SDK binding, observed
behaviour or test scaffolding -- most of what a working provider is.
- internal/render all the logic. Every value a template consumes is a
finished string, so templates branch on presence and never on meaning.
- internal/templates the emitted shape as reviewable text, per house
convention, embedded in its own package.
- internal/emit plan, format with gofumpt, refuse to overwrite files the
tool does not own.
- pilot/thousandeyes a nested module holding a real generated provider, built
and tested in CI. Deliberately a separate module: the toolkit emits text and
must never depend on terraform-plugin-framework itself.
Deliberate choices worth knowing:
- Generated files carry no timestamp and no tool version. Either would make
every regeneration a diff and destroy the drift check, which is the only
thing keeping committed output honest.
- Enums generate documented values but no OneOf validator. The SDK's enums are
open by design; a validator would turn a routine upstream addition into a
plan failure.
- Absent fields flatten to null rather than the zero value, which is the usual
cause of a provider with a permanent diff.
- gofumpt rather than go/format, because the target repositories autofix with
gofumpt, gci and golines; merely gofmt-clean output gets rewritten on first
open and then reads as drift with no source change.
Verified: 267 tests across both modules; both gofumpt-clean, so emitted code is
a formatter fixed point. Regenerating reports 0 written, 7 unchanged. Editing a
generated file makes verify exit 1 with a step-summary diffstat; deleting one is
reported as missing rather than drifted.
Deferred with reasons recorded in the task list: OpenAPI ingestion (phase 2),
codegen-spec interop (3), the prober (4), generated tests and mocks (5). The
anchor region-patcher is unnecessary while the pilot is greenfield, since
registration files are generated whole. sdkbind should come next: a binding was
written against the SDK working tree rather than the pinned v0.1.0, where the
services hang off Client.API rather than Client, and only the compile gate
caught it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
13 tasks
ShocOne
added a commit
that referenced
this pull request
Aug 5, 2026
…oards-wave fix+feat(kiota): schema-default stripping (run #7's five 400s) + the final wave — full 23-resource parity
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request Description
Summary
Bootstraps this repository from the template scaffold and lands a walking skeleton: a blueprint goes in, a compiling and
terraform plan-able provider comes out.Issue Reference
No issue; this is the first implementation phase of the agreed plan. The charter line is in
share/engineering_culture/Forge.md: "Evaluate code generation tools … for schema-first development and reducing boilerplate."Motivation and Context
An OpenAPI document records what an API's fields are called. It does not record which are writable, which are immutable, what the server normalises on the way in, or what it defaults when a field is omitted. Those are the facts that decide whether a Terraform provider actually works.
Providers built on the specification alone therefore accumulate those facts as hand-maintained special cases, discovered one production bug at a time.
terraform-provider-thousandeyesdoes exactly this, in a 1,165-line runtime reflection engine (thousandeyes/util.go) whose special cases —sensitiveFields,emptyStringToNilTypes,preserveNestedSensitiveFields,resourceFixups— are a catalogue of precisely that.terraform-provider-microsoft365solved it properly, with ~400 resources on a rigid six-file archetype and zero runtime reflection. But all 400 were hand-written by copy-pasting_resource_template/.Meanwhile
go-sdk-thousandeyesalready proves this org can do industrial codegen: 97 service packages and ~90k lines from an OpenAPI snapshot, with a CI drift gate. The SDK layer is generated and the provider layer is hand-written. This closes that gap.Later phases derive the missing behaviour empirically by probing a live API and committing the HTTP transcripts as evidence.
Why not HashiCorp's toolchain:
terraform-plugin-codegen-specv0.2.0,-frameworkv0.4.1 and-openapiv0.3.0 have had no functional commits since September 2024, generate no CRUD logic at all, and cannot expressdynamic,int32/float32, blocks, resource identity or write-only attributes. Every renderer in-frameworklives underinternal/, so there is nothing to import. Their Provider Code Specification is adopted as an interop format (phase 3), not as the model.Dependencies
mvdan.cc/gofumpt v0.9.1only. It deliberately does not depend onterraform-plugin-framework— the toolkit emits text, and depending on the framework would couple every consumer to one framework version.pilot/thousandeyesis a nested module pinning framework v1.19.0, plugin-testing v1.16.0 and httpmock v1.4.1, matching the ms365 provider exactly so nobody reconciles two framework versions across the org. It pinsgo-sdk-thousandeyes v0.1.0, which resolves from the proxy — noreplaceneeded.go.work; it breaks dependabot's per-directory resolution. CI enters the nested module explicitly, which is also the honest simulation of a downstream consumer.go 1.25.0, matching the sibling repos rather than the newer local toolchain.Type of Change
Nothing here is breaking: the repository previously contained no Go code.
Testing
267 tests across both modules. The ones worth reviewing:
TestUnit_Emit_IsDeterministicverifyfail on a run that changed nothing, which destroys the whole drift design.TestUnit_Emit_CarriesNoTimestampOrVersionTestUnit_Naming_TerraformNameHTTPProxy,iOSVersion,ipv6Address).TestUnit_Naming_TerraformNameAndGoFieldName_AgreeOnWordBoundariestfsdktag silently does not match its schema attribute.TestUnit_Convert_NilPointerFlattensToNullTestUnit_Convert_EnumUsesRawWireValueString()renders an unlisted value asAccessType(raw)for logging; putting that in state would corrupt it.TestUnit_Emit_RefusesToOverwriteHandWrittenFiles-outdestroying work with no recovery.TestUnit_CLI_Dispatch_ImplementationClaimsAreTrueManually verified, not assumed:
0 written, 7 unchanged.verifyexits 1 with a$GITHUB_STEP_SUMMARYdiffstat; deleting one → reported as missing rather than drifted; re-emitting restores both.terraform planagainst a dev-override build renders the resource with zero errors. It needs no credentials: a plan for a new resource makes no API calls.Quality Checklist
Additional Notes
What to actually review
The emitted output, not the emitter.
pilot/thousandeyes/internal/services/resources/tags/v7/tag/state.gois the file that replaces runtime reflection — 15 direct assignments, each a compile error if wrong. Theninternal/templates/*.tmpl, which is the emitted shape as ordinary reviewable text.Design choices worth disagreeing with
cmd/rather than the housescripts/<Area>/<Command>/main.go. Mechanical reason:.golangci.ymlexcludesscripts/.*and only rescuesinternal/.*\.go$, so underscripts/the CLI's flag validation and exit codes would be permanently unlinted. Also one import path for downstream repos to pin instead of six. What is kept is the substantive part — stdlibflag, no cobra.stringvalidator.OneOf. The SDK's enums are open by design; a validator would turn a routine upstream addition into a plan failure — reintroducing exactly the fragility the SDK avoided.updateStyleis mandatory when a resource has an update operation.UpdateTagis PUT, so an omitted field is cleared. Guessing this wrong silently erases attributes the practitioner never mentioned, so the blueprint refuses to be ambiguous about it.Two judgement calls in the tag blueprint that need probing
Both are recorded in the blueprint's own descriptions rather than hidden, and phase 4 settles them:
color,access_typeandmatch_typearecomputed_optionalon the assumption the API assigns defaults. If it does not, those attributes show(known after apply)unnecessarily.legacy_idisfloat64because the specification saysnumber, though observed values are integral. A wrong type in a published schema is breaking to fix.A real bug this caught, worth acting on
I wrote a binding against the local
go-sdk-thousandeyesworking tree, which is ahead of the tag. At the pinned v0.1.0 the client isClient{Transport, API *API}with services onAPI, sor.client.Tagsdoes not exist and the accessor isr.client.API.Tags. Only the compile gate caught it, as four identical errors.This is plan risk #10 ("the pilot's SDK is a moving target") arriving on day one, and it argues for pulling
sdkbindforward — an AST scan of the pinned module would have said "CreateTag not found on *thousandeyes.Client" at blueprint time instead.Known gaps in this PR
codegen-verify.ymlyet.verifyworks and is proven to fail correctly, but nothing runs it in CI, so drift is currently only caught locally. It is the SDK repo's workflow with the remediation command swapped and an addedcd pilot/thousandeyes.dependabot.ymlhas no/pilot/thousandeyesentry, so the pilot's framework pins will rot unwatched.SECURITY.mdstill points at GitHub's own bug bounty (inherited from the template, shared across your repos) — left alone deliberately, since changing a house-wide file is not mine to decide.Deferred, with reasons
Phase 2 OpenAPI ingestion · 3 codegen-spec interop · 4 the prober · 5 generated tests and mocks · 6 breadth and docs. The anchor region-patcher is unnecessary while the pilot is greenfield; it is only needed to adopt the toolkit into ms365 or jamfpro, which is phase 7.
🤖 Generated with Claude Code