feat(tfacc): run upstream terraform-provider-aws acc tests against fakecloud#369
Merged
vieiralucas merged 4 commits intomainfrom Apr 14, 2026
Merged
feat(tfacc): run upstream terraform-provider-aws acc tests against fakecloud#369vieiralucas merged 4 commits intomainfrom
vieiralucas merged 4 commits intomainfrom
Conversation
…kecloud New `fakecloud-tfacc` crate invokes `hashicorp/terraform-provider-aws` acceptance tests (`TestAcc*`) against a live fakecloud instance. This is the semantic test layer that catches waiter / field-presence / drift bugs like PR #362, which pure SDK round-trip tests miss. - Crate shallow-clones terraform-provider-aws@v5.97.0 into `target/tfacc/` on first run and patches `go.mod` to remove a go1.24-incompatible godebug directive. Skips cleanly when go or terraform are missing. - Two-layer opt-in: `SERVICES` allow-list for services; each service carries a deny-list for upstream tests that are either structurally unsupportable or track a known fakecloud gap. Every deny entry has a reason comment. - One `#[tokio::test]` per allow-listed service, invoking `go test -parallel 8` with `AWS_ENDPOINT_URL_*` env vars pointed at a fresh fakecloud process per Rust test. - CI workflow fans out one runner per service (matrix generated by the new `tfacc_services` helper binary, same pattern as e2e.yml) and caches `~/.cache/go-build`, `~/go/pkg/mod`, and `target/tfacc` so the provider checkout survives across runs. - Batch 1 covers DynamoDB: 38 `TestAccDynamoDBTable_*` tests green against fakecloud in ~2.5 min locally; 38 tests denied with reasons (Global Tables, INFREQUENT_ACCESS, streams gaps, GSI update gaps, deletion_protection, backup encryption, etc). Later batches expand the allow-list and drive the "gap" deny entries to zero. - Small fakecloud fix revealed during triage: DynamoDB DescribeTable no longer emits an `SSEDescription` block for tables using the default AWS-owned key. Real AWS omits the field in that case; the previous stub made `TestAccDynamoDBTable_basic` fail on the drift check. Follow-ups tracked in `crates/fakecloud-tfacc/src/allowlist.rs`: - widen the dynamodb `run_regex` beyond `TestAccDynamoDBTable_` - drive the "gap:" deny entries for dynamodb to zero - add sqs and other services to the allow-list
There was a problem hiding this comment.
1 issue found across 10 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="crates/fakecloud-tfacc/src/lib.rs">
<violation number="1" location="crates/fakecloud-tfacc/src/lib.rs:99">
P2: `strip_godebug` rewrites `go.mod` on every call because the comparison ignores trailing newline behavior from `lines()`.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
- strip_godebug: skip the write when no godebug line exists (Cubic P2 finding). The previous comparison swallowed the file's trailing newline via `lines()`, causing the harness to rewrite go.mod on every invocation even when nothing changed. - e2e: rename `dynamodb_sse_default_aes256` to `dynamodb_sse_default_omitted` and assert the new, correct behaviour — DescribeTable no longer emits an SSEDescription block when the table uses the default AWS-owned key.
…skip Silently passing when the toolchain is absent just hides regressions — if someone runs this crate they've opted into exercising the upstream Terraform suite, and a missing binary should be a loud failure with install instructions, not a green check on a machine that never ran the code. Replaces `toolchain_available() -> bool` with `require_toolchain()` which panics with an actionable message naming the missing binaries.
The new tfacc crate hard-fails when go/terraform are missing (by design — silently skipping would hide regressions). The generic `cargo test --workspace` job in ci.yml/release.yml doesn't install that toolchain, so it must opt out the same way it already opts out of fakecloud-e2e and fakecloud-conformance.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
fakecloud-tfacccrate runshashicorp/terraform-provider-awsTestAcc*against a live fakecloud. This is the semantic test layer PR Fix DynamoDB DescribeTable warm throughput response #362 highlighted as a gap: pure SDK round-trip tests tolerate missing optional fields, but the Terraform provider enforces waiters, field presence, and drift. Now we catch that class of bug in CI.TestAccDynamoDBTable_*tests green against fakecloud in ~2.5 min locally; 38 tests denied with reasons (Global Tables, INFREQUENT_ACCESS, GSI update gaps, deletion_protection, etc). Future batches widen the allow-list and drive the `gap:` entries to zero./.cache/go-build`, `/go/pkg/mod`, and the shallow provider checkout in `target/tfacc`.How it works
Test plan
Summary by cubic
Adds
fakecloud-tfaccto run upstreamhashicorp/terraform-provider-awsacceptance tests against a live fakecloud. First batch enables DynamoDB with 38 passing table tests and fixes SSE parity to prevent Terraform drift.New Features
fakecloud-tfacccrate runs upstreamgo testagainst fakecloud viaAWS_ENDPOINT_URL_*.TestAccDynamoDBTable_*(38 pass, 38 skipped).terraform-provider-aws@v5.97.0intotarget/tfacc/and strips the Go 1.24-incompatiblegodebugdirective.tfacc_services), caches~/.cache/go-build,~/go/pkg/mod, andtarget/tfacc, and excludesfakecloud-tfaccfrom genericci.yml/release.ymltest jobs so it runs only intfacc.yml.goorterraformare missing.Bug Fixes
DescribeTableno longer returnsSSEDescriptionwhen using the default AWS-owned key; updated e2e test to assert omission to match AWS and fix provider drift checks.go.modrewrites when nogodebug tlskyberline exists; preserve trailing newlines during patching.Written for commit 6afe4b0. Summary will update on new commits.