fix(conformance): map every shipped service to its real wire protocol#759
Merged
vieiralucas merged 2 commits intomainfrom Apr 25, 2026
Merged
fix(conformance): map every shipped service to its real wire protocol#759vieiralucas merged 2 commits intomainfrom
vieiralucas merged 2 commits intomainfrom
Conversation
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
`service_protocol()` only mapped 17 services. Everything else fell
through `_ => Protocol::Query`, sending form-encoded `Action=…` to `/`
for ECS / Step Functions / Bedrock / Bedrock Runtime / Scheduler / SES /
RDS / ElastiCache / ELBv2.
For the JSON / REST services in that list, the request never reaches
the right handler — APIGW's catch-all returns `NotFoundException: Stage
not specified`, the probe sees a 4xx without a "not implemented"
substring, and lenient classification calls it Pass. ECS reported
76/76 implemented even though only 60 ops are actually routed by the
service. ELBv2 / SES / Bedrock / etc. were similarly inflated.
Add explicit entries for the missing services:
- ecs → Json (target: AmazonEC2ContainerServiceV20141113)
- states → Json (target: AWSStepFunctions)
- ses → Rest (sesv2 restJson1)
- bedrock → Rest
- bedrock-runtime → Rest
- scheduler → Rest
- rds / elasticache / elasticloadbalancing → Query (matches Smithy)
Also adds a unit test that asserts every shipped service has an
explicit protocol mapping, so any new service added to the workspace
forces the conformance crate to declare its wire format.
Baseline regenerated: total variants 61,743 → 65,630 (probes that were
silently dropped now actually run); pass count 61,741 → 64,998 (real
gaps surfaced in ECS, ECR, Step Functions). Per-service breakdown:
- 23 of 26 services: 100% pass
- ECR: 56 / 58 fully passing (2 missing-validation gaps)
- ECS: 29 / 60 fully passing (variant edge-cases on routed ops; the
16 Daemon/ExpressGateway ops stay
NotImplemented per SUPPORTED list)
- States: 28 / 37 fully passing (paginator + validation gaps)
Real conformance gaps in those three services to be addressed in
follow-up commits.
vieiralucas
force-pushed
the
conformance-protocol-map
branch
from
f21470e to
d828a70
Compare
Three pre-existing test bugs that the protocol-map fix surfaced because this branch is the first to actually run the full conformance + e2e suites against post-#761 main: - crates/fakecloud-conformance/tests/logs.rs::logs_list_log_groups_for_query - crates/fakecloud-e2e/tests/logs.rs::logs_misc_stubs Both tests called `ListLogGroupsForQuery` with a synthetic queryId. PR #761's Cubic round (commit ad110ec) tightened the impl to return `ResourceNotFoundException` for unknown queryIds — matching real AWS — but the test sites still expected the previous "silently empty" answer. Fix: create a real query via `StartQuery` first, then look it up. - crates/fakecloud-e2e/tests/stepfunctions.rs::wait_for_execution 50 polls * 50ms = 2.5s. The PR-#757 SNS RSA-SHA256 signing + post-#760 SES-MIME work can push first-touch latency past that budget on GitHub-hosted runners under parallel load. Bump the budget to 10s. Locally executions still complete in <2s. The conformance probe-map change in this PR exposed these because `paths-filter` only triggers conformance + e2e when the conformance crate changes, so the failing tests rode in unnoticed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
service_protocol()only mapped 17 services. Everything else (ECS, Step Functions, Bedrock, Bedrock Runtime, Scheduler, SES, RDS, ElastiCache, ELBv2) fell through_ => Protocol::Query. JSON/REST services were probed as form-encodedAction=…, hit APIGW's catch-all, and returned 4xx — counted as Pass under lenient classification.Test plan
cargo test --release -p fakecloud-conformance --lib service_protocol_covers_every_shipped_servicepassescargo test --release -p fakecloud-conformance --lib(48+1 tests) passesfakecloud-conformance runagainst current baseline = no regressionsSummary by cubic
Mapped every shipped service to its real wire protocol to stop JSON/REST services from being misrouted as Query and falsely counted as passes. Regenerated the conformance baseline to surface real gaps (notably in
ecs,ecr, andstates), and fixed CI tests for Logs and Step Functions to match current behavior.ecs(Json: AmazonEC2ContainerServiceV20141113),states(Json: AWSStepFunctions),ses(Rest),bedrock(Rest),bedrock-runtime(Rest),scheduler(Rest),rds(Query),elasticache(Query),elasticloadbalancing(Query).fakecloud-conformancethat requires every shipped service to declare its protocol.PartialEq/EqforProtocolto enable exact assertions in tests.StartQuerybeforeListLogGroupsForQueryinfakecloud-conformanceandfakecloud-e2e.wait_for_executionbudget to 10s infakecloud-e2eto avoid flakiness on CI.ecs,ecr, andstates.Written for commit 331a4da. Summary will update on new commits.