ci: isolate dataplane conformance from control plane - #132
Draft
lucarlig wants to merge 3 commits into
Draft
Conversation
Signed-off-by: lucarlig <luca.carlig@ibm.com>
Signed-off-by: lucarlig <luca.carlig@ibm.com>
lucarlig
force-pushed
the
user/luca/standalone-conformance
branch
from
September 4, 2026 15:24
de756e6 to
754a713
Compare
Signed-off-by: lucarlig <luca.carlig@ibm.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Run
make conformanceandmake conformance-blessthrough the standalone external lane: Redis, dataplane, nginx, and the official fixture, without control-plane startup or publisher-schema timing. The workflow installscf-integration0.3.1.The companion harness change discovers all fixture routes and preserves the real tool schemas instead of maintaining an incomplete hardcoded catalog. Regenerating the baseline removes four false findings: the extra-parameter warning and three capability checks that previously hit missing tool routes. The external server baseline now contains 35 findings, down from 39; fixture-direct and client baselines are unchanged.
Document the limits of the baseline in the testing wiki. Strict matching detects regressions; it does not establish full conformance. Catalog-dependent scenarios cannot exercise checks after the dataplane rejects
tools/list, and the pinned fixture lacks a custom-header-annotated tool. Those coverage gaps remain explicit alongside existing protocol failures.Dependency
Depends on contextforge-org/contextforge-dev-tools#13. Keep this PR draft until
cf-integration0.3.1 is published with the fixture discovery fix.Validation
make conformance-bless, followed by clean strictmake conformance, using the companion PR's local binary and an image built from this PR's dataplane code.mdbook build _context/wikiandmdbook test _context/wikipassed.