Add conformance runner for the cross-SDK parity suite - #14
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a Python conformance runner (conformance/run.py) intended to execute a single cross-SDK parity scenario (read from stdin), invoke the corresponding real SDK surface, and emit a normalized JSON decision to stdout—mirroring the TypeScript conformance runner behavior.
Changes:
- Introduces a scenario-driven runner that supports
verify,enforce,customer-obtain, andcustomer-match. - Normalizes enforcement outputs by extracting key RSL headers into a stable shape.
- Implements
customer-matchby calling the same internal selection helpers used byobtain_license_token.
Comments suppressed due to low confidence (1)
conformance/run.py:105
SupertabConnect.set_base_url(MOCK_ORIGIN)has no effect onobtain_license_token()(it doesn’t consult the merchant client base URL). Dropping this avoids an unnecessary global state mutation.
async def _customer_obtain(inp: dict) -> dict:
SupertabConnect.set_base_url(MOCK_ORIGIN)
try:
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+117
to
+131
| async def _main() -> None: | ||
| scn = json.loads(sys.stdin.read()) | ||
| surface = scn["surface"] | ||
| inp = scn["input"] | ||
| if surface == "verify": | ||
| out = await _verify(inp) | ||
| elif surface == "enforce": | ||
| out = await _enforce(inp) | ||
| elif surface == "customer-match": | ||
| out = _customer_match(inp) | ||
| elif surface == "customer-obtain": | ||
| out = await _customer_obtain(inp) | ||
| else: | ||
| raise SystemExit(f"unhandled surface: {surface}") | ||
| sys.stdout.write(json.dumps(out)) |
_enforce configures supertab_base_url on the instance; obtain_license_token derives its origin from resource_url. Neither consults the global base URL, so the set_base_url mutations were redundant/dead. (Copilot review.)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
conformance/run.py— the Python SDK's runner for the cross-SDK behavioral-parity suite (connect-sdk-conformance).The runner reads one scenario on stdin, drives the real SDK entrypoint for the scenario's surface, and prints a normalized decision on stdout — the exact mirror of the TypeScript runner. It covers all four surfaces:
verify→verify_license_tokenenforce→SupertabConnect.handle_requestcustomer-obtain→obtain_license_tokencustomer-match→ the same private selection helpersobtain_license_tokenuses (_parse_content_elements→_find_serverless_usage_content→_find_best_matching_content), since there is no single public "match this request" entrypoint.Wired into the suite, the first two-SDK run is fully green — the Python SDK reaches identical decisions to TypeScript across the whole corpus, no overrides needed.
Additive only: one new file, no changes to SDK source.
🤖 Generated with Claude Code