test(test-vectors): add differential harness across sdk versions - #163
Open
DSOTec wants to merge 4 commits into
Open
test(test-vectors): add differential harness across sdk versions#163DSOTec wants to merge 4 commits into
DSOTec wants to merge 4 commits into
Conversation
No safety net today catches a version bump that silently changes cryptographic output for existing inputs — semver numbers alone don't verify actual behavior. Add differential.ts: it installs the pinned reference version (vN-1, named in differential.config.json) into a throwaway directory, runs every vectors/stellar.json fixture through both that build and the workspace tip via the shared differential-runner.mjs, and diffs the outputs field by field. Unwaived diffs fail the run; waivers in differences.json require a non-empty reason and the script refuses to run without one. Running the harness for real against the pinned reference (1.3.0) surfaced two genuine issues: - generateStealthAddress's default view-tag scheme changed (legacy prefix to the domain-separated v2 prefix) in commit f7def68 (PR wraith-protocol#93), before the 1.4.0 minor, without a major bump. This is a real, already-shipped protocol change — documented and waived in differences.json with the commit reference, exactly the kind of entry this file exists to hold. - signWithScalar had a leftover debug `console.log` that wrote to stdout on every signing call, corrupting the runner's JSON-over-stdout protocol (and explaining the stray "SCALAR:" noise in the existing test output). Removed it. Wire a `differential` job into ci.yml that runs on every PR touching src/chains/** (via dorny/paths-filter) and unconditionally on push to main/develop, so the acceptance criterion of a green run on develop is checked directly rather than assumed. Closes wraith-protocol#132
|
@DSOTec Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
DSOTec
force-pushed
the
feat/132-test-vectors-differential-harness
branch
from
July 28, 2026 10:49
afe9429 to
638bd55
Compare
The 'Merge branch develop into feat/132-test-vectors-differential-harness' merge commit left pnpm-lock.yaml missing an entry for @stellar/stellar-sdk@13.3.0, so every CI job died within seconds at the first 'pnpm install --frozen-lockfile' step, before continue-on-error steps further down even ran. Regenerate the lockfile with 'pnpm install --no-frozen-lockfile'. Verified locally after the fix: pnpm install --frozen-lockfile succeeds, format/build/test all pass (1073 tests), the differential harness still runs clean against the merged-in chains code, and every Examples CI matrix example (including the new stellar-chrome-extension from upstream) installs, type-checks, and builds.
CI/bun was cancelled after the full 6-hour job ceiling:
test/leaks/scan-leak.test.ts:
Error: The operation was canceled.
Reproduced locally: `bun test test/leaks/scan-leak.test.ts` hangs
indefinitely with zero output (killed after 30s), most likely inside its
v8.getHeapSnapshot() stream handling, which bun's own --timeout doesn't
preempt. That file is already excluded from the pnpm `test` script for
being slow/resource-intensive; bun just turns "slow" into "hangs forever."
Excluding it isn't enough on its own, though: upstream/develop already
removed this job entirely in 885ec6e ("ci: drop the aspirational bun job"),
because bun test also fails ~45 more tests unrelated to the hang (vi.hoisted
and vi.importActual aren't implemented in bun's vitest-compat layer, so
every sdk-react/sdk-vue mocking-based test fails, plus a stale dist/
build artifact and a genuine assertion-style mismatch in one pipeline
test). None of that has anything to do with this PR's changes.
The job is only present on this branch because an earlier merge from
develop resolved the conflict in ci.yml by keeping it instead of taking
upstream's deletion. Remove it again so this branch matches the decision
upstream already made and merged, rather than re-fighting it here.
Verified: this branch's ci.yml now diffs from upstream/develop's only by
the addition of this PR's own `differential` job. format/build/test all
still pass locally (1073 tests).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No safety net today catches a version bump that silently changes cryptographic output for existing inputs — semver numbers alone don't verify actual behavior. Add differential.ts: it installs the pinned reference version (vN-1, named in differential.config.json) into a throwaway directory, runs every vectors/stellar.json fixture through both that build and the workspace tip via the shared differential-runner.mjs, and diffs the outputs field by field. Unwaived diffs fail the run; waivers in differences.json require a non-empty reason and the script refuses to run without one.
Running the harness for real against the pinned reference (1.3.0) surfaced two genuine issues:
console.logthat wrote to stdout on every signing call, corrupting the runner's JSON-over-stdout protocol (and explaining the stray "SCALAR:" noise in the existing test output). Removed it.Wire a
differentialjob into ci.yml that runs on every PR touching src/chains/** (via dorny/paths-filter) and unconditionally on push to main/develop, so the acceptance criterion of a green run on develop is checked directly rather than assumed.Closes #132