fix: resolve CLI path for tier-check conformance subprocess#175
Merged
pcarleton merged 3 commits intomodelcontextprotocol:mainfrom Mar 25, 2026
Merged
Conversation
Use process.argv[1] instead of relative dist/index.js so tier-check works when invoked via npx or from outside the package directory.
pcarleton
approved these changes
Mar 12, 2026
commit: |
Member
|
Thank you for this! |
pcarleton
added a commit
to jeffhandley/mcp-conformance
that referenced
this pull request
Mar 25, 2026
Combines the npx-compatibility fix from modelcontextprotocol#175 with this PR's execFileSync approach. process.argv[1] resolves to the absolute path of the running CLI regardless of working directory, so tier-check works when invoked via npx or from any cwd.
9 tasks
pcarleton
added a commit
that referenced
this pull request
Mar 25, 2026
* Fix Windows compatibility in tier-check client conformance Two issues prevented client conformance tests from running on Windows: 1. Single-quote quoting in execSync: The --command argument was wrapped in single quotes, which cmd.exe treats as literal characters rather than string delimiters. This caused the CLI to receive only the first word of the command. Switched to execFileSync with an args array to bypass shell quoting entirely. 2. Backslash path separators in parseOutputDir: dirname() on Windows returns auth\scenario-name but reconcileWithExpected compares against auth/scenario-name, causing auth scenarios to be double-counted. Normalize backslashes to forward slashes in parsed scenario names. * fix: use process.argv[1] for CLI path in subprocess calls Combines the npx-compatibility fix from #175 with this PR's execFileSync approach. process.argv[1] resolves to the absolute path of the running CLI regardless of working directory, so tier-check works when invoked via npx or from any cwd. --------- Co-authored-by: Paul Carleton <paulc@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
tier-checkshells out to run conformance tests using a relative path:When invoked via
npx(or from any directory other than the conformance package root),process.cwd()points to the user's working directory, not the package directory.dist/index.jsdoesn't exist there, so the command fails silently (the error is caught on the next line), the output directory is empty, andreconcileWithExpectedcounts every expected scenario as a failure — producing 0% conformance scores.Fix
Replace
node dist/index.jswithnode "${process.argv[1]}", which resolves to the absolute path of the currently running CLI entry point regardless of working directory. Also removes the now-unnecessarycwd: process.cwd().Testing
Ran
tier-checkfrom/tmpagainst my Swift SDK's conformance server.Related
PR #168 addresses a similar issue on Windows (single-quote quoting in
execSync) but retains the relativedist/index.jspath, so thenpxproblem would persist even after that fix.Breaking Changes
None
Types of changes
Checklist
I have added appropriate error handling(n.a.)I have added or updated documentation as needed(n.a.)