Summary
Unit tests now run on Ubuntu, macOS, and Windows via the CI matrix (issue #25). However, VS Code extension integration tests and UI tests only run on the self-hosted macOS ARM64 runner.
Gap
Platform-specific differences in VS Code API behavior (file paths, URI handling, shell execution) are not tested on Linux or Windows. The extension supports remote environments (WSL, SSH, dev containers) which are primarily Linux-based.
Considerations
- Integration tests require
@vscode/test-electron which needs a display (Xvfb on Linux, screen on Windows)
- UI tests use ExTester and depend on VS Code's native window management
- The self-hosted runner handles macOS-specific window suppression via
scripts/hide-test-vscode.sh
Proposed approach
- Add a Linux integration test job using
ubuntu-latest with xvfb-run
- Consider Windows integration tests if significant user adoption occurs on that platform
- Keep UI tests (ExTester) on macOS self-hosted only since they depend on native window management
Priority
Low. Unit tests catch most logic issues. Integration tests are primarily validating VS Code API interactions which are largely platform-independent.
Summary
Unit tests now run on Ubuntu, macOS, and Windows via the CI matrix (issue #25). However, VS Code extension integration tests and UI tests only run on the self-hosted macOS ARM64 runner.
Gap
Platform-specific differences in VS Code API behavior (file paths, URI handling, shell execution) are not tested on Linux or Windows. The extension supports remote environments (WSL, SSH, dev containers) which are primarily Linux-based.
Considerations
@vscode/test-electronwhich needs a display (Xvfb on Linux, screen on Windows)scripts/hide-test-vscode.shProposed approach
ubuntu-latestwithxvfb-runPriority
Low. Unit tests catch most logic issues. Integration tests are primarily validating VS Code API interactions which are largely platform-independent.