feat(e2e): enhance CI/CD with parallel E2E testing and performance optimizations #145
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ommands - Reduced test timeout for server capability collection from 0.5s to 0.2s to enhance performance during testing. - Enhanced error handling in `setServer` function to create a new configuration if the config file is not found, improving robustness. - Updated E2E tests to provide clearer error messages and better assertions for various command scenarios, including handling of disk space and permission errors. - Refactored command workflows to ensure consistent handling of server enable/disable/remove operations, processing them one by one for better clarity. - Improved assertions in MCP add command tests to ensure existing servers are maintained and provide detailed error messages when expectations are not met.
…iguration - Introduced a new script `test-e2e-performance.js` to evaluate E2E performance improvements through sequential and parallel execution. - Updated `vitest.e2e.config.ts` to enable file-level parallelism and set maximum concurrency for faster test execution. - Added a new command `test:e2e:performance` in `package.json` to facilitate performance testing. - Enhanced GitHub Actions workflow to run E2E tests in parallel with sharding for improved efficiency on every push/PR.
…ntManager mocks - Replaced ServerManager mock with ClientManager mock in E2E tests for OAuth notifications. - Updated tests to utilize `completeOAuthAndReconnect` method for handling OAuth flow. - Changed mock reset strategy from `resetAllMocks` to `clearAllMocks` to preserve mock implementations. - Enhanced assertions to ensure correct behavior during OAuth callback success and failure scenarios.
…narios - Updated assertions in the command workflows integration tests to account for varying output messages in CI environments, particularly for the backup command. - Added checks for backup creation messages, absence of configuration files, and skipped messages to ensure robust test coverage across different scenarios.
xizhibei
changed the title
…g script - Updated GitHub Actions workflow to run E2E tests in parallel using 4 shards, improving execution speed by approximately 75%. - Enhanced the `test-e2e-performance.js` script to include statistical analysis for sequential and multi-worker executions, providing detailed performance metrics. - Improved timeout handling in the `collectServerCapabilities` function for better performance during testing. - Refactored the `setServer` function to check for the existence of the configuration file before loading, enhancing robustness.
- Updated GitHub Actions workflow to conditionally skip E2E tests when the reference type is a tag, enhancing release efficiency. - This change aims to reduce unnecessary test execution during version tagging, streamlining the CI process.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🚀 Features & Enhancements
E2E Testing Performance Improvements
scripts/test-e2e-performance.jsfor comprehensive testing of sequential vs parallel executionpnpm test:e2e:performancefor performance benchmarkingCI/CD Workflow Optimization
Code Quality & Test Reliability
setServer()utility🛠 Technical Details
Performance Optimizations
fail-fast: falseensures all shards complete for comprehensive resultsTesting Infrastructure
Code Improvements
🧪 Testing
📊 Performance Impact
This change significantly improves the development workflow by making E2E testing automated, faster, and more reliable while maintaining comprehensive test coverage.