Skip to content

wheels-docker-20251004-12c36f19

@oberstet oberstet tagged this 04 Oct 07:43
* Add WebSocket conformance testing workflow (wstest.yml)

## New Workflow: wstest.yml

### WebSocket RFC 6455 Conformance Testing
- Tests all 4 combinations: Twisted/asyncio × client/server
- Uses official Autobahn|Testsuite Docker container
- Supports quick (CI) and full (comprehensive) test modes

### Key Features
- **Matrix Testing**: 4 scenarios covering both frameworks and roles
- **Smart Configuration**: Quick vs full test modes via workflow dispatch
- **Docker Integration**: Proper container lifecycle management
- **Robust Execution**: Timeout protection and error handling
- **Professional Reports**: Consolidated HTML reports with landing page

### Test Configuration
- Uses existing wstest/ config files (fuzzingclient/server quick/full.json)
- Mounts config and reports directories into testsuite container
- Executes appropriate testee scripts (testee_client/server_aio/tx.py)

### Report Generation
- Individual reports per backend/role combination
- Consolidated index page linking all test results
- 30-day artifact retention for analysis
- Ready for RTD documentation integration

### Architecture

This establishes the foundation for:
1. NVX XOR masker validation (once implemented)
2. Documentation integration with comprehensive test reports
3. Professional WebSocket conformance verification

Usage:
- Automatic: Runs 'quick' mode on push/PR
- Manual: workflow_dispatch with 'quick' or 'full' modes

* ensure GITHUB_TOKEN is set to work around download rate limits

* add audit file for branch

* add wstest recipes to justfile for local testing

* add more wstest recipes to justfile for local testing

* Fix Docker commands to use sudo in wstest recipes

All Docker commands in justfile now use 'sudo docker' prefix
to work with systems requiring sudo for Docker access.

* Complete WebSocket server testing workflow implementation

- Add missing justfile recipes: wstest-fuzzingclient, wstest-testeeserver-twisted, wstest-testeeserver-asyncio
- Fix Python 3.14 asyncio event loop compatibility in testee_client_aio.py and testee_server_aio.py
- Update fuzzingclient configs to test Twisted (port 9011) and asyncio (port 9012) servers
- Add URL parameters to server recipes with sensible defaults
- Fix Docker networking with --network host for localhost server access
- Enable consolidated server conformance testing workflow

* Rework wstest.yml workflow to use justfile recipes

- Replace complex Docker orchestration with justfile recipe calls
- Split into 3 jobs: client-conformance, server-conformance, consolidate-reports
- Client testing: Matrix strategy using wstest-testeeclient-twisted/asyncio recipes
- Server testing: Concurrent servers using wstest-testeeserver-* + wstest-fuzzingclient recipes
- Improved error handling with proper background process management (nohup + PID files)
- Enhanced consolidated report with separate client/server sections and modern styling
- Ensure perfect parity between local development and CI testing workflows

* Fix asyncio WebSocket server URL parsing

- Add missing host parameter in create_server() call
- Previously was only binding to port, causing server startup failures
- Ensures proper host:port binding for WebSocket server conformance testing

* use just recipe to start testee server

* Fix GitHub Actions workflow syntax error in consolidate-reports job

- Fix incorrect EOF marker indentation in heredoc blocks
- Both client and server report generation had improper EOF placement
- Resolves 'syntax error: unexpected end of file' in CI workflow

* Rework WebSocket conformance testing workflow for Sphinx integration

- Remove complex HTML generation from GitHub Actions workflow
- Simplify consolidate-reports job to copy HTML/JSON reports to docs/_static/
- Create docs/websocket/conformance.rst documentation page
- Add conformance testing to main documentation TOC
- Generate ZIP archive of all JSON test reports for download
- Integrate test results properly with Sphinx documentation system
- Fixes YAML syntax issues and improves maintainability

* Expand WebSocket conformance testing to support multiple Python versions

- Add quick|full test mode control to wstest-fuzzingserver recipe
- Add quick|full test mode control to wstest-fuzzingclient recipe
- Create wstest-consolidate-reports recipe for local docs integration
- Expand fuzzingclient configs to test 6 server URLs (twisted|asyncio X cpy311|cpy314|pypy311)
- Update conformance.rst with comprehensive local testing workflow
- Support testing matrix: (client|server) X (twisted|asyncio) X (cpy311|cpy314|pypy311) X (quick|full)
- Enable local generation of docs/_static content for Sphinx builds

* fixed & polish for websocket conformance testing & docs integration

* sync FS after write, before du

* Rework GitHub Actions workflow for full Python version matrix testing

- Add Python version matrix: cpy311, cpy314, pypy311
- Simplify workflow to use proven justfile recipes directly
- Client testing: 6 jobs (twisted|asyncio × 3 Python versions)
- Server testing: 1 job testing all 6 server combinations simultaneously
- Remove complex shell scripting in favor of battle-tested justfile recipes
- Support both quick and full test modes via workflow dispatch
- Streamlined artifact organization and consolidation
- Matrix coverage: (client|server) × (twisted|asyncio) × (cpy311|cpy314|pypy311)

* Fix GitHub Actions workflow triggers and restore original behavior

- Restore push/pull_request triggers for automatic execution
- Fix workflow name back to 'wstest'
- Restore original UV_CACHE_DIR location and comments
- Keep the improved Python version matrix and justfile-based approach
- Workflow will now run automatically on pushes and PRs again

* Fix fuzzingserver startup timing and error handling in CI

- Increase startup timeout from 10s to 30s with retry loop
- Add proper error logging when fuzzingserver fails to start
- Add PID tracking and cleanup for fuzzingserver processes
- Better visibility into Docker startup issues in CI environment
- Prevent client test jobs from hanging on startup failures

* add GITHUB_TOKEN; improve logging when server failed to start on port

* add GITHUB_TOKEN; improve logging when server failed to start on port - REALLY

* remove tox from dev deps

* Fix WebSocket server startup by using sequential start with port confirmation

- Start 6 server combinations sequentially instead of simultaneously
- Wait for each server port to be listening before starting the next one
- 30-second timeout per server with detailed error logging
- Clear progress reporting for each server startup
- Prevents race conditions and dependency installation conflicts
- More reliable server startup in CI environment

* setup python and install autobahn with all deps in job step before starting testee servers

* Fix WebSocket server startup by bypassing justfile dependencies

- Replace justfile server recipes with direct Python script execution
- Eliminates unnecessary dependency installation during server startup
- Fixes failure where 'just wstest-testeeserver-*' was reinstalling packages
- Servers now start directly: .venvs/{python}/bin/python ./wstest/testee_server_{tx|aio}.py
- Resolves 'Error: Process completed with exit code 1' during cpy314 server startup
- Much faster server startup without redundant package installation
Assets 2
Loading