-
Notifications
You must be signed in to change notification settings - Fork 37
Open
Labels
Description
Objective
Develop a comprehensive integration test suite that validates the agent container environment has parity with the GitHub Actions runner environment for essential tools, libraries, and environment variables.
Context
As we close the gap between the agent container and the action runner, we need automated tests to verify that:
- Mounted utilities are accessible and functional
- Runtime tools (Python, Node.js, Go, Ruby) can be found and executed
- Environment variables are correctly mirrored
- Shared libraries are properly linked
Without these tests, we risk regressions and incomplete implementations.
Approach
- Create a test workflow that runs inside the agent container and verifies:
- Essential
/usr/binutilities are accessible (jq,curl,git, etc.) - All runtime binaries are in PATH and executable (
node,python3,go,ruby) - Environment variables are set correctly (
JAVA_HOME,ANDROID_HOME, etc.) - Shared libraries can be loaded (test with
lddon key binaries)
- Essential
- Add Go integration tests in
pkg/workflow/*_test.gofiles - Create a smoke test workflow in
.github/workflows/agent-runner-parity.md - Document the test approach in
specs/agent-container-testing.md
Files to Create/Modify
- Create:
pkg/workflow/agent_parity_test.go(integration tests) - Create:
.github/workflows/agent-runner-parity.md(smoke test workflow) - Create:
specs/agent-container-testing.md(test documentation) - Modify:
Makefile(addmake test-paritytarget)
Acceptance Criteria
- Integration tests cover utility accessibility (at least 10 utilities)
- Integration tests verify runtime availability (node, python3, go, ruby)
- Integration tests validate environment variable presence (at least 5 vars)
- Smoke test workflow runs successfully in CI
- Test documentation explains testing strategy and coverage
- All tests pass consistently across multiple runs
Related to epic: build/test environment for agentic workflowย #11970
AI generated by Plan Command for #11970
Copilot