Conversation
a48c5f8 to
403c630
Compare
8934d62 to
63a80cb
Compare
There was a problem hiding this comment.
Pull Request Overview
This PR adds comprehensive end-to-end (e2e) testing infrastructure to the tiger-bench tool. The e2e test validates that the tool runs correctly and produces expected output structure by comparing results against a reference file.
- Adds Python test validation script to verify results.json structure and content
- Includes shell script to orchestrate complete e2e test using KinD cluster
- Integrates e2e testing into CI/CD pipeline and build system
Reviewed Changes
Copilot reviewed 7 out of 9 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| test_validate_results.py | Python unittest script that validates results.json structure against reference and config |
| results.json.reference | Reference JSON file containing expected test output structure for validation |
| kind-config.yaml | KinD cluster configuration for e2e testing environment |
| e2e-testconfig.yaml | Test configuration defining benchmarks to run during e2e tests |
| e2e-test.sh | Shell script orchestrating complete e2e test workflow |
| Makefile | Adds e2e-test target and related cleanup functionality |
| .github/workflows/ci.yml | Integrates e2e testing into CI pipeline |
Comments suppressed due to low confidence (1)
test_validate_results.py:1
- Missing 'unit' field in the pod-pod throughput data structure on line 175, while pod-svc-pod has it on line 186. This creates inconsistent structure in the reference data.
import json
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| -v "$KUBECONFIG_PATH:/kubeconfig:ro" \ | ||
| -v "$(pwd)/$TEST_YAML:/testconfig.yaml:ro" \ |
There was a problem hiding this comment.
Inconsistent kubeconfig path mapping. The script uses '/kubeconfig' as the container path but doesn't verify this matches what the tiger-bench tool expects for the kubeconfig location.
| -v "$KUBECONFIG_PATH:/kubeconfig:ro" \ | |
| -v "$(pwd)/$TEST_YAML:/testconfig.yaml:ro" \ | |
| -v "$(pwd)/$TEST_YAML:/testconfig.yaml:ro" \ | |
| -e KUBECONFIG=/kubeconfig \ |
There was a problem hiding this comment.
The code and Dockerfile for the tiger-bench tool are in this repo. I think we want the e2e-test.sh to run the test exactly as specified in the README, including assuming where the tiger-bench tool expects to find the kubeconfig.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
Example workflow run from my fork: |
|
We should probably have CI run on PRs. I think I'll do that in a new PR. |
This adds an e2e test to the repo.