Skip to content

Add e2e test#24

Merged
lwr20 merged 3 commits intoprojectcalico:mainfrom
lwr20:lwr-e2e
Aug 28, 2025
Merged

Add e2e test#24
lwr20 merged 3 commits intoprojectcalico:mainfrom
lwr20:lwr-e2e

Conversation

@lwr20
Copy link
Member

@lwr20 lwr20 commented Aug 22, 2025

This adds an e2e test to the repo.

@lwr20 lwr20 changed the title Lwr e2e Add e2e test Aug 22, 2025
@lwr20 lwr20 force-pushed the lwr-e2e branch 7 times, most recently from a48c5f8 to 403c630 Compare August 26, 2025 11:36
@lwr20 lwr20 marked this pull request as ready for review August 26, 2025 15:17
@lwr20 lwr20 force-pushed the lwr-e2e branch 2 times, most recently from 8934d62 to 63a80cb Compare August 26, 2025 18:45
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds comprehensive end-to-end (e2e) testing infrastructure to the tiger-bench tool. The e2e test validates that the tool runs correctly and produces expected output structure by comparing results against a reference file.

  • Adds Python test validation script to verify results.json structure and content
  • Includes shell script to orchestrate complete e2e test using KinD cluster
  • Integrates e2e testing into CI/CD pipeline and build system

Reviewed Changes

Copilot reviewed 7 out of 9 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
test_validate_results.py Python unittest script that validates results.json structure against reference and config
results.json.reference Reference JSON file containing expected test output structure for validation
kind-config.yaml KinD cluster configuration for e2e testing environment
e2e-testconfig.yaml Test configuration defining benchmarks to run during e2e tests
e2e-test.sh Shell script orchestrating complete e2e test workflow
Makefile Adds e2e-test target and related cleanup functionality
.github/workflows/ci.yml Integrates e2e testing into CI pipeline
Comments suppressed due to low confidence (1)

test_validate_results.py:1

  • Missing 'unit' field in the pod-pod throughput data structure on line 175, while pod-svc-pod has it on line 186. This creates inconsistent structure in the reference data.
import json

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment on lines +42 to +43
-v "$KUBECONFIG_PATH:/kubeconfig:ro" \
-v "$(pwd)/$TEST_YAML:/testconfig.yaml:ro" \
Copy link

Copilot AI Aug 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inconsistent kubeconfig path mapping. The script uses '/kubeconfig' as the container path but doesn't verify this matches what the tiger-bench tool expects for the kubeconfig location.

Suggested change
-v "$KUBECONFIG_PATH:/kubeconfig:ro" \
-v "$(pwd)/$TEST_YAML:/testconfig.yaml:ro" \
-v "$(pwd)/$TEST_YAML:/testconfig.yaml:ro" \
-e KUBECONFIG=/kubeconfig \

Copilot uses AI. Check for mistakes.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code and Dockerfile for the tiger-bench tool are in this repo. I think we want the e2e-test.sh to run the test exactly as specified in the README, including assuming where the tiger-bench tool expects to find the kubeconfig.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copy link

@radTuti radTuti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lwr20 is it possible to see a workflow run as well?

@lwr20
Copy link
Member Author

lwr20 commented Aug 28, 2025

Example workflow run from my fork:
https://github.com/lwr20/tiger-bench/actions/runs/17272703204

@lwr20
Copy link
Member Author

lwr20 commented Aug 28, 2025

We should probably have CI run on PRs. I think I'll do that in a new PR.

@lwr20 lwr20 merged commit 2649caa into projectcalico:main Aug 28, 2025
1 check passed
@lwr20 lwr20 deleted the lwr-e2e branch August 28, 2025 17:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants