MAF-19287: feat(tests): enhance E2E testing framework and add functional tests#81
Merged
MAF-19287: feat(tests): enhance E2E testing framework and add functional tests#81
Conversation
…nal tests - Introduced new Makefile targets for E2E testing, including `test-e2e-functional`, `setup-test-e2e`, and `cleanup-test-e2e` to manage Kind cluster lifecycle. - Added a new GitHub Actions workflow for functional E2E tests on a product cluster, ensuring prerequisites are met and results are published. - Updated existing E2E tests to align with new structure, emphasizing hardcoded configurations and environment variable management. - Removed deprecated configurations and streamlined test setup for improved clarity and maintainability. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
… integration - Added a new curl job to send inference requests, replacing the previous inline curl command. - Refactored the InferenceService lifecycle tests to utilize the new curl job for better modularity and maintainability. - Updated the InferenceService creation function to improve clarity in parameter naming. - Introduced new configuration options for Loki in the mif-values.yaml template. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…io revision support - Added support for Istio revision labels in the Gateway configuration. - Updated the InferenceService lifecycle tests to pass the Istio revision as a parameter when creating Gateway resources. - Enhanced data structure for template rendering to accommodate the new Istio revision field.
…ke tests - Updated Makefile to replace functional E2E tests with smoke tests, enhancing clarity in test categorization. - Introduced a new GitHub Actions workflow for smoke E2E tests, ensuring proper setup and execution. - Added multiple smoke test files to validate InferenceService functionality, including curl job integration for request verification. - Enhanced documentation in AGENTS.md to reflect changes in test structure and naming conventions.
…all v0.7.1 - Updated PD disaggregation and prefix cache test files to reference Heimdall version v0.7.1. - Refactored InferenceService configurations to improve clarity and maintainability. - Enhanced test setup by creating model persistent volumes and claims for better resource management. - Adjusted curl job configurations to utilize the new model and ensure accurate request verification.
- Deleted the prefix_cache_test.go file as part of the cleanup process. - This change streamlines the test suite by removing outdated or redundant tests.
- Deleted the e2e-smoke-p-cluster GitHub Actions workflow file as part of the cleanup process. - This change helps streamline the testing framework by removing an outdated workflow that is no longer needed.
…EQUISITE is consistently set and maintain ISTIO_REV configuration
…inference performance cluster workflow
…d structure - Clarified shared configuration values to include infrastructure component versions. - Updated suite-level setup/teardown instructions for better clarity on prerequisites. - Added a new section to ensure AGENTS.md is kept in sync with test structure changes. - Specified default behavior for the SKIP_PREREQUISITE environment variable.
- Eliminated redundant debug steps for test report files in both e2e-inference and e2e-quality benchmark workflows. - This cleanup enhances workflow clarity and focuses on essential tasks.
Contributor
There was a problem hiding this comment.
Pull request overview
E2E 테스트 프레임워크를 정리하고(Kind 클러스터 수명주기 분리, 템플릿 렌더링 방식 정리), smoke/performance/quality 테스트를 기능 테스트 형태로 확장·정비하는 PR입니다.
Changes:
- Kind 클러스터 생성/삭제 로직을 Go 테스트 코드에서 제거하고 Makefile 타깃(
setup-test-e2e/cleanup-test-e2e)로 이전 - E2E 리소스 템플릿 처리 개선(파일 템플릿 vs 문자열 템플릿 분리) 및 smoke 테스트 스위트/시나리오 추가
- performance/quality 테스트의 템플릿 파일들을 테스트 코드 내 const YAML로 이동하고, 워크플로우/환경변수 문서 업데이트
Reviewed changes
Copilot reviewed 31 out of 33 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| test/utils/settings/constants.go | 테스트 인프라 관련 상수 정리(일부 제거/재배치) |
| test/utils/models.go | 모델 PV/PVC 생성 시 파일 템플릿 렌더링 헬퍼로 변경 |
| test/utils/mif.go | MIF values 템플릿을 파일 렌더링 헬퍼로 변경 |
| test/utils/kind.go | Kind 클러스터 제어 유틸 제거(수명주기 외부화) |
| test/utils/inference_service.go | InferenceService 생성 API를 “YAML 문자열 템플릿” 기반으로 단순화 |
| test/utils/heimdall.go | Heimdall 설치 시 버전 인자화 및 Helm repo URL 하드코딩 |
| test/utils/gateway.go | Gateway 컨트롤러 설치/게이트웨이 리소스 생성 흐름 조정 및 Gateway service name 조회 함수 이동 |
| test/utils/config/model-pvc.yaml.tmpl | Go 템플릿 스타일 정리(공백 포함) |
| test/utils/config/model-pv.yaml.tmpl | Go 템플릿 스타일 정리(공백 포함) |
| test/utils/config/mif-values.yaml.tmpl | MIF values 템플릿 내용 변경(loki 캐시 설정 등) |
| test/utils/config/gateway-kgateway.yaml.tmpl | Gateway 이름을 고정값으로 변경 |
| test/utils/config/gateway-istio.yaml.tmpl | Istio gateway 리소스 포트 설정 추가 및 Gateway 이름 고정값 변경 |
| test/utils/common.go | RenderTemplate를 문자열 템플릿 렌더로 재정의 + 파일 렌더링 헬퍼 추가 |
| test/utils/certmanager.go | cert-manager 설치 및 “설치 여부 판단” 로직 개선(CRD+deployment) |
| test/e2e/smoke/suite_test.go | 신규 smoke 스위트(사전요건 설치/정리 포함) |
| test/e2e/smoke/pd_disaggregation_test.go | PD disaggregation smoke 시나리오 추가 |
| test/e2e/smoke/inference_service_test.go | InferenceService smoke 시나리오 추가 |
| test/e2e/quality/suite_test.go | Kind 관련 분기 제거 및 cert-manager 설치체크 변경 |
| test/e2e/quality/quality_test.go | 템플릿 파일 제거 후 const YAML 기반으로 품질 벤치마크 구성 정리 |
| test/e2e/quality/config/quality-benchmark-job.yaml.tmpl | 품질 벤치마크 Job 템플릿 파일 삭제(코드 내 const로 이동) |
| test/e2e/quality/config/inference-service.yaml.tmpl | InferenceService 템플릿 파일 삭제(코드 내 const로 이동) |
| test/e2e/quality/config/heimdall-values.yaml.tmpl | Heimdall values 템플릿 파일 삭제(코드 내 const로 이동) |
| test/e2e/performance/suite_test.go | Kind 관련 분기 제거 및 cert-manager 설치체크 변경 |
| test/e2e/performance/performance_test.go | 템플릿 파일 제거 후 const YAML 기반으로 성능 테스트 구성 정리 + S3 prefix 리포팅 추가 |
| test/e2e/performance/config/inference-service.yaml.tmpl | InferenceService 템플릿 파일 삭제(코드 내 const로 이동) |
| test/e2e/performance/config/inference-perf-job.yaml.tmpl | inference-perf Job 템플릿 파일 삭제(코드 내 const로 이동) |
| test/e2e/performance/config/heimdall-values.yaml.tmpl | Heimdall values 템플릿 파일 삭제(코드 내 const로 이동) |
| test/e2e/envs/env_vars_doc.go | 환경변수 안내 문구 업데이트(고정값 하드코딩 정책 반영) |
| test/e2e/envs/env_vars.go | 환경변수 축소 및 기본값/카테고리 재정의(특히 SKIP_PREREQUISITE 기본값 변경) |
| test/AGENTS.md | test/ 디렉터리 규칙 업데이트(클러스터 수명주기 외부화 등) |
| Makefile | smoke 타깃 추가 및 Kind setup/cleanup 타깃 신설 |
| .github/workflows/e2e-quality-benchmark-p-cluster.yaml | 불필요 env 제거 및 워크플로우 단순화 |
| .github/workflows/e2e-inference-perf-p-cluster.yaml | 불필요 env 제거, 워크로드 네임스페이스 규칙 변경 등 |
…te checks are skipped - Modified the test-e2e-kind target to set SKIP_PREREQUISITE to false, ensuring that prerequisite checks are bypassed during smoke tests on a local Kind cluster.
hhk7734
approved these changes
Mar 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
이 PR으로 e2e 테스트 코드를 정리하고, PR merge 이후에 heimdall에서의 기능 테스트로 마이그레이션 진행하겠습니다.