feat: Add aibrix profile for E2E testing framework

[yehudit1987]

• Add AIBrix profile implementing 5-stage deployment:

  • Semantic Router (Helm)
  • AIBrix Dependencies (Envoy Gateway CRDs)
  • AIBrix Core (Gateway Plugins, Metadata Service, Controller)
  • Demo LLM + Gateway API Resources
  • Environment verification with stabilization wait

• Implement robust error handling:

  • Track deployment state at each stage
  • Partial cleanup on failure to prevent orphaned resources
  • Panic recovery with defer-based cleanup

• Add configurable AIBrix version via AIBRIX_VERSION env var

• Integrate into CI with matrix strategy for parallel testing

  • Both ai-gateway and aibrix profiles tested on every PR
  • Separate test reports per profile

• Support all test cases: chat-completions, domain-classify,
  semantic-cache, pii-detection, jailbreak-detection, stress tests

  Resolve [E2E] Add aibrix profile for E2E testing framework #660

[netlify]

✅ Deploy Preview for vllm-semantic-router ready!

Name	Link
🔨 Latest commit	dd72c08
🔍 Latest deploy log	https://app.netlify.com/projects/vllm-semantic-router/deploys/691cc4f5e89d1f0008791024
😎 Deploy Preview	https://deploy-preview-688--vllm-semantic-router.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[github-actions]

👥 vLLM Semantic Team Notification

The following members have been identified for the changed files in this PR and have been automatically assigned:

📁 e2e

Owners: @Xunzhuo
Files changed:

• e2e/profiles/aibrix/profile.go
• e2e/README.md
• e2e/cmd/e2e/main.go

📁 Root Directory

Owners: @rootfs, @Xunzhuo
Files changed:

• .github/workflows/integration-test-k8s.yml

📁 deploy

Owners: @rootfs, @Xunzhuo
Files changed:

• deploy/kubernetes/aibrix/semantic-router-values/values.yaml

📁 tools

Owners: @yuluo-yx, @rootfs, @Xunzhuo
Files changed:

• tools/make/e2e.mk

---

🎉 Thanks for your contributions!

This comment was automatically generated based on the OWNER files in the repository.

[Copilot]

Pull Request Overview

This PR adds a new AIBrix profile to the E2E testing framework to enable testing of Semantic Router with vLLM AIBrix integration. The implementation follows the existing pattern established by the ai-gateway profile while adding robust error handling and cleanup mechanisms.

Key changes:

• Implemented 5-stage deployment for AIBrix (Semantic Router, AIBrix Dependencies, AIBrix Core, Demo LLM, environment verification)
• Added comprehensive error handling with partial cleanup and panic recovery
• Integrated into CI with matrix strategy for parallel testing of both ai-gateway and aibrix profiles

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
e2e/profiles/aibrix/profile.go	New AIBrix profile implementation with 5-stage deployment, error handling, and cleanup logic
e2e/cmd/e2e/main.go	Registered the new aibrix profile in the profile factory
tools/make/e2e.mk	Updated help text to document the aibrix profile
e2e/README.md	Updated documentation to list aibrix as a supported profile (moved from future to current)
.github/workflows/integration-test-k8s.yml	Added matrix strategy to test both profiles in parallel with separate artifact names

Comments suppressed due to low confidence (2)

.github/workflows/integration-test-k8s.yml:28

• Go version '1.24' does not exist as of January 2025. The latest stable Go version is 1.23. This should be corrected to a valid Go version like '1.23' or '1.22'.

          go-version: '1.24'

.github/workflows/integration-test-k8s.yml:33

• Rust version 1.90 does not exist as of January 2025. The latest stable Rust version is around 1.84. This should be corrected to a valid Rust version or use 'stable' instead.

          toolchain: 1.90

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[rootfs]

@yehudit1987 this is ready to go, can you resolve the conflict so we can run the latest e2e test. Thanks

[rootfs]

reading the log, PII policy is disabled and that caused the test failure

================================================================================
TEST RESULTS
================================================================================
✅ PASSED - chat-completions-request (2.439804138s)
✅ PASSED - chat-completions-stress-request (56.247452209s)
❌ FAILED - domain-classify (2m54.006573145s)
  Error: domain classification test failed: 0% accuracy (0/280 correct)
✅ PASSED - semantic-cache (14.79522197s)
❌ FAILED - pii-detection (50.703493022s)
  Error: PII detection test failed: 0% accuracy (0/100 correct)
✅ PASSED - jailbreak-detection (59.186211484s)
✅ PASSED - chat-completions-progressive-stress (32.544681604s)

[yehudit1987]

reading the log, PII policy is disabled and that caused the test failure

================================================================================
TEST RESULTS
================================================================================
✅ PASSED - chat-completions-request (2.439804138s)
✅ PASSED - chat-completions-stress-request (56.247452209s)
❌ FAILED - domain-classify (2m54.006573145s)
  Error: domain classification test failed: 0% accuracy (0/280 correct)
✅ PASSED - semantic-cache (14.79522197s)
❌ FAILED - pii-detection (50.703493022s)
  Error: PII detection test failed: 0% accuracy (0/100 correct)
✅ PASSED - jailbreak-detection (59.186211484s)
✅ PASSED - chat-completions-progressive-stress (32.544681604s)

All tests are passing now.

================================================================================
TEST RESULTS
================================================================================
✅ PASSED - chat-completions-request (2.703768398s)
✅ PASSED - chat-completions-stress-request (1m44.397821511s)
✅ PASSED - domain-classify (5m50.878113089s)
✅ PASSED - semantic-cache (21.788123278s)
✅ PASSED - pii-detection (1m18.554515807s)
✅ PASSED - jailbreak-detection (2m2.531104517s)
✅ PASSED - chat-completions-progressive-stress (48.51256606s)
================================================================================
Total: 7 | Passed: 7 | Failed: 0

[rootfs]

thanks @yehudit1987 let run the latest code for one last time

[Xunzhuo]

thanks a lot, nice work! would you like to try adding more testcases to make sure the new signal-decsion engines works well?

[yehudit1987]

thanks a lot, nice work! would you like to try adding more testcases to make sure the new signal-decsion engines works well?

Hi @Xunzhuo sure, I will and thanks for the feedback. Do we have an open issue for that? Want me to create one?

[yehudit1987]

@Xunzhuo will start to work on #692 let me know if the requirements are different or need some modifications.