[E2E] Add llm-d profile for E2E testing framework

[Xunzhuo]

Description

Add a new llm-d profile to the E2E testing framework to test Semantic Router integration with LLM-D (LLM Distributed System).

Background

The E2E testing framework introduced in #655 provides an extensible profile-based architecture. We need to add an llm-d profile to test Semantic Router deployment and functionality in LLM-D distributed environments.

Tasks

• Create e2e/profiles/llm-d/ directory structure
• Implement Profile interface for llm-d
  • Setup: Deploy LLM-D components
  • Setup: Deploy Semantic Router with LLM-D integration
  • Setup: Configure distributed inference settings
  • Teardown: Clean up LLM-D resources
• Implement test cases:
  • Basic health check with LLM-D
  • Distributed inference verification
  • Request routing across LLM-D nodes
  • Failover and recovery testing
  • Performance benchmarking
• Add documentation for llm-d profile usage
• Update CI workflow to run llm-d tests

Implementation Details

Profile Structure

type Profile struct {
    verbose bool
}

func (p *Profile) Setup(ctx context.Context, opts *framework.SetupOptions) error {
    // 1. Deploy LLM-D control plane
    // 2. Deploy LLM-D worker nodes
    // 3. Deploy Semantic Router with LLM-D backend
    // 4. Configure distributed inference settings
}

Test Cases

1. LLM-D Health Check: Verify all LLM-D components are healthy
2. Distributed Inference: Test inference across multiple nodes
3. Request Routing: Verify intelligent routing to LLM-D nodes
4. Failover: Test graceful failover when nodes fail
5. Performance: Measure distributed inference performance

Acceptance Criteria

• LLM-D profile can be run with make e2e-test PROFILE=llm-d
• All test cases pass successfully
• Documentation is complete and clear
• CI integration works correctly

References

• E2E Framework PR: [Feat] Add automate e2e test framework for extensible integration tests #655
• LLM-D Documentation: (add link)
• E2E Framework README: e2e/README.md

Related Issues

Part of the E2E testing framework expansion effort.

• Related to [E2E] Add Istio profile for E2E testing framework #656 (Istio profile)
• Related to [E2E] Add production-stack profile for E2E testing framework #657 (production-stack profile)

[Xunzhuo]

/area test-and-release
/priority P2
/label enhancement
/label e2e
/label help wanted

[samzong]

/assign @samzong

I am also paying attention to llm-d. let me try to it.

[Xunzhuo]

nice catch! try make e2e-test first in local to see how it works

[rootfs]

cc @srampal