Skip to content

Add AI Semantic Router Plugin #11305

@danehans

Description

@danehans

Add a plugin that utilizes Envoy External Processor (ExtProc) to act as an external Mixture-of-Models (MoM) router. The router should intelligently direct OpenAI API requests to the most suitable backend GenAI model from a defined pool based on the semantic understanding of the request's intent.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Area: InferenceActivities related to Gateway API Inference Extension support.Priority: HighRequired in next 3 months to make progress, bugs that affect multiple users, or very bad UXhelp wantedDenotes an issue that needs help from a contributor.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions