Provider Routing

Provider Routing

PolicyAware supports vendor-neutral routing across local and managed model providers.

Supported Adapter Types

• Simulated local provider for development and tests.
• OpenAI-compatible APIs.
• Azure OpenAI.
• Anthropic.
• Amazon Bedrock.
• Google Vertex AI.
• Ollama.
• vLLM.

Routing Flow

flowchart TD
    A["Allowed request"] --> B["Model router"]
    B --> C["Filter by policy"]
    C --> D["Filter by region and capability"]
    D --> E["Score cost, quality, latency, availability"]
    E --> F["Primary provider"]
    F -->|failure or unavailable| G["Fallback provider"]
    F --> H["Runtime evaluation"]
    G --> H

Loading

Provider Registry Example

from policyaware import Gateway
from policyaware.policy import PolicyEngine
from policyaware.providers import (
    ProviderRegistry,
    SimulatedProvider,
    OllamaProvider,
    VLLMProvider,
)

registry = ProviderRegistry(
    {
        "local": SimulatedProvider(),
        "ollama": OllamaProvider(base_url="http://localhost:11434"),
        "vllm": VLLMProvider(base_url="http://localhost:8000"),
    }
)

gateway = Gateway(
    policy_engine=PolicyEngine.from_file("examples/policies/basic.yaml"),
    provider_registry=registry,
)

Environment-Based Cloud Providers

Set provider credentials through environment variables or your enterprise secrets manager.

Examples:

set POLICYAWARE_AZURE_OPENAI_ENDPOINT=https://example.openai.azure.com
set POLICYAWARE_AZURE_OPENAI_API_KEY=...
set POLICYAWARE_ANTHROPIC_API_KEY=...
set POLICYAWARE_VERTEX_ACCESS_TOKEN=...

Avoid hard-coding API keys in source code or YAML policies.

Routing Principles

• Keep policy decisions separate from provider implementation.
• Prefer local or lower-cost models for low-risk tasks when quality is sufficient.
• Use stronger models for high-risk summarization, regulated RAG, or approval-bound workflows.
• Enforce region and tenant restrictions before selecting a provider.
• Capture model choice in the audit trace.