Architecture 3 Providers

Providers Module Architecture

Code References

• Primary implementation: coda/providers/base.py
• Registry system: coda/providers/registry.py
• LiteLLM provider: coda/providers/litellm_provider.py
• Ollama provider: coda/providers/ollama_provider.py
• OCI GenAI provider: coda/providers/oci_genai.py
• Mock provider: coda/providers/mock_provider.py
• Utilities: coda/providers/utils.py
• Tests: tests/unit/test_providers.py, tests/providers/

Overview

The Providers module abstracts different AI model providers behind a common interface, allowing Coda Code Assistant to work with multiple LLM backends. The module implements a provider registry system with factory pattern for consistent provider creation and management. The base interface is defined in coda/providers/base.py:94-231.

Module Structure

coda/providers/
├── __init__.py           # Module exports (verified)
├── base.py              # Abstract base provider and data models (verified)
├── registry.py          # Provider registration and factory (verified)
├── litellm_provider.py  # LiteLLM integration (100+ providers) (verified)
├── ollama_provider.py   # Ollama local models (verified)
├── oci_genai.py        # Oracle Cloud Infrastructure GenAI (verified)
├── mock_provider.py     # Mock provider for testing (verified)
└── utils.py            # Shared utilities (verified)

Key Components

Component 1: Base Provider Interface

Location: coda/providers/base.py:94-231

Purpose: Defines the abstract interface all providers must implement

Core Methods:

• name (property, line 101): Provider identifier
• chat() (lines 108-118): Synchronous chat completion
• chat_stream() (lines 120-130): Streaming chat completion
• achat() (lines 132-142): Async chat completion
• achat_stream() (lines 144-154): Async streaming completion
• list_models() (lines 156-163): Available model discovery

Component 2: Provider Registry

Location: coda/providers/registry.py:18-126

Purpose: Manages provider registration and instance caching

Key Features:

• Static class-level registry (lines 21-22)
• Singleton instance caching (lines 104-114)
• Config-based cache key generation (lines 75-99)

Component 3: Provider Factory

Location: coda/providers/registry.py:129-164

Purpose: Creates provider instances with merged configuration

Key Methods:

• __init__() (lines 133-134): Initialize with global config
• create() (lines 136-164): Create provider with config merging

Component 4: Data Models

Location: coda/providers/base.py:10-91

Purpose: Common data structures for provider communication

Key Classes:

• Message (lines 10-28): Chat message with role and content
• Tool (lines 31-35): Tool definition
• ToolCall (lines 38-43): Tool invocation request
• ChatCompletion (lines 51-72): Non-streaming response
• ChatCompletionChunk (lines 75-91): Streaming response chunk

Implementation Details

Design Patterns Used

Abstract Factory Pattern

Implementation: coda/providers/base.py:94-231 and coda/providers/registry.py:129-164

class BaseProvider(ABC):  # line 94
    """Abstract base class for all providers."""
    
    @abstractmethod
    async def achat(self, messages: List[Message], ...) -> ChatCompletion:  # line 132
        """Abstract method all providers must implement."""

Registry Pattern

Implementation: coda/providers/registry.py:18-126

class ProviderRegistry:  # line 18
    """Registry for managing provider classes and instances."""
    
    _providers: Dict[str, Type[BaseProvider]] = {}  # line 21
    _instances: Dict[str, BaseProvider] = {}  # line 22
    
    @classmethod
    def register(cls, name: str, provider_class: Type[BaseProvider]) -> None:  # line 25
        """Register a provider class."""
        cls._providers[name] = provider_class  # line 35

Singleton Pattern with Caching

Implementation: coda/providers/registry.py:104-114

# Generate cache key from config
config_key = f"{name}_{cls._hash_config(kwargs)}"  # line 104
if config_key in cls._instances:  # line 105
    return cls._instances[config_key]  # line 107

Provider Discovery Flow

sequenceDiagram
    participant CLI
    participant ProviderFactory
    participant ProviderRegistry
    participant Provider
    participant LLM Service
    
    CLI->>ProviderFactory: create(provider_name)
    ProviderFactory->>ProviderFactory: Merge configs
    ProviderFactory->>ProviderRegistry: get_provider(name, config)
    
    alt Provider cached
        ProviderRegistry->>CLI: Return cached instance
    else New provider
        ProviderRegistry->>Provider: Create instance
        Provider->>Provider: Initialize client
        Provider->>LLM Service: Test connection
        ProviderRegistry->>ProviderRegistry: Cache instance
        ProviderRegistry->>CLI: Return new instance
    end
    
    CLI->>Provider: list_models()
    Provider->>LLM Service: Fetch models
    LLM Service->>Provider: Model list
    Provider->>CLI: Return filtered models

Loading

Provider Implementations

LiteLLM Provider

Location: coda/providers/litellm_provider.py:15-269

Features:

• Gateway to 100+ LLM providers (line 29 docstring)
• Unified interface for OpenAI, Anthropic, Google, etc.
• Full async and streaming support (lines 162-214)
• Tool/function calling support (lines 127-135)

Ollama Provider

Location: coda/providers/ollama_provider.py:18-345

Features:

• Local model execution (line 18 docstring)
• HTTP client using httpx (lines 39-55)
• Model management commands (pull, delete) (lines 302-326)
• Custom streaming parser (lines 136-159)

OCI GenAI Provider

Location: coda/providers/oci_genai.py:21-545

Features:

• Oracle Cloud integration (line 21 docstring)
• Model discovery with caching (lines 177-205)
• Provider-specific formatting (Cohere vs Generic) (lines 333-386)
• Tool calling for Cohere models (lines 439-492)

Mock Provider

Location: coda/providers/mock_provider.py:14-358

Features:

• Testing and development (line 14 docstring)
• Context-aware responses (lines 146-194)
• Simulated streaming (lines 195-237)
• Configurable behaviors (lines 38-85)

Configuration

Config File: ~/.config/coda/config.toml Config Class: coda/configuration.py:CodaConfig

Configuration Hierarchy

1. Method parameters (highest priority)
2. Provider-specific config section
3. Factory configuration
4. Global configuration
5. Environment variables (lowest priority)

Provider-Specific Configuration

# From config.example.toml
[providers.oci_genai]
compartment_id = "ocid1.compartment.oc1.."
model = "cohere.command-r-plus"

[providers.ollama]
base_url = "http://localhost:11434"
model = "llama3"

[providers.litellm]
model = "gpt-4"
api_key = "${OPENAI_API_KEY}"

Dependencies

Internal Dependencies

• coda.configuration: Config management (used in all providers)
• coda.constants: Provider name constants (registry.py:8)

External Dependencies

• httpx>=0.24.0: HTTP client for Ollama
• litellm>=1.0.0: Multi-provider gateway
• oci>=2.88.0: Oracle Cloud SDK
• pydantic>=2.0: Data validation

Testing

Test Coverage

• Unit tests: tests/unit/test_providers.py
• Provider tests: tests/providers/test_*.py
• Integration tests: tests/integration/test_*_integration.py

Key Test Cases

1. Registry Tests (test_provider_registry.py:15-67): Provider registration and caching
2. Mock Provider (test_mock_provider_*.py): Comprehensive behavior testing
3. Streaming Tests (test_providers.py:234-289): Streaming response handling

Error Handling

Common Exceptions

# From base.py:234-241
class ProviderError(Exception):
    """Base exception for provider errors."""
    
class ModelNotFoundError(ProviderError):
    """Raised when requested model is not available."""
    
class AuthenticationError(ProviderError):
    """Raised when authentication fails."""

Provider-Specific Error Handling

# From ollama_provider.py:85-93
except httpx.HTTPStatusError as e:
    if e.response.status_code == 404:
        raise ModelNotFoundError(
            f"Model '{model}' not found. "
            f"Run 'ollama pull {model}' first."
        )

Performance Considerations

1. Instance Caching: Providers cached by config hash (registry.py:104-114)
2. Model List Caching: 24-hour cache for OCI models (oci_genai.py:43)
3. Connection Pooling: HTTP clients reused (ollama_provider.py:39)
4. Async First: All providers support async operations

Security Considerations

1. Credential Management: API keys from environment variables
2. OCI Authentication: Uses OCI config file (~/.oci/config)
3. No Credential Storage: Keys never persisted to disk
4. Secure Defaults: HTTPS preferred for all connections

Examples

Basic Usage

# From tests/unit/test_providers.py:45
from coda.providers import ProviderFactory

factory = ProviderFactory(config)
provider = factory.create("ollama", model="llama3")

response = await provider.achat([
    {"role": "user", "content": "Hello!"}
])

Streaming Usage

# From litellm_provider.py:203-214
async for chunk in provider.achat_stream(messages):
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

Integration Points

With Other Modules

1. CLI Module: Provider initialization in provider_manager.py
2. Agent Module: Providers used for AI completions
3. Session Module: Provider responses stored in message history

Extension Points

1. New Providers: Extend BaseProvider and register
2. Custom Models: Override list_models() for filtering
3. Response Processing: Override chat methods for custom logic

Known Limitations

1. Tool Calling: Only supported by some providers (see provider docs)
2. Model Names: Must match provider's expected format
3. Streaming Variability: Chunk sizes vary by provider

Verification Checklist

• All file paths verified to exist
• All line numbers checked for accuracy
• All class/function names validated
• Code snippets verified from source
• Diagrams match actual code flow
• Configuration examples tested
• Cross-references verified

References

Related Documentation

• Architecture Overview
• CLI Module Architecture
• Agent Module Architecture

Source Files

All files referenced in this document:

• coda/providers/base.py
• coda/providers/registry.py
• coda/providers/litellm_provider.py
• coda/providers/ollama_provider.py
• coda/providers/oci_genai.py
• coda/providers/mock_provider.py
• coda/providers/utils.py
• tests/unit/test_providers.py
• tests/providers/