Skip to content

v0.1.0 - Initial Release

Choose a tag to compare

View all tags
@timofriedlberlin timofriedlberlin released this 13 Feb 00:42
· 319 commits to main since this release
v0.1.0
2da02ee

LLM Provider Abstraction Library v0.1.0

A production-ready Go library providing a unified interface for interacting with multiple LLM providers through consistent streaming APIs.

🚀 Features

Unified Provider Interface

  • Single API for multiple LLM providers
  • Channel-based streaming for backpressure control
  • Context cancellation support for all streams
  • Registry pattern for automatic provider discovery

Multi-Provider Support

  • Claude Code (anthropic:claude-code) - Local CLI wrapper via cc-sdk-go
  • Anthropic API (anthropic) - Direct API access with OAuth token refresh
  • Ollama (ollama) - Local models with 11 curated, tested defaults
  • OpenRouter (openrouter) - 229 tool-enabled models with full metadata

Tool Calling

  • Consistent tool/function calling across all providers
  • Structured tool definitions with JSON Schema parameters
  • Multi-turn conversations with tool results

Production Ready

  • ✅ Race-free concurrent code (verified with -race)
  • ✅ Proper error handling (no panics, wrapped errors)
  • ✅ Context propagation and cancellation
  • ✅ Comprehensive integration tests
  • ✅ Full godoc documentation

📦 Installation

go get github.com/codewandler/llm@v0.1.0

🎯 Quick Start

package main

import (
    "context"
    "fmt"
    "github.com/codewandler/llm"
    "github.com/codewandler/llm/provider"
)

func main() {
    ctx := context.Background()
    
    // Use provider/model format with default registry
    events, err := provider.CreateStream(ctx, llm.StreamOptions{
        Model: "ollama/glm-4.7-flash",
        Messages: []llm.Message{
            {Role: llm.RoleUser, Content: "What is Go?"},
        },
    })
    if err != nil {
        panic(err)
    }
    
    // Process streaming response
    for event := range events {
        switch event.Type {
        case llm.StreamEventDelta:
            fmt.Print(event.Delta)
        case llm.StreamEventDone:
            fmt.Println("\nDone!")
        case llm.StreamEventError:
            fmt.Printf("Error: %v\n", event.Error)
        }
    }
}

🏗️ Architecture

llm/                          # Core domain types
├── api.go                   # Message, Role, Model, ToolCall
├── provider.go              # Provider interface, StreamEvent
├── registry.go              # Model resolution & routing
├── tool.go                  # ToolDefinition
│
├── provider/                # Provider implementations
│   ├── register.go          # Default registry with env config
│   ├── anthropic/           # Claude Code + Direct API
│   ├── ollama/              # Local models (11 curated)
│   ├── openrouter/          # 229 tool-enabled models
│   └── fake/                # Test provider
│
└── cmd/llm/                 # CLI demo

🔧 Provider Configuration

Environment Variables

# OpenRouter (required for OpenRouter provider)
export OPENROUTER_API_KEY="your-api-key"

# Ollama (optional, defaults to http://localhost:11434)
export OLLAMA_BASE_URL="http://localhost:11434"

# Anthropic OAuth (optional, for direct API access)
export ANTHROPIC_CLIENT_ID="your-client-id"
export ANTHROPIC_CLIENT_SECRET="your-client-secret"
export ANTHROPIC_REFRESH_TOKEN="your-refresh-token"

Model Reference Format

anthropic:claude-code/sonnet              # Claude Code CLI
anthropic/claude-3-5-sonnet-20241022      # Direct Anthropic API
ollama/glm-4.7-flash                      # Ollama (default curated model)
ollama/llama3.2:1b                        # Ollama small model
openrouter/anthropic/claude-sonnet-4.5    # OpenRouter proxy
openrouter/google/gemini-2.0-flash-001    # OpenRouter proxy

🧪 Testing

Comprehensive test suite with:

  • Unit tests for all core functionality
  • Integration tests for all providers
  • Ollama compatibility matrix (11 models tested)
  • Race detector verified
  • Context cancellation tests
# Run all tests
go test ./...

# Run with race detector
go test -race ./...

# Run integration tests
go test -v ./... -run TestProviders

📊 Ollama Curated Models

11 models tested for full compatibility (streaming, tools, conversations):

Model Size Use Case
glm-4.7-flash 4.7B Default - Fast, efficient, great tool calling
ministral-3:8b 8B Code generation, structured output
rnj-1 Small Lightweight assistant
functiongemma - Function calling specialist
devstral-small-2 Small Development tasks
nemotron-3-nano:30b 30B High quality responses
llama3.2:1b 1B Ultra-fast inference
qwen3:1.7b 1.7B Multilingual support
qwen3:0.6b 0.6B Edge deployment
granite3.1-moe:1b 1B Code-focused MoE
qwen2.5:0.5b 0.5B Minimal footprint

🛠️ Key Components

Provider Interface

type Provider interface {
    Name() string
    Models() []Model
    CreateStream(ctx context.Context, opts StreamOptions) (<-chan StreamEvent, error)
}

Stream Events

const (
    StreamEventDelta     // Text delta from model
    StreamEventReasoning // Chain-of-thought reasoning
    StreamEventToolCall  // Tool/function call request
    StreamEventDone      // Stream complete (includes usage)
    StreamEventError     // Error occurred
)

Registry Pattern

// Create custom registry
reg := llm.NewRegistry()
reg.Register(ollama.New("http://localhost:11434"))
reg.Register(openrouter.New(apiKey))

// Use provider/model format
events, err := reg.CreateStream(ctx, llm.StreamOptions{
    Model: "openrouter/anthropic/claude-sonnet-4.5",
    Messages: []llm.Message{
        {Role: llm.RoleUser, Content: "Hello!"},
    },
})

🔄 Tool Calling Example

tools := []llm.ToolDefinition{
    {
        Name:        "get_weather",
        Description: "Get current weather",
        Parameters: map[string]any{
            "type": "object",
            "properties": map[string]any{
                "location": map[string]any{
                    "type": "string",
                    "description": "City name",
                },
            },
            "required": []string{"location"},
        },
    },
}

events, _ := provider.CreateStream(ctx, llm.StreamOptions{
    Model:    "ollama/glm-4.7-flash",
    Messages: []llm.Message{
        {Role: llm.RoleUser, Content: "What's the weather in Paris?"},
    },
    Tools: tools,
})

for event := range events {
    if event.Type == llm.StreamEventToolCall {
        fmt.Printf("Tool: %s\nArgs: %+v\n", 
            event.ToolCall.Name, 
            event.ToolCall.Arguments)
    }
}

📝 What's Changed

Initial Release

  • Core provider abstraction with streaming interface
  • 4 provider implementations (Claude Code, Anthropic API, Ollama, OpenRouter)
  • Registry pattern with provider/model resolution
  • Tool calling support across all providers
  • Context cancellation in all stream parsers
  • Comprehensive test suite with race detector verification
  • Full documentation and examples

Commits

  • c1eaf14 - Initial commit: LLM provider abstraction library
  • 014dfc7 - fix: replace panics with errors and add context cancellation to streams
  • 19e7eb5 - docs: add comprehensive README with usage examples
  • 2da02ee - refactor: rename SendMessage to CreateStream and SendOptions to StreamOptions

🐛 Known Issues

None at this time. Please report issues at https://github.com/codewandler/llm/issues

📚 Documentation

🙏 Credits

📄 License

[Add license information]

Full Changelog: https://github.com/codewandler/llm/commits/v0.1.0

Assets 2
Loading