Refactor

LLMFactory Release Notes

Version 0.0.3 - Modular Provider Architecture

Major Refactoring: Provider Module Reorganization

We've restructured the LLMFactory codebase to improve maintainability, modularity, and scalability. The monolithic llm.py file (1,269 lines) has been refactored into a clean, modular provider architecture.

What's New

Modular Provider Structure

All LLM provider implementations have been moved to dedicated modules under LLMFactory/providers/:

LLMFactory/
├── llm.py (109 lines - 91% reduction!)
└── providers/
    ├── __init__.py         # Provider exports
    ├── base.py            # Abstract base classes and utilities
    ├── ollama.py          # Ollama providers
    ├── lmstudio.py        # LM Studio provider
    ├── anthropic.py       # Anthropic providers
    ├── openai.py          # OpenAI providers
    ├── gemini.py          # Google Gemini provider
    ├── embeddings.py      # Sentence Transformers
    └── llamacpp.py        # llama.cpp provider

Benefits

• ✨ Better Organization: Each provider is now in its own focused module
• 📦 Improved Maintainability: Easier to find, update, and test individual providers
• 🚀 Future-Ready: Simplified process for adding new providers
• 📖 Better Code Navigation: Clear separation of concerns

Breaking Changes

None! This release maintains 100% backward compatibility.

All existing code continues to work without modification:

# Still works exactly as before
from LLMFactory.llm import OllamaInference, LMStudioInference
from LLMFactory.llm import LLMModelFactory

# You can now also import from providers directly
from LLMFactory.providers import AnthropicInference
from LLMFactory.providers.openai import OpenAIInference

Technical Details

New Provider Modules

• base.py: Contains InferenceModel abstract base class and shared utilities like _encode_image()
• ollama.py: OllamaInference and OllamaEmbedInference classes
• lmstudio.py: LMStudioInference with remote connection support
• anthropic.py: AnthropicInference and AnthropicBedrockInference classes
• openai.py: OpenAIInference and CustomOAIInference classes
• gemini.py: GeminiInference class
• embeddings.py: SentenceTransformerInference class
• llamacpp.py: LlamacppInference class for local GGUF models

Validation

• ✅ All 93 unit tests pass
• ✅ No import conflicts with external packages (e.g., openai package)
• ✅ Full backward compatibility verified
• ✅ Type hints preserved
• ✅ Documentation intact

Migration Guide

No migration needed! Your existing code will continue to work without any changes.

However, if you want to take advantage of the new structure:

# Old style (still works)
from LLMFactory.llm import OllamaInference

# New style (more explicit)
from LLMFactory.providers.ollama import OllamaInference

# Both import the exact same class

Developer Notes

For contributors and developers extending LLMFactory:

1. Adding a New Provider: Create a new file in LLMFactory/providers/ (e.g., cohere.py)
2. Export Provider: Add your provider to providers/__init__.py
3. Register in Factory: Add your provider to LLMModelFactory._models in llm.py
4. Export from Main: Add your provider to __all__ in llm.py

Example:

# LLMFactory/providers/cohere.py
from .base import InferenceModel

class CohereInference(InferenceModel):
    def _get_provider(self) -> str:
        return "cohere"
    # ... implementation

Looking Forward

This refactoring sets the foundation for:

• Easier addition of new LLM providers
• Better testing isolation for individual providers
• Potential for lazy loading to reduce import time
• Clearer documentation and examples per provider
• Simplified maintenance and bug fixes