Review and Refactor Current Implementation #44
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…AI-compatible APIs This major update integrates cloud-based LLM providers alongside existing local options, providing flexibility in deployment and usage. ## New Features ### LLM Provider Infrastructure - Created unified `LLMProvider` trait for consistent provider interface - Implemented `LLMProviderFactory` for dynamic provider instantiation - Added comprehensive provider characteristics and configuration types ### Cloud Provider Implementations - **Anthropic Claude**: Full support for Claude 3.5 (Opus, Sonnet, Haiku) - 200K token context window - Automatic retry with exponential backoff - Feature-gated with `anthropic` flag - **OpenAI**: Complete GPT-4o and GPT-4 Turbo support - 128K token context window - Organization ID support - Feature-gated with `openai-llm` flag - **OpenAI-Compatible**: Generic support for any OpenAI-compatible endpoint - Works with LM Studio, vLLM, Ollama v1 API, etc. - Optional API key authentication - Feature-gated with `openai-compatible` flag ### Updated Existing Providers - Refactored `QwenClient` to implement `LLMProvider` trait - Maintained backward compatibility with existing code - Enhanced with consistent error handling and metrics ### Interactive Setup Wizard - Created `codegraph-setup` binary for guided configuration - Interactive prompts for all provider types - Supports both embedding and LLM provider selection - Automatic configuration file generation at ~/.codegraph/config.toml ### Configuration Enhancements - Extended `LLMConfig` with cloud provider fields: - `anthropic_api_key`, `openai_api_key` - `openai_compatible_url` for custom endpoints - `max_tokens`, `timeout_secs` for fine-tuning - Updated `.codegraph.toml.example` with comprehensive examples - Added environment variable support (ANTHROPIC_API_KEY, OPENAI_API_KEY) ### Documentation - Created comprehensive `docs/CLOUD_PROVIDERS.md` guide - Provider comparison matrix - Setup instructions for each provider - Usage examples and code snippets - Troubleshooting guide - Security best practices - Updated README.md with cloud provider quickstart - Added provider-specific configuration examples ## Technical Details ### Architecture - All cloud providers follow the same `LLMProvider` trait - Async-first design with proper error propagation - Configurable retry logic with exponential backoff - Token usage tracking and metrics - Streaming support (prepared for future implementation) ### Code Quality - Feature flags for optional dependencies - Comprehensive test coverage for factory and providers - Type-safe configuration with serde - Clear error messages for missing API keys ### Dependencies Added - dialoguer: Interactive CLI prompts - dirs: Cross-platform config directory detection - Feature flags for optional cloud providers ## Migration Guide Existing configurations continue to work without changes. To use cloud providers: 1. Enable desired features during build: ```bash cargo build --features anthropic,openai-llm,openai-compatible ``` 2. Run setup wizard or manually configure in .codegraph.toml: ```toml [llm] enabled = true provider = "anthropic" # or "openai", "openai-compatible" model = "claude-3-5-sonnet-20241022" anthropic_api_key = "sk-ant-..." ``` ## Files Changed - Core implementation: 5 new provider files in codegraph-ai - Factory: New llm_factory.rs for provider instantiation - Configuration: Updated config_manager.rs with cloud fields - Setup wizard: New codegraph-setup binary - Documentation: New CLOUD_PROVIDERS.md guide - Examples: Updated .codegraph.toml.example ## Breaking Changes None. All changes are backward compatible with existing configurations. ## Future Enhancements - Streaming response support - Function calling for supported providers - Response caching layer - Cost tracking and optimization - Multi-provider fallback strategies
BREAKING CHANGE: OpenAI provider now uses Responses API (/v1/responses) instead
of Chat Completions API. This is required for reasoning models (o1, o3, o4-mini).
## Major Changes
### Responses API Migration
- **OpenAI Provider**: Completely rewritten to use `/v1/responses` endpoint
- **OpenAI-Compatible Provider**: Supports both Responses API and Chat Completions API with automatic fallback
- **Request Format**: Changed from `messages` array to `input` string + `instructions`
- **Response Format**: Changed from `choices[0].message.content` to `output_text`
### Reasoning Model Support
Added full support for OpenAI's reasoning models (o1, o3, o4-mini, GPT-5):
1. **Reasoning Effort Parameter**: Control thinking depth with "minimal", "low", "medium", "high"
- `minimal`: Fast, basic reasoning (GPT-5 only)
- `low`: Quick responses with light reasoning
- `medium`: Balanced reasoning (recommended)
- `high`: Deep reasoning for complex problems
2. **max_output_tokens Parameter**: New token limit parameter for Responses API
- Replaces `max_tokens` for reasoning models
- Falls back to `max_tokens` if not set for backward compatibility
3. **Automatic Model Detection**: OpenAI provider detects reasoning models and:
- Disables temperature/top_p (not supported by reasoning models)
- Enables reasoning_effort parameter
- Uses proper token parameter names
### Configuration Updates
**GenerationConfig** (crates/codegraph-ai/src/llm_provider.rs):
```rust
pub struct GenerationConfig {
pub temperature: f32, // Not supported by reasoning models
pub max_tokens: Option<usize>, // Legacy parameter
pub max_output_tokens: Option<usize>, // NEW: For Responses API
pub reasoning_effort: Option<String>, // NEW: For reasoning models
pub top_p: Option<f32>, // Not supported by reasoning models
// ...
}
```
**LLMConfig** (crates/codegraph-core/src/config_manager.rs):
```rust
pub struct LLMConfig {
pub max_tokens: usize, // Legacy
pub max_output_tokens: Option<usize>, // NEW
pub reasoning_effort: Option<String>, // NEW
// ...
}
```
### Provider Implementations
**OpenAI Provider** (crates/codegraph-ai/src/openai_llm_provider.rs):
- Uses `/v1/responses` endpoint exclusively
- Automatic reasoning model detection
- Proper parameter handling based on model type
- Request: `{ model, input, instructions, max_output_tokens, reasoning_effort }`
- Response: `{ output_text, usage: { prompt_tokens, output_tokens, reasoning_tokens } }`
**OpenAI-Compatible Provider** (crates/codegraph-ai/src/openai_compatible_provider.rs):
- Defaults to Responses API (`use_responses_api: true`)
- Falls back to Chat Completions API for compatibility
- Supports both `max_output_tokens` and `max_completion_tokens`
- Works with LM Studio, Ollama v1 endpoint, and custom APIs
### Documentation Updates
**docs/CLOUD_PROVIDERS.md**:
- Added "Responses API & Reasoning Models" section
- Detailed explanation of API format differences
- Configuration examples for reasoning models
- Reasoning effort level descriptions
- Migration guide from Chat Completions API
**.codegraph.toml.example**:
- Added `max_output_tokens` parameter with documentation
- Added `reasoning_effort` parameter with options
- Clarified which parameters apply to reasoning vs standard models
### Backward Compatibility
- OpenAI-compatible provider can fall back to Chat Completions API
- `max_output_tokens` falls back to `max_tokens` if not set
- Configuration with only `max_tokens` continues to work
- Standard models (gpt-4o, gpt-4-turbo) work as before
### Testing
Added tests for:
- Reasoning model detection (o1, o3, o4, gpt-5)
- Standard model detection (gpt-4o, gpt-4-turbo)
- OpenAI-compatible provider configuration
- Both API format support
## Migration Guide
### For OpenAI Users
**Before (Chat Completions API)**:
```toml
[llm]
provider = "openai"
model = "gpt-4o"
max_tokens = 4096
```
**After (Responses API)** - Still works, but consider:
```toml
[llm]
provider = "openai"
model = "gpt-4o"
max_output_tokens = 4096 # Preferred for Responses API
```
**For Reasoning Models**:
```toml
[llm]
provider = "openai"
model = "o3-mini"
max_output_tokens = 25000
reasoning_effort = "medium" # NEW: Control reasoning depth
# Note: temperature/top_p ignored for reasoning models
```
### For OpenAI-Compatible Users
No changes required - the provider automatically uses Responses API if available and falls back to Chat Completions API otherwise.
To force Chat Completions API (e.g., for older systems):
```rust
let config = OpenAICompatibleConfig {
use_responses_api: false, // Force legacy API
...
};
```
## Why This Change?
1. **Future-Proof**: Responses API is OpenAI's modern standard
2. **Reasoning Models**: Required for o1, o3, o4-mini support
3. **Better Features**: More granular control over model behavior
4. **Token Tracking**: Separate tracking of reasoning tokens
5. **Performance**: Optimized for latest models
## Files Modified
- `crates/codegraph-ai/src/llm_provider.rs`: Added reasoning parameters to GenerationConfig
- `crates/codegraph-ai/src/openai_llm_provider.rs`: Complete rewrite for Responses API
- `crates/codegraph-ai/src/openai_compatible_provider.rs`: Dual API support
- `crates/codegraph-core/src/config_manager.rs`: Added reasoning config fields
- `.codegraph.toml.example`: Documented new parameters
- `docs/CLOUD_PROVIDERS.md`: Comprehensive Responses API documentation
## References
- OpenAI Responses API: https://platform.openai.com/docs/api-reference/responses
- Reasoning Models: https://platform.openai.com/docs/guides/reasoning
- Azure OpenAI Reasoning: https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/reasoning
Rewrote README from scratch to make installation and setup crystal clear for users. The old README was confusing about which features to enable and what was required for each provider type. ## Major Improvements ### 1. "Choose Your Setup" Decision Tree Added a clear decision section at the top with 4 options: - **Option 1: Local Setup** (ONNX + Ollama) - Free, private - **Option 2: LM Studio** - Best Mac performance - **Option 3: Cloud Providers** - Best quality - **Option 4: Hybrid** - Mix and match Each option clearly states: - What it's best for - Pros and cons - Direct link to setup instructions ### 2. Step-by-Step Provider-Specific Instructions Complete setup guides for each provider type: **Local Setup (ONNX + Ollama):** - Install Ollama - Pull models - Build with correct features - Configure - Run **LM Studio Setup:** - Download LM Studio - Download models in app - Start server - Build with correct features - Configure - Run **Cloud Setup (Anthropic & OpenAI):** - Get API keys - Build with cloud features - Use wizard or manual config - Run - Includes reasoning model configuration **Hybrid Setup:** - Mix local and cloud - Example configs - Build command ### 3. Clear Feature Flags Table Added comprehensive table showing: - Feature flag name - What it enables - When to use it Plus common build command examples: ```bash # Local only cargo build --release --features "onnx,ollama,faiss" # LM Studio cargo build --release --features "openai-compatible,faiss" # Cloud cargo build --release --features "anthropic,openai-llm,openai,faiss" ``` ### 4. Improved Structure **Before:** Confusing mix of information, hard to find what you need **After:** Clear sections with table of contents - Choose Your Setup (decision tree) - Installation (step-by-step by provider) - Configuration (with examples) - Usage (basic commands) - Feature Flags Reference (table) - Performance (metrics) - Troubleshooting (common issues) - Advanced Features (moved to end) ### 5. Better Configuration Examples - Removed confusing `.env` approach - Use `~/.codegraph/config.toml` consistently - Full working examples for each provider - Reasoning model configuration included ### 6. Troubleshooting Section Added common issues and solutions: - Build issues (missing FAISS, feature flags) - Runtime issues (API keys, models, connections) - Where to get help ### 7. Improved Readability - Used emojis for visual scanning - Clear headers with consistent formatting - Code blocks with proper syntax highlighting - Checkmarks (✅) to mark completion - Pros/cons for each option - "Best for" callouts ## What Was Removed - Confusing `.env` variable approach - Scattered information - Unclear prerequisites - Mixed local/cloud instructions - Hard-to-find feature flag info ## What Was Added - Decision tree at the top - Complete step-by-step guides - Feature flags table - Troubleshooting section - Clear pros/cons for each option - Common build commands - Reasoning model examples ## User Experience Improvements **Before:** - User reads entire README, still confused - Unclear which features to enable - Don't know if they need cloud or local - Can't find complete setup for their use case **After:** - User picks their setup in 30 seconds - Jumps to relevant section - Follows step-by-step instructions - Has working system ## File Structure ``` README.md (completely rewritten) ├── Overview ├── Table of Contents ├── Choose Your Setup (NEW) │ ├── Option 1: Local │ ├── Option 2: LM Studio │ ├── Option 3: Cloud │ └── Option 4: Hybrid ├── Installation (REORGANIZED) │ ├── Prerequisites │ ├── Local Setup │ ├── LM Studio Setup │ ├── Cloud Setup │ └── Hybrid Setup ├── Configuration ├── Usage ├── Feature Flags Reference (NEW TABLE) ├── Performance ├── Troubleshooting (NEW) ├── Advanced Features └── Learn More ``` This restructuring makes CodeGraph accessible to users of all levels while maintaining depth for advanced users.
Complete implementation of 22+ versioning API handlers by connecting them to the transactional graph infrastructure. Changes: - Add TransactionalGraph to AppState with initialization - Wire up transaction management handlers (begin, commit, rollback, stats) - Wire up version management handlers (create, get, list, tag, compare) - Wire up branch management handlers (create, list, get, delete, merge, resolve) - Wire up snapshot management handlers (create, get) - Wire up recovery handlers (stats, integrity check, backup, restore) All handlers now use the stub implementations from graph_stub.rs instead of returning placeholder data. This provides a functional API layer ready for integration with actual persistent storage backends. Related to: codebase audit findings - 22+ incomplete API handlers
…actionalGraph Complete critical missing functionality in the transactional graph layer: 1. Content-Addressed Storage in add_node: - Serialize nodes to JSON - Store content using VersionedRocksDbStorage.store_content() - Use content hash in write set operations - Enables proper persistence of node data 2. Complete get_node Implementation: - Check write set first for uncommitted changes - Handle deleted nodes (return None) - Retrieve modified nodes from content store by hash - Fall back to reading from snapshot for unchanged nodes - Properly implements MVCC read semantics 3. Fix update_node to Store Content: - Store both old and new versions in content store - Generate content hashes via store_content() - Maintains proper before/after images for rollback 4. Implement ReadOnlyTransactionalGraph::get_node: - Read directly from specific snapshot - Used for version checkout and historical queries - No write set to check (read-only) These changes enable the versioning API handlers to function with actual persistent storage instead of returning placeholder data. Related to: codebase audit - complete transactional graph implementation
Implement find_nodes_by_name with full transactional isolation: 1. TransactionalGraph::find_nodes_by_name: - Retrieves snapshot to get baseline node set - Filters out deleted nodes from write set - Returns modified nodes from write set (uncommitted changes) - Falls back to snapshot for unchanged nodes - Includes newly inserted nodes not yet in snapshot - Provides consistent view across transaction lifetime 2. ReadOnlyTransactionalGraph::find_nodes_by_name: - Reads directly from specific snapshot - No write set considerations (read-only) - Used for historical queries and version checkout This enables proper MVCC (Multi-Version Concurrency Control) behavior: - Transactions see their own uncommitted changes - Transactions are isolated from other concurrent transactions - Reads are consistent within transaction scope - No locking required for reads Related to: codebase audit - complete transactional graph implementation
Create a comprehensive CLI for CodeGraph that can be integrated into
TypeScript applications without requiring an HTTP server.
New Components:
1. Rust CLI Binary (crates/codegraph-cli):
- Comprehensive command-line interface
- Commands: transaction, version, branch, search, status
- Multiple output formats: JSON, pretty, table
- Full integration with TransactionalGraph
- Colorized human-readable output with 'colored' crate
- Built with clap for robust argument parsing
2. TypeScript Wrapper (sdk/codegraph-cli-wrapper.ts):
- Type-safe wrapper for spawning Rust CLI
- Promise-based async API
- Automatic JSON parsing
- Timeout handling and error recovery
- Full TypeScript types for all operations
3. Integration Example (sdk/examples/cli-usage.ts):
- Complete workflow demonstration
- Transaction management example
- Version and branch operations
- Error handling patterns
4. Comprehensive Documentation:
- CLI README with usage examples
- TypeScript integration guide
- API reference
- Troubleshooting section
- Performance tips
Key Features:
Transaction Management:
- Begin/commit/rollback with isolation levels
- Transaction statistics
- Serializable isolation support
Version Management:
- Create versions with metadata
- List, get, tag versions
- Compare versions (diffs)
- Parent-child relationships
Branch Management:
- Create, list, get, delete branches
- Merge branches with conflict detection
- Branch metadata tracking
Architecture Benefits:
✅ No HTTP server needed - Direct process spawning
✅ Type-safe TypeScript integration
✅ Standalone tool - works independently
✅ Simple IPC - JSON over stdin/stdout
✅ Fast - Native Rust performance
✅ Unix philosophy - Composable CLI tools
Integration Pattern:
┌────────────────────────────┐
│ TypeScript CLI │
│ const cg = CodeGraph() │
│ await cg.createVersion() │
└──────────┬─────────────────┘
│ spawn()
┌──────────▼─────────────────┐
│ Rust CLI Binary │
│ codegraph version create │
└────────────────────────────┘
Usage:
// TypeScript
import CodeGraph from './sdk/codegraph-cli-wrapper';
const cg = new CodeGraph();
await cg.createVersion({
name: 'v1.0.0',
description: 'Release',
author: 'user@example.com'
});
// Or directly:
$ codegraph version create --name v1.0 --description "Release" --author user
Build:
cargo build --release --bin codegraph
Related to: User request for TypeScript CLI integration
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull Request
Description
Brief description of the changes and the motivation behind them.
Type of Change
Testing
Performance Impact
Security Considerations
Documentation
Checklist
Related Issues
Fixes #(issue number)
Related to #(issue number)
Screenshots (if applicable)
Add screenshots to help explain the changes.
Additional Notes
Any additional information that would be helpful for reviewers.