Red Candle Provider

[cpetersen]

What this does

This PR adds support for the Red Candle provider, enabling local LLM execution using quantized GGUF models directly in Ruby without requiring external API calls.

Key Implementation Details

Red Candle is fundamentally different from other providers: While all other RubyLLM providers communicate via HTTP APIs, Red Candle runs models locally using the Candle Rust crate. This brings true local inference to Ruby, with no network
latency or API costs.

Dependency Management

Since Red Candle requires a Rust toolchain at build time, we've made it optional at two levels:

• For end users: red-candle is NOT a gemspec dependency. Users must explicitly add gem 'red-candle' to their Gemfile to use this provider.
• For contributors: We've added an optional Bundler group so developers can work on RubyLLM without installing Rust. Enable with bundle config set --local with red_candle.

Testing Strategy

We implemented a comprehensive mocking system to keep tests fast:

• Stubbed mode (default): Uses MockCandleModel to simulate responses without actual inference
• Real inference mode: Set RED_CANDLE_REAL_INFERENCE=true to run actual model inference (downloads models on first run, ~4.5 GBs)
• Not installed mode: Tests skip gracefully when Red Candle isn't available

Changes Made

• Added RubyLLM::Providers::RedCandle with full chat support including streaming
• Implemented model management with automatic GGUF file downloads from HuggingFace
• Created comprehensive test mocks in red_candle_test_helper.rb
• Added conditional loading in ruby_llm.rb and spec_helper.rb to handle optional dependency
• Updated models_to_test.rb to conditionally include Red Candle models
• Added documentation in CONTRIBUTING.md for managing the optional dependency
• Implemented proper Content object handling for structured responses

How to Test

# Test without Red Candle (default for new contributors)
bundle install
bundle exec rspec  # Red Candle tests will be skipped

# Test with Red Candle stubbed (fast)
bundle config set --local with red_candle
bundle install
bundle exec rspec  # Uses mocked responses

# Test with real inference (slow, downloads models)
bundle config set --local with red_candle
bundle install
huggingface-cli login # Make sure to accept mistral terms
RED_CANDLE_REAL_INFERENCE=true bundle exec rspec

Once red-candle is enabled turn it back off with:

bundle config unset with

And turn it BACK on with:

bundle config set --local with red_candle

Try it out

bundle exec irb

require 'ruby_llm'

chat = RubyLLM.chat(
  provider: :red_candle,
  model: 'Qwen/Qwen2.5-1.5B-Instruct-GGUF' # 'TheBloke/Mistral-7B-Instruct-v0.2-GGUF' is another option
)
response = chat.ask("What are the benefits of functional programming?")
puts response.content

Type of change

• Bug fix
• New feature
• Breaking change
• Documentation
• Performance improvement

Scope check

• I read the Contributing Guide
• This aligns with RubyLLM's focus on LLM communication
• This isn't application-specific logic that belongs in user code
• This benefits most users, not just my specific use case

Quality check

• I ran overcommit --install and all hooks pass
• I tested my changes thoroughly
  • For provider changes: Re-recorded VCR cassettes with bundle exec rake vcr:record[provider_name]
  • All tests pass: bundle exec rspec
• I updated documentation if needed
• I didn't modify auto-generated files manually (models.json, aliases.json)

API changes

• Breaking change
• New public methods/classes
• Changed method signatures
• No API changes

Related issues

Fixes crmne#394

[orangewolf]

this looks great!