Agent with custom context

[dmlond]

I can create a custom context that connects to a custom openai compatible endpoint.

ctx = RubyLLM.context do |config|
  config.openai_api_key = 'mykey'
  config.openai_api_base = 'https://some_endpoint'
end

chat = ctx.chat(model: 'some_model, provider: :openai, assume_model_exists: true)
resp = chat.ask 'write a poem about ruby'

I tried to create an Agent configured to connect to a specific openai compatible custom endpoint.

class MyCustomAgent < RubyLLM::Agent
  chat_model Chat
  model model: 'some  model', provider: :openai, assume_model_exists: true
  context do |config|
    config.openai_api_key = 'mykey'
    config.openai_api_base = 'https://some_endpoint'
  end
end

chat = MyCustomAgent.create
chat.ask 'write a poem about ruby'
in '<main>': OpenAI provider is not configured. Add this to your initialization: (RubyLLM::ConfigurationError)

RubyLLM.configure do |config|
  config.openai_api_key = ENV['OPENAI_API_KEY']
end

What am I doing wrong?

[crmne]

You're right to expect that to work. Agent.context currently expects a context object and silently ignores the block, which leaves it using the global configuration. The docs don't make that clear enough.

For now, you can define the context separately and pass it to the agent:

# config/initializers/custom_ai_context.rb
CustomAIContext = RubyLLM.context do |config|
  config.openai_api_key = ENV.fetch('CUSTOM_AI_API_KEY')
  config.openai_api_base = ENV.fetch('CUSTOM_AI_API_BASE')
end

# Inside MyCustomAgent
context CustomAIContext

I've implemented the block form locally and updated the docs with both approaches. It's not released yet, but your original context do |config| ... end syntax is what the fix supports. Thanks for flagging this!

[dmlond]

Thanks for the answer. do you want me to create a proper issue :)

[crmne]

I have already fixed that in main!