crashes with gemini via instructor

[academo]

I tried running the basic chat example with gemini https://github.com/BrainBlend-AI/atomic-agents/blob/main/atomic-examples/quickstart/quickstart/1_basic_chatbot.py

all the same except the client is defined as this:

import google.generativeai as genai

# ...
client = instructor.from_gemini(
    client=genai.GenerativeModel(
        model_name="models/gemini-1.5-flash-latest",
    ),
    mode=instructor.Mode.GEMINI_JSON,
)

agent = BaseAgent(
    config=BaseAgentConfig(
        client=client,
        model="models/gemini-1.5-flash-latest",
        memory=memory,
    )
)

crash:

Traceback (most recent call last):
  File "/home/academo/repos/llm-auto-update-docs/main.py", line 68, in <module>
    response = agent.run(input_schema)
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/academo/repos/llm-auto-update-docs/.venv/lib/python3.12/site-packages/atomic_agents/agents/base_agent.py", line 189, in run
    response = self.get_response(response_model=self.output_schema)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/academo/repos/llm-auto-update-docs/.venv/lib/python3.12/site-packages/atomic_agents/agents/base_agent.py", line 165, in get_response
    response = self.client.chat.completions.create(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/academo/repos/llm-auto-update-docs/.venv/lib/python3.12/site-packages/instructor/client.py", line 176, in create
    return self.create_fn(
           ^^^^^^^^^^^^^^^
  File "/home/academo/repos/llm-auto-update-docs/.venv/lib/python3.12/site-packages/instructor/patch.py", line 187, in new_create_sync
    response_model, new_kwargs = handle_response_model(
                                 ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/academo/repos/llm-auto-update-docs/.venv/lib/python3.12/site-packages/instructor/process_response.py", line 750, in handle_response_model
    response_model, new_kwargs = mode_handlers[mode](response_model, new_kwargs)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/academo/repos/llm-auto-update-docs/.venv/lib/python3.12/site-packages/instructor/process_response.py", line 445, in handle_gemini_json
    "model" not in new_kwargs
AssertionError: Gemini `model` must be set while patching the client, not passed as a parameter to the create method

I suspect the implementation is too biased towards openai.

i also tried removing the model param from the BaseConfigAgent without success

[KennyVaneetvelde]

Yeah, the gemini client expects the model parameter at a different time.

Last time we had this issue we decided the cleanest way would probably, at least for now, be to use gemini using the openai-compatible API like in the example here: https://github.com/BrainBlend-AI/atomic-agents/blob/main/atomic-examples/quickstart/quickstart/4_basic_chatbot_different_providers.py

It should work flawlessly!

[academo]

@KennyVaneetvelde thanks for the quick reply! I didn't see this file since I was following the readme and documentation. Perhaps you should consider adding this note in the quick start sections for people that are not using openai run into this issue.

[KennyVaneetvelde]

Good idea!

Yeah the current documentation is quite new - already a big improvement over the old way (which was just API docs and all examples were in the code only)

But I'll take it into account as I am still planning on improving the documentation with even more practical examples, thank you!