Add support for cohere models

[lidiyam]

Trying to add support for cohere models (command-r and command-r-plus) leveraging their function calling ability.

Looking at the tool-use prompt template from the docs here, the tools are defined as python functions and so I had the following idea:

1. generate python code that defines the desired pydantic object based on it's .model_json_schema()
2. insert it in the tool-use prompt that defines the tool function (which will take in the pydantic object as the sole parameter)
3. let llm generate the required arguments for the tool function as usual
4. parse it
5. retry if failed to parse till max_retries is reached

This leads to less retries than asking the model to output json based on .model_json_schema() directly, but is more costly for simple use cases (non-nested pydantic objects) where the extra llm call is not really necessary.

---

	🚀 This PR description was created by Ellipsis for commit ba6c246.

Summary:

This PR adds support for Cohere models in the Instructor library, introducing a new mode and function to handle Cohere's function calling ability, along with necessary updates in various files, tests, and dependencies.

Key points:

• Added support for Cohere models in instructor library.
• Introduced a new mode, COHERE_TOOLS, in instructor/mode.py.
• Added a new function, from_cohere, in instructor/client_cohere.py to handle Cohere's function calling ability.
• Updated instructor/__init__.py to import from_cohere if Cohere is available.
• Updated instructor/function_calls.py to handle COHERE_TOOLS mode.
• Updated instructor/process_response.py to handle COHERE_TOOLS mode.
• Updated instructor/retry.py to handle COHERE_TOOLS mode.
• Updated instructor/utils.py to include COHERE in Provider enum.
• Updated pyproject.toml and requirements.txt to include Cohere as a dependency.
• Added tests for the new functionality in tests/test_new_client.py.

---

Generated with ❤️ by ellipsis.dev

[ellipsis-dev]

👍 Looks good to me!

• Reviewed the entire pull request up to ba6c246
• Looked at 457 lines of code in 11 files
• Took 2 minutes and 43 seconds to review

More info

• Skipped 1 files when reviewing.
• Skipped posting 9 additional comments because they didn't meet confidence threshold of 85%.

1. instructor/client_cohere.py:160:

• Assessed confidence : 0%
• Comment:
  The new file client_cohere.py is added to support cohere models. The function from_cohere is defined to create an instructor for cohere models. The function create_cohere_structured_output is defined to generate a tool snippet for a pydantic object, format a tool use prompt with the tool snippet, and send the tool use prompt to cohere client chat. The function prompt_model_for_python_class_def is defined to generate a prompt to generate python code that defines the provided pydantic object. The function generate_tool_snippet_pydantic is defined to generate a string representing Python code to define a tool which takes pydantic object as a parameter. The function render_chat_history is defined to render chat history as a string. The function format_tool_use_prompt is defined to format tool use prompt with code for tools being provided by tool_snippet. The code seems to be logically correct and follows the best practices.
• Reasoning:
  The new file client_cohere.py is added to support cohere models. The function from_cohere is defined to create an instructor for cohere models. The function create_cohere_structured_output is defined to generate a tool snippet for a pydantic object, format a tool use prompt with the tool snippet, and send the tool use prompt to cohere client chat. The function prompt_model_for_python_class_def is defined to generate a prompt to generate python code that defines the provided pydantic object. The function generate_tool_snippet_pydantic is defined to generate a string representing Python code to define a tool which takes pydantic object as a parameter. The function render_chat_history is defined to render chat history as a string. The function format_tool_use_prompt is defined to format tool use prompt with code for tools being provided by tool_snippet. The code seems to be logically correct and follows the best practices.

2. instructor/function_calls.py:160:

• Assessed confidence : 0%
• Comment:
  The function parse_cohere_tools is added to parse the response from cohere tools. The function parse_cohere_tools extracts the json text from the response, loads it into a dictionary, extracts the parameters from the dictionary, and validates the parameters using the pydantic model. The code seems to be logically correct and follows the best practices.
• Reasoning:
  The function parse_cohere_tools is added to parse the response from cohere tools. The function parse_cohere_tools extracts the json text from the response, loads it into a dictionary, extracts the parameters from the dictionary, and validates the parameters using the pydantic model. The code seems to be logically correct and follows the best practices.

3. instructor/mode.py:17:

• Assessed confidence : 0%
• Comment:
  The mode COHERE_TOOLS is added to the Mode enum. This mode is used to indicate that the cohere tools are being used. The code seems to be logically correct and follows the best practices.
• Reasoning:
  The mode COHERE_TOOLS is added to the Mode enum. This mode is used to indicate that the cohere tools are being used. The code seems to be logically correct and follows the best practices.

4. instructor/process_response.py:329:

• Assessed confidence : 0%
• Comment:
  The function handle_response_model is updated to handle the case when the mode is COHERE_TOOLS. In this case, the response_model is added to the kwargs. The code seems to be logically correct and follows the best practices.
• Reasoning:
  The function handle_response_model is updated to handle the case when the mode is COHERE_TOOLS. In this case, the response_model is added to the kwargs. The code seems to be logically correct and follows the best practices.

5. instructor/retry.py:48:

• Assessed confidence : 0%
• Comment:
  The function reask_messages is updated to handle the case when the mode is COHERE_TOOLS. In this case, a user message is yielded with the validation error and a request to recall the function correctly and fix the errors. The code seems to be logically correct and follows the best practices.
• Reasoning:
  The function reask_messages is updated to handle the case when the mode is COHERE_TOOLS. In this case, a user message is yielded with the validation error and a request to recall the function correctly and fix the errors. The code seems to be logically correct and follows the best practices.

6. instructor/utils.py:50:

• Assessed confidence : 0%
• Comment:
  The function extract_python_from_codeblock is added to extract python code from a code block in a string. The function finds the start and end of the python code block and returns the python code. The code seems to be logically correct and follows the best practices.
• Reasoning:
  The function extract_python_from_codeblock is added to extract python code from a code block in a string. The function finds the start and end of the python code block and returns the python code. The code seems to be logically correct and follows the best practices.

7. pyproject.toml:33:

• Assessed confidence : 0%
• Comment:
  The cohere package is added as an optional dependency. This package is required to use the cohere models. The code seems to be logically correct and follows the best practices.
• Reasoning:
  The cohere package is added as an optional dependency. This package is required to use the cohere models. The code seems to be logically correct and follows the best practices.

8. requirements.txt:10:

• Assessed confidence : 0%
• Comment:
  The cohere package is added to the requirements file. This package is required to use the cohere models. The code seems to be logically correct and follows the best practices.
• Reasoning:
  The cohere package is added to the requirements file. This package is required to use the cohere models. The code seems to be logically correct and follows the best practices.

9. tests/test_new_client.py:232:

• Assessed confidence : 0%
• Comment:
  The test test_client_cohere_response is added to test the response from the cohere client. The test test_client_cohere_response_with_nested_classes is added to test the response from the cohere client with nested classes. The tests seem to be logically correct and follow the best practices.
• Reasoning:
  The test test_client_cohere_response is added to test the response from the cohere client. The test test_client_cohere_response_with_nested_classes is added to test the response from the cohere client with nested classes. The tests seem to be logically correct and follow the best practices.

Workflow ID: wflow_pLNZL6k6ZoxnkEmk

---

Not what you expected? You can customize the content of the reviews using rules. Learn more here.

[jxnl]

oh my god this is brutal

[jxnl]

is this their system message???

[jxnl]

Do you think we're better off just asking for a json?

[lidiyam]

Ok asking for json directly removes a lot of the gnarliness 😅

[jxnl]

is there an async client we need to watch out for?

[lidiyam]

oh yes there's cohere.AsyncClient

for some reason instructor.utils.is_async is not recognizing its client.chat function as async and so patching doesn't work...will send a follow up a bit later today

[lidiyam]

Should support both now!

[jxnl]

do you mind just update the docs or adding a generic /hub example? so we can tweet this out and het some cohere attention :)

[lidiyam]

Ok added a few basic examples (like with User or Group extraction) and then a "tutorial" on doing document segmentation with a cohere client (since cmd-r+ has 128k context length it's a good use case for it).

[jxnl]

you'll have to add this in the mkdocs.yml file
or even reference it in examples/index.md