Skip to content

Conversation

@CalebCourier
Copy link
Collaborator

@CalebCourier CalebCourier commented Sep 18, 2024

Fixes: #979

This PR has two main focuses:

  1. Fix support for custom llm_api callables
    • Fix and safeguard when checking if the llm_api is one of OpenAI's static functions
    • Root cause of issue, if OPENAI_API_KEY is not in the environment nor set in an openai global, openai will raise an exception when you try to reference any of the static completion functions even if you're not calling them. We were not properly handling this exception. Now we do.
  2. Method signature enforcement for custom llm_api callables
    • In order to give users upfront feedback, we now inspect custom llm_api callables to ensure they allow for a positional argument for the prompt and accept a spread/rest variable for keyword arguments.
    • We also warn if we do not see keyword arguments for instructions or msg_history since not destructuring out can have unintended consequences when passing **kwargs to any downstream code.

We've also increased test coverage to catch any future regressions to custom llm_api support.

TODO

  • Doc on custom llm_api callables

@CalebCourier CalebCourier requested a review from a team September 18, 2024 20:34
@CalebCourier CalebCourier marked this pull request as draft September 18, 2024 20:41
@CalebCourier CalebCourier marked this pull request as ready for review September 18, 2024 20:58
@CalebCourier CalebCourier marked this pull request as draft September 18, 2024 21:07
@CalebCourier CalebCourier marked this pull request as ready for review September 18, 2024 21:16
@zsimjee zsimjee merged commit eb212ba into main Sep 19, 2024
12 checks passed
@zsimjee zsimjee deleted the custom-llms branch September 19, 2024 22:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug] Trouble calling guard without using openai or azure openai APIs

2 participants