Skip to content

Conversation

@chenmoneygithub
Copy link
Collaborator

We introduced litellm logging suppression in #1768, but we were configuring litellm inside the __init__ call. So if users do:

lm = dspy.LM("...")
dspy.settings.configure(suppress_debug_info=False)
lm("some query")

Then the logging won't be turned on. In this PR I am introducing a pair of util functions: dspy.clients.enable_litellm_logging() and dspy.client.disable_litellm_logging() to turn on/off the litellm logging dynamically.

@okhat
Copy link
Collaborator

okhat commented Nov 8, 2024

Thank you, @chenmoneygithub! I like this! But should it just be dspy.configure(litellm_logging=True/False)?

If you think the answer is no, feel free to merge this!

@chenmoneygithub
Copy link
Collaborator Author

chenmoneygithub commented Nov 9, 2024

@okhat Thanks! I took a second look at dspy.settings.configure(), and seems right now it's just updating the global state dict instead of running any dynamic configuration:

    def configure(self, inherit_config: bool = True, **kwargs):
        """Set configuration settings.

        Args:
            inherit_config (bool, optional): Set configurations for the given, and use existing configurations for the rest. Defaults to True.
        """
        if inherit_config:
            config = {**self.config, **kwargs}
        else:
            config = {**kwargs}

        self.__append(config)

So for now I feel it's better not change this pattern. But in the future (after we deprecate the dsp directory) we may want to have a centralized namespace like dspy.settings that is a collection of dynamic configs, e.g., dspy.settings.disable_litellm_logging() and dspy.settings.enable_verbose_logging().

@chenmoneygithub chenmoneygithub merged commit d69ec39 into stanfordnlp:main Nov 9, 2024
4 checks passed
@chenmoneygithub chenmoneygithub deleted the random-fix-4 branch December 27, 2024 22:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants