Skip to content

Conversation

@karthikscale3
Copy link
Collaborator

Description

This PR introduces support for the following:

  • Support for LiteLLM
  • Experiment tracking for DSPy experiments. Note that, for experiments to show up, pass the following additional attributes using the inject_additional_attributes. This way Langtrace knows that you are running an experiment:
  1. (Required) experiment - Experiment name. Ex: experiment 1.
  2. (Optional) description - Some useful description about the experiment.
  3. (Optional) run_id - When you want to associate traces to a specific runs, pass a unique run ID. This is useful when you are running Evaluate() as part of your experiment where the traces specific to the Evaluate() will appear as an individual entry.
  • The Eval Chart on Langtrace Client will appear when you run Evaluate(). Note: Currently the score ranges it supports are between 0 and 100. So if you have scores that do not fall within this range, it could cause some UI issues.
from langtrace_python_sdk import inject_additional_attributes

predictor = inject_additional_attributes(lambda: compiled_rag(my_question), {'experiment': 'experiment 1', 'description': 'some useful description', 'run_id': 'run_1'})

Checklist for adding new integration:

  • Defined APIS in constants folder.
  • Updated SERVICE_PROVIDERS in common.py
  • Created a folder under instrumentation with the name of the integration with atleast patch.py and instrumentation.py files.
  • Added instrumentation in all_instrumentations in langtrace.py and to the InstrumentationType in types.py files.
  • Added examples for the new integration in the examples folder.
  • Updated pyproject.toml to install new dependencies
  • Updated the README.md of langtrace-python-sdk to include the new integration in the supported integrations table.
  • Updated the README.md of Langtrace's repository to include the new integration in the supported integrations table.
  • Added new integration page to supported integrations in Langtrace Docs

@karthikscale3 karthikscale3 merged commit 854e320 into development Sep 26, 2024
@karthikscale3 karthikscale3 deleted the karthik/dspy-enhancements branch September 26, 2024 00:26
karthikscale3 added a commit that referenced this pull request Sep 26, 2024
* remove logs

* remove requirements

* Bump version

* Squash

* DSPy enhancements (#362)

* fix dspy issue

* DSPy enhancements v1

* Add support for LiteLLM

* update readme

* DSPy enhancements

* bump version
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants