diff --git a/README.md b/README.md index dcec366e..ee4044e5 100644 --- a/README.md +++ b/README.md @@ -1,53 +1,113 @@ -# [Langtrace](https://www.langtrace.ai) +
+

Langtrace Python SDK

+

Open Source & Open Telemetry(OTEL) Observability for LLM Applications

-## Open Source & Open Telemetry(OTEL) Observability for LLM applications - -![Static Badge](https://img.shields.io/badge/License-Apache--2.0-blue) ![Static Badge](https://img.shields.io/badge/npm_@langtrase/typescript--sdk-1.2.9-green) ![Static Badge](https://img.shields.io/badge/pip_langtrace--python--sdk-1.2.8-green) ![Static Badge](https://img.shields.io/badge/Development_status-Active-green) + ![Static Badge](https://img.shields.io/badge/License-Apache--2.0-blue) + ![Static Badge](https://img.shields.io/badge/pip_langtrace--python--sdk-1.2.8-green) + ![Static Badge](https://img.shields.io/badge/Development_status-Active-green) + [![Downloads](https://static.pepy.tech/badge/langtrace-python-sdk)](https://static.pepy.tech/badge/langtrace-python-sdk) + [![Deploy](https://railway.app/button.svg)](https://railway.app/template/8dNq1c?referralCode=MA2S9H) +
--- -Langtrace is an open source observability software which lets you capture, debug and analyze traces and metrics from all your applications that leverages LLM APIs, Vector Databases and LLM based Frameworks. - -## Open Telemetry Support +## 📚 Table of Contents +- [✨ Features](#-features) +- [🚀 Quick Start](#-quick-start) +- [🔗 Integrations](#-supported-integrations) +- [🌐 Getting Started](#-getting-started) +- [⚙️ Configuration](#-configuration) +- [🔧 Advanced Features](#-advanced-features) +- [📐 Examples](#-examples) +- [🏠 Self Hosting](#-langtrace-self-hosted) +- [🤝 Contributing](#-contributions) +- [🔒 Security](#-security) +- [❓ FAQ](#-frequently-asked-questions) +- [📜 License](#-license) -The traces generated by Langtrace adhere to [Open Telemetry Standards(OTEL)](https://opentelemetry.io/docs/concepts/signals/traces/). We are developing [semantic conventions](https://opentelemetry.io/docs/concepts/semantic-conventions/) for the traces generated by this project. You can checkout the current definitions in [this repository](https://github.com/Scale3-Labs/langtrace-trace-attributes/tree/main/schemas). Note: This is an ongoing development and we encourage you to get involved and welcome your feedback. +Langtrace is an open source observability software which lets you capture, debug and analyze traces and metrics from all your applications that leverages LLM APIs, Vector Databases and LLM based Frameworks. ---- +## ✨ Features -## Langtrace Cloud ☁️ +- 📊 **Open Telemetry Support**: Built on OTEL standards for comprehensive tracing +- 🔄 **Real-time Monitoring**: Track LLM API calls, vector operations, and framework usage +- 🎯 **Performance Insights**: Analyze latency, costs, and usage patterns +- 🔍 **Debug Tools**: Trace and debug your LLM application workflows +- 📈 **Analytics**: Get detailed metrics and visualizations +- 🛠️ **Framework Support**: Extensive integration with popular LLM frameworks +- 🔌 **Vector DB Integration**: Support for major vector databases +- 🎨 **Flexible Configuration**: Customizable tracing and monitoring options -To use the managed SaaS version of Langtrace, follow the steps below: +## 🚀 Quick Start -1. Sign up by going to [this link](https://langtrace.ai). -2. Create a new Project after signing up. Projects are containers for storing traces and metrics generated by your application. If you have only one application, creating 1 project will do. -3. Generate an API key by going inside the project. -4. In your application, install the Langtrace SDK and initialize it with the API key you generated in the step 3. -5. The code for installing and setting up the SDK is shown below - -## Getting Started - -Get started by adding simply three lines to your code! - -```python +```bash pip install langtrace-python-sdk ``` ```python -from langtrace_python_sdk import langtrace # Must precede any llm module imports -langtrace.init(api_key=) +from langtrace_python_sdk import langtrace +langtrace.init(api_key='') # Get your API key at langtrace.ai ``` -OR +## 🔗 Supported Integrations -```python -from langtrace_python_sdk import langtrace # Must precede any llm module imports -langtrace.init() # LANGTRACE_API_KEY as an ENVIRONMENT variable -``` +Langtrace automatically captures traces from the following vendors: -## FastAPI Quick Start +### LLM Providers +| Provider | TypeScript SDK | Python SDK | +|----------|:-------------:|:----------:| +| OpenAI | ✅ | ✅ | +| Anthropic | ✅ | ✅ | +| Azure OpenAI | ✅ | ✅ | +| Cohere | ✅ | ✅ | +| Groq | ✅ | ✅ | +| Perplexity | ✅ | ✅ | +| Gemini | ❌ | ✅ | +| Mistral | ❌ | ✅ | +| AWS Bedrock | ✅ | ✅ | +| Ollama | ❌ | ✅ | +| Cerebras | ❌ | ✅ | + +### Frameworks +| Framework | TypeScript SDK | Python SDK | +|-----------|:-------------:|:----------:| +| Langchain | ❌ | ✅ | +| LlamaIndex | ✅ | ✅ | +| Langgraph | ❌ | ✅ | +| LiteLLM | ❌ | ✅ | +| DSPy | ❌ | ✅ | +| CrewAI | ❌ | ✅ | +| VertexAI | ✅ | ✅ | +| EmbedChain | ❌ | ✅ | +| Autogen | ❌ | ✅ | +| HiveAgent | ❌ | ✅ | +| Inspect AI | ❌ | ✅ | + +### Vector Databases +| Database | TypeScript SDK | Python SDK | +|----------|:-------------:|:----------:| +| Pinecone | ✅ | ✅ | +| ChromaDB | ✅ | ✅ | +| QDrant | ✅ | ✅ | +| Weaviate | ✅ | ✅ | +| PGVector | ✅ | ✅ (SQLAlchemy) | +| MongoDB | ❌ | ✅ | +| Milvus | ❌ | ✅ | + +## 🌐 Getting Started + +### Langtrace Cloud ☁️ + + +1. Sign up by going to [this link](https://langtrace.ai). +2. Create a new Project after signing up. Projects are containers for storing traces and metrics generated by your application. If you have only one application, creating 1 project will do. +3. Generate an API key by going inside the project. +4. In your application, install the Langtrace SDK and initialize it with the API key you generated in the step 3. +5. The code for installing and setting up the SDK is shown below -Initialize FastAPI project and add this inside the `main.py` file +### Framework Quick Starts +#### FastAPI ```python from fastapi import FastAPI from langtrace_python_sdk import langtrace @@ -61,259 +121,321 @@ client = OpenAI() def root(): client.chat.completions.create( model="gpt-4", - messages=[{"role": "user", "content": "Say this is a test three times"}], + messages=[{"role": "user", "content": "Say this is a test"}], stream=False, ) return {"Hello": "World"} ``` -## Django Quick Start - -Initialize django project and add this inside the `__init.py__` file - +#### Django ```python +# settings.py from langtrace_python_sdk import langtrace -from openai import OpenAI +langtrace.init() +# views.py +from django.http import JsonResponse +from openai import OpenAI -langtrace.init() client = OpenAI() -client.chat.completions.create( - model="gpt-4", - messages=[{"role": "user", "content": "Say this is a test three times"}], - stream=False, -) - +def chat_view(request): + response = client.chat.completions.create( + model="gpt-4", + messages=[{"role": "user", "content": request.GET.get('message', '')}] + ) + return JsonResponse({"response": response.choices[0].message.content}) ``` -## Flask Quick Start - -Initialize flask project and this inside `app.py` file - +#### Flask ```python from flask import Flask from langtrace_python_sdk import langtrace from openai import OpenAI +app = Flask(__name__) langtrace.init() client = OpenAI() -app = Flask(__name__) - -@app.route("/") -def main(): - client.chat.completions.create( +@app.route('/') +def chat(): + response = client.chat.completions.create( model="gpt-4", - messages=[{"role": "user", "content": "Say this is a test three times"}], - stream=False, + messages=[{"role": "user", "content": "Hello!"}] ) - return "Hello, World!" + return {"response": response.choices[0].message.content} ``` -## Langtrace Self Hosted +#### LangChain +```python +from langtrace_python_sdk import langtrace +from langchain.chat_models import ChatOpenAI +from langchain.prompts import ChatPromptTemplate -Get started by adding simply two lines to your code and see traces being logged to the console! +langtrace.init() -```python -pip install langtrace-python-sdk +# LangChain operations are automatically traced +chat = ChatOpenAI() +prompt = ChatPromptTemplate.from_messages([ + ("system", "You are a helpful assistant."), + ("user", "{input}") +]) +chain = prompt | chat +response = chain.invoke({"input": "Hello!"}) ``` +#### LlamaIndex ```python -from langtrace_python_sdk import langtrace # Must precede any llm module imports -langtrace.init(write_spans_to_console=True) -``` +from langtrace_python_sdk import langtrace +from llama_index import VectorStoreIndex, SimpleDirectoryReader -## Langtrace self hosted custom exporter +langtrace.init() -Get started by adding simply three lines to your code and see traces being exported to your remote location! +# Document loading and indexing are automatically traced +documents = SimpleDirectoryReader('data').load_data() +index = VectorStoreIndex.from_documents(documents) -```python -pip install langtrace-python-sdk +# Queries are traced with metadata +query_engine = index.as_query_engine() +response = query_engine.query("What's in the documents?") ``` +#### DSPy ```python -from langtrace_python_sdk import langtrace # Must precede any llm module imports -langtrace.init(custom_remote_exporter=, batch=) -``` +from langtrace_python_sdk import langtrace +import dspy +from dspy.teleprompt import BootstrapFewShot -### Configure Langtrace +langtrace.init() -| Parameter | Type | Default Value | Description | -| -------------------------- | ----------------------------------- | ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | -| `api_key` | `str` | `LANGTRACE_API_KEY` or `None` | The API key for authentication. | -| `batch` | `bool` | `True` | Whether to batch spans before sending them. | -| `write_spans_to_console` | `bool` | `False` | Whether to write spans to the console. | -| `custom_remote_exporter` | `Optional[Exporter]` | `None` | Custom remote exporter. If `None`, a default `LangTraceExporter` will be used. | -| `api_host` | `Optional[str]` | `https://langtrace.ai/` | The API host for the remote exporter. | -| `disable_instrumentations` | `Optional[DisableInstrumentations]` | `None` | You can pass an object to disable instrumentation for specific vendors ex: `{'only': ['openai']}` or `{'all_except': ['openai']}` | +# DSPy operations are automatically traced +lm = dspy.OpenAI(model="gpt-4") +dspy.settings.configure(lm=lm) -### Error Reporting to Langtrace +class SimpleQA(dspy.Signature): + """Answer questions with short responses.""" + question = dspy.InputField() + answer = dspy.OutputField(desc="short answer") -By default all sdk errors are reported to langtrace via Sentry. This can be disabled by setting the following enviroment variable to `False` like so `LANGTRACE_ERROR_REPORTING=False` +compiler = BootstrapFewShot(metric=dspy.metrics.Answer()) +program = compiler.compile(SimpleQA) +``` -### Additional Customization +#### CrewAI +```python +from langtrace_python_sdk import langtrace +from crewai import Agent, Task, Crew -- `@with_langtrace_root_span` - this decorator is designed to organize and relate different spans, in a hierarchical manner. When you're performing multiple operations that you want to monitor together as a unit, this function helps by establishing a "parent" (`LangtraceRootSpan` or whatever is passed to `name`) span. Then, any calls to the LLM APIs made within the given function (fn) will be considered "children" of this parent span. This setup is especially useful for tracking the performance or behavior of a group of operations collectively, rather than individually. +langtrace.init() -```python -from langtrace_python_sdk import with_langtrace_root_span +# Agents and tasks are automatically traced +researcher = Agent( + role="Researcher", + goal="Research and analyze data", + backstory="Expert data researcher", + allow_delegation=False +) -@with_langtrace_root_span() -def example(): - response = client.chat.completions.create( - model="gpt-4", - messages=[{"role": "user", "content": "Say this is a test three times"}], - stream=False, - ) - return response +task = Task( + description="Analyze market trends", + agent=researcher +) + +crew = Crew( + agents=[researcher], + tasks=[task] +) + +result = crew.kickoff() ``` -- `inject_additional_attributes` - this function is designed to enhance the traces by adding custom attributes to the current context. These custom attributes provide extra details about the operations being performed, making it easier to analyze and understand their behavior. +For more detailed examples and framework-specific features, visit our [documentation](https://docs.langtrace.ai). + +## ⚙️ Configuration + +### Initialize Options + +The SDK can be initialized with various configuration options to customize its behavior: ```python -from langtrace_python_sdk import inject_additional_attributes +langtrace.init( + api_key: Optional[str] = None, # API key for authentication + batch: bool = True, # Enable/disable batch processing + write_spans_to_console: bool = False, # Console logging + custom_remote_exporter: Optional[Any] = None, # Custom exporter + api_host: Optional[str] = None, # Custom API host + disable_instrumentations: Optional[Dict] = None, # Disable specific integrations + service_name: Optional[str] = None, # Custom service name + disable_logging: bool = False, # Disable all logging + headers: Dict[str, str] = {}, # Custom headers +) +``` +#### Configuration Details +| Parameter | Type | Default Value | Description | +|-----------|------|---------------|-------------| +| `api_key` | `str` | `LANGTRACE_API_KEY` or `None` | The API key for authentication. Can be set via environment variable | +| `batch` | `bool` | `True` | Whether to batch spans before sending them to reduce API calls | +| `write_spans_to_console` | `bool` | `False` | Enable console logging for debugging purposes | +| `custom_remote_exporter` | `Optional[Exporter]` | `None` | Custom exporter for sending traces to your own backend | +| `api_host` | `Optional[str]` | `https://langtrace.ai/` | Custom API endpoint for self-hosted deployments | +| `disable_instrumentations` | `Optional[Dict]` | `None` | Disable specific vendor instrumentations (e.g., `{'only': ['openai']}`) | +| `service_name` | `Optional[str]` | `None` | Custom service name for trace identification | +| `disable_logging` | `bool` | `False` | Disable SDK logging completely | +| `headers` | `Dict[str, str]` | `{}` | Custom headers for API requests | -def do_llm_stuff(name=""): - response = client.chat.completions.create( - model="gpt-4", - messages=[{"role": "user", "content": "Say this is a test three times"}], - stream=False, - ) - return response +### Environment Variables +Configure Langtrace behavior using these environment variables: -def main(): - response = inject_additional_attributes(lambda: do_llm_stuff(name="llm"), {'user.id': 'userId'}) +| Variable | Description | Default | Impact | +|----------|-------------|---------|---------| +| `LANGTRACE_API_KEY` | Primary authentication method | Required* | Required if not passed to init() | +| `TRACE_PROMPT_COMPLETION_DATA` | Control prompt/completion tracing | `true` | Set to 'false' to opt out of prompt/completion data collection | +| `TRACE_DSPY_CHECKPOINT` | Control DSPy checkpoint tracing | `true` | Set to 'false' to disable checkpoint tracing | +| `LANGTRACE_ERROR_REPORTING` | Control error reporting | `true` | Set to 'false' to disable Sentry error reporting | +| `LANGTRACE_API_HOST` | Custom API endpoint | `https://langtrace.ai/` | Override default API endpoint for self-hosted deployments | - # if the function do not take arguments then this syntax will work - response = inject_additional_attributes(do_llm_stuff, {'user.id': 'userId'}) -``` +> **Performance Note**: Setting `TRACE_DSPY_CHECKPOINT=false` is recommended in production environments as checkpoint tracing involves state serialization which can impact latency. -- `with_additional_attributes` - is behaving the same as `inject_additional_attributes` but as a decorator, this will be deprecated soon. +> **Security Note**: When `TRACE_PROMPT_COMPLETION_DATA=false`, no prompt or completion data will be collected, ensuring sensitive information remains private. -```python -from langtrace_python_sdk import with_langtrace_root_span, with_additional_attributes +## 🔧 Advanced Features +### Root Span Decorator -@with_additional_attributes({"user.id": "1234"}) -def api_call1(): - response = client.chat.completions.create( - model="gpt-4", - messages=[{"role": "user", "content": "Say this is a test three times"}], - stream=False, - ) - return response +Use the root span decorator to create custom trace hierarchies: +```python +from langtrace_python_sdk import langtrace -@with_additional_attributes({"user.id": "5678"}) -def api_call2(): - response = client.chat.completions.create( - model="gpt-4", - messages=[{"role": "user", "content": "Say this is a test three times"}], - stream=False, - ) - return response +@langtrace.with_langtrace_root_span(name="custom_operation") +def my_function(): + # Your code here + pass +``` +### Additional Attributes -@with_langtrace_root_span() -def chat_completion(): - api_call1() - api_call2() +Inject custom attributes into your traces: + +```python +# Using decorator +@langtrace.with_additional_attributes({"custom_key": "custom_value"}) +def my_function(): + pass + +# Using context manager +with langtrace.inject_additional_attributes({"custom_key": "custom_value"}): + # Your code here + pass ``` -- `get_prompt_from_registry` - this function is designed to fetch the desired prompt from the `Prompt Registry`. You can pass two options for filtering `prompt_version` & `variables`. +### Prompt Registry + +Register and manage prompts for better traceability: ```python -from langtrace_python_sdk import get_prompt_from_registry +from langtrace_python_sdk import langtrace -prompt = get_prompt_from_registry(, options={"prompt_version": 1, "variables": {"foo": "bar"} }) +# Register a prompt template +langtrace.register_prompt("greeting", "Hello, {name}!") + +# Use registered prompt +response = client.chat.completions.create( + model="gpt-4", + messages=[{"role": "user", "content": langtrace.get_prompt("greeting", name="Alice")}] +) ``` -### Opt out of tracing prompt and completion data +### User Feedback System -By default, prompt and completion data are captured. If you would like to opt out of it, set the following env var, +Collect and analyze user feedback: -`TRACE_PROMPT_COMPLETION_DATA=false` +```python +from langtrace_python_sdk import langtrace -### Enable/Disable checkpoint tracing for DSPy +# Record user feedback for a trace +langtrace.record_feedback( + trace_id="your_trace_id", + rating=5, + feedback_text="Great response!", + metadata={"user_id": "123"} +) +``` -By default, checkpoints are traced for DSPy pipelines. If you would like to disable it, set the following env var, +### DSPy Checkpointing -`TRACE_DSPY_CHECKPOINT=false` +Manage DSPy checkpoints for workflow tracking: -Note: Checkpoint tracing will increase the latency of executions as the state is serialized. Please disable it in production. +```python +from langtrace_python_sdk import langtrace -## Supported integrations +# Enable checkpoint tracing (disabled by default in production) +langtrace.init( + api_key="your_api_key", + dspy_checkpoint_tracing=True +) +``` -Langtrace automatically captures traces from the following vendors: +### Vector Database Operations -| Vendor | Type | Typescript SDK | Python SDK | -| ------------- | --------------- | ------------------ | ------------------------------- | -| OpenAI | LLM | :white_check_mark: | :white_check_mark: | -| Anthropic | LLM | :white_check_mark: | :white_check_mark: | -| Azure OpenAI | LLM | :white_check_mark: | :white_check_mark: | -| Cohere | LLM | :white_check_mark: | :white_check_mark: | -| Groq | LLM | :x: | :white_check_mark: | -| Perplexity | LLM | :white_check_mark: | :white_check_mark: | -| Gemini | LLM | :x: | :white_check_mark: | -| Mistral | LLM | :x: | :white_check_mark: | -| Langchain | Framework | :x: | :white_check_mark: | -| Langgraph | Framework | :x: | :white_check_mark: | -| LlamaIndex | Framework | :white_check_mark: | :white_check_mark: | -| AWS Bedrock | Framework | :white_check_mark: | :white_check_mark: | -| LiteLLM | Framework | :x: | :white_check_mark: | -| DSPy | Framework | :x: | :white_check_mark: | -| CrewAI | Framework | :x: | :white_check_mark: | -| Ollama | Framework | :x: | :white_check_mark: | -| VertexAI | Framework | :x: | :white_check_mark: | -| Vercel AI SDK | Framework | :white_check_mark: | :x: | -| EmbedChain | Framework | :x: | :white_check_mark: | -| Autogen | Framework | :x: | :white_check_mark: | -| Pinecone | Vector Database | :white_check_mark: | :white_check_mark: | -| ChromaDB | Vector Database | :white_check_mark: | :white_check_mark: | -| QDrant | Vector Database | :white_check_mark: | :white_check_mark: | -| Weaviate | Vector Database | :white_check_mark: | :white_check_mark: | -| PGVector | Vector Database | :white_check_mark: | :white_check_mark: (SQLAlchemy) | +Track vector database operations: ---- +```python +from langtrace_python_sdk import langtrace + +# Vector operations are automatically traced +with langtrace.inject_additional_attributes({"operation_type": "similarity_search"}): + results = vector_db.similarity_search("query", k=5) +``` -## Feature Requests and Issues +For more detailed examples and use cases, visit our [documentation](https://docs.langtrace.ai). -- To request for features, head over [here to start a discussion](https://github.com/Scale3-Labs/langtrace/discussions/categories/feature-requests). -- To raise an issue, head over [here and create an issue](https://github.com/Scale3-Labs/langtrace/issues). + ---- +## 📐 Examples -## Contributions + -We welcome contributions to this project. To get started, fork this repository and start developing. To get involved, join our [Discord](https://discord.langtrace.ai) workspace. +## 🏠 Langtrace Self Hosted -- If you want to run any of the examples go to `run_example.py` file, you will find `ENABLED_EXAMPLES`. choose the example you want to run and just toggle the flag to `True` and run the file using `python src/run_example.py` + +Get started with self-hosted Langtrace: -- If you want to run tests, make sure to install dev & test dependencies: +```python +from langtrace_python_sdk import langtrace +langtrace.init(write_spans_to_console=True) # For console logging +# OR +langtrace.init(custom_remote_exporter=, batch=) # For custom exporter +``` - ```python - pip install '.[test]' && pip install '.[dev]' - ``` +## 🤝 Contributing - then run `pytest` using: +We welcome contributions! To get started: - ```python - pytest -v - ``` +1. Fork this repository and start developing +2. Join our [Discord](https://discord.langtrace.ai) workspace +3. Run examples: + ```python + # In run_example.py, set ENABLED_EXAMPLES flag to True for desired example + python src/run_example.py + ``` +4. Run tests: + ```python + pip install '.[test]' && pip install '.[dev]' + pytest -v + ``` ---- +## 🔒 Security -## Security +To report security vulnerabilities, email us at . You can read more on security [here](https://github.com/Scale3-Labs/langtrace/blob/development/SECURITY.md). -To report security vulnerabilites, email us at . You can read more on security [here](https://github.com/Scale3-Labs/langtrace/blob/development/SECURITY.md). +## ❓ Frequently Asked Questions ---- + -## License +## 📜 License -- Langtrace application is [licensed](https://github.com/Scale3-Labs/langtrace/blob/development/LICENSE) under the AGPL 3.0 License. You can read about this license [here](https://www.gnu.org/licenses/agpl-3.0.en.html). -- Langtrace SDKs are licensed under the Apache 2.0 License. You can read about this license [here](https://www.apache.org/licenses/LICENSE-2.0). +Langtrace Python SDK is licensed under the Apache 2.0 License. You can read about this license [here](https://www.apache.org/licenses/LICENSE-2.0).