|
| 1 | +--- |
| 2 | +title: Haystack <> Langfuse Integration |
| 3 | +date: 2024/05/16 |
| 4 | +description: Easily monitor and trace your Haystack pipelines with this new Langfuse integration. |
| 5 | +tag: integration |
| 6 | +ogImage: /images/blog/haystack/haystack_og.png |
| 7 | +author: Lydia |
| 8 | +--- |
| 9 | + |
| 10 | +import { BlogHeader } from "@/components/blog/BlogHeader"; |
| 11 | + |
| 12 | +<BlogHeader |
| 13 | + title="Haystack <> Langfuse Integration" |
| 14 | + description="OSS observability and analytics for the popular RAG application framework." |
| 15 | + date="May 17, 2024" |
| 16 | + authors={["lydiayou"]} |
| 17 | +/> |
| 18 | + |
| 19 | +We're excited to highlight a new Langfuse integration with Haystack! |
| 20 | + |
| 21 | +This integration allows you to easily trace your Haystack pipelines in the Langfuse UI. We've previously launched integrations with popular tools that devs love -- including [LlamaIndex](/blog/llama-index-integration), [LangChain](/docs/integrations/langchain/tracing) and [LiteLLM](/docs/integrations/litellm) -- and we're excited to be continuing that with Haystack. |
| 22 | + |
| 23 | +Thanks to the team at deepset for developing the integration. We're excited to see how you use it! |
| 24 | + |
| 25 | +## What's New |
| 26 | + |
| 27 | +The `langfuse-haystack` package integrates tracing capabilities into Haystack (2.x) pipelines using Langfuse. You can then add `LangfuseConnector` as a tracer to automatically trace the operations and data flow within the pipeline. |
| 28 | + |
| 29 | +<CloudflareVideo |
| 30 | + videoId="36a42457d879923f84681f8fe62443e4" |
| 31 | + aspectRatio={16 / 10.26} |
| 32 | + title="Haystack Trace" |
| 33 | + gifStyle |
| 34 | +/> |
| 35 | + |
| 36 | +## What is Haystack? |
| 37 | + |
| 38 | +[Haystack](https://haystack.deepset.ai/) is the open-source Python framework developed by deepset. Its modular design allows users to implement custom pipelines to build production-ready LLM applications, like retrieval-augmented generative (RAG) pipelines and state-of-the-art search systems. It integrates with Hugging Face Transformers, Elasticsearch, OpenSearch, OpenAI, Cohere, Anthropic and others, making it an extremely popular framework for teams of all sizes. |
| 39 | + |
| 40 | +**RAG has proven to be a pragmatic and efficient way of working with LLMs**. The integration of custom data sources through RAG can significantly enhance the quality of an LLM’s response, improving user experience. Haystack is a lightweight and powerful tool to build data-augmented LLM applications; you can read more about their approach to the building blocks of pipelines [here](https://docs.haystack.deepset.ai/docs/intro). |
| 41 | + |
| 42 | +Haytsack recently introduced Haystack 2.0 with a new architecture and modular design. Building on top of Haystack makes applications easier to understand, maintain and extend. |
| 43 | + |
| 44 | +## How Can Langfuse Help? |
| 45 | + |
| 46 | +[Langfuse tracing](/docs/tracing) can be helpful for Haystack pipelines in the following ways: |
| 47 | + |
| 48 | +- Capture comprehensive details of each execution trace in a beautiful UI dashboard |
| 49 | + - Latency |
| 50 | + - Token usage |
| 51 | + - Cost |
| 52 | + - Scores |
| 53 | +- Capture the full context of the execution |
| 54 | +- Monitor and score traces |
| 55 | +- Build fine-tuning and testing datasets |
| 56 | + |
| 57 | +Langfuse integration with a tool like Haystack can help monitor model performance, can assist with pinpointing areas for improvement, or create datasets from your pipeline executions for fine-tuning and testing. |
| 58 | + |
| 59 | +## Overview |
| 60 | + |
| 61 | +<CloudflareVideo |
| 62 | + videoId="aaa30e674c281a9c9591af03b5f668d2" |
| 63 | + aspectRatio={16 / 10} |
| 64 | + title="Haystack Integration Overview" |
| 65 | +/> |
| 66 | + |
| 67 | +## Quickstart |
| 68 | + |
| 69 | +Here are the steps to get started! First, add your API keys. You can find your Langfuse public and private keys on the dashboard. Make sure to set `HAYSTACK_CONTENT_TRACING_ENABLED` to `"True"`. |
| 70 | + |
| 71 | +```python |
| 72 | +import os |
| 73 | + |
| 74 | +# Get keys for your project from the project settings page |
| 75 | +# https://cloud.langfuse.com |
| 76 | +os.environ["LANGFUSE_PUBLIC_KEY"] = "pk-lf-..." |
| 77 | +os.environ["LANGFUSE_SECRET_KEY"] = "sk-lf-..." |
| 78 | +os.environ["LANGFUSE_HOST"] = "https://cloud.langfuse.com" # 🇪🇺 EU region |
| 79 | +# os.environ["LANGFUSE_HOST"] = "https://us.cloud.langfuse.com" # 🇺🇸 US region |
| 80 | +os.environ["HAYSTACK_CONTENT_TRACING_ENABLED"] = "True" |
| 81 | + |
| 82 | +# Your openai key |
| 83 | +os.environ["OPENAI_API_KEY"] = "sk-proj-..." |
| 84 | +``` |
| 85 | + |
| 86 | +Here's how you add `LangfuseConnector` as a tracer to the pipeline: |
| 87 | + |
| 88 | +```python |
| 89 | +from datasets import load_dataset |
| 90 | +from haystack import Document, Pipeline |
| 91 | +from haystack.components.builders import PromptBuilder |
| 92 | +from haystack.components.embedders import SentenceTransformersDocumentEmbedder, SentenceTransformersTextEmbedder |
| 93 | +from haystack.components.generators import OpenAIGenerator |
| 94 | +from haystack.components.retrievers import InMemoryEmbeddingRetriever |
| 95 | +from haystack.document_stores.in_memory import InMemoryDocumentStore |
| 96 | +from haystack_integrations.components.connectors.langfuse import LangfuseConnector |
| 97 | + |
| 98 | +basic_rag_pipeline = Pipeline() |
| 99 | + # Add components to your pipeline |
| 100 | + basic_rag_pipeline.add_component("tracer", LangfuseConnector("Basic RAG Pipeline")) |
| 101 | + basic_rag_pipeline.add_component( |
| 102 | + "text_embedder", SentenceTransformersTextEmbedder(model="sentence-transformers/all-MiniLM-L6-v2") |
| 103 | + ) |
| 104 | + basic_rag_pipeline.add_component("retriever", retriever) |
| 105 | + basic_rag_pipeline.add_component("prompt_builder", prompt_builder) |
| 106 | + basic_rag_pipeline.add_component("llm", OpenAIGenerator(model="gpt-3.5-turbo", generation_kwargs={"n": 2})) |
| 107 | +``` |
| 108 | + |
| 109 | +For each trace, you can see: |
| 110 | + |
| 111 | +- Latency for each component of the pipeline |
| 112 | +- Input and output for each step |
| 113 | +- For generations, token usage and costs are automatically calculated. |
| 114 | + |
| 115 | +If you want to learn more about traces and what they can do in Langfuse, read our [documentation](/docs/tracing). |
| 116 | + |
| 117 | +## Dive In |
| 118 | + |
| 119 | +Head to the [Langfuse Docs](/docs/integrations/haystack/get-started) or see an example integration in this [end-to-end cookbook](/docs/integrations/haystack/example-python) to dive straight in. |
| 120 | + |
| 121 | +import { FileCode, BookOpen, Video } from "lucide-react"; |
| 122 | + |
| 123 | +<Cards num={3}> |
| 124 | + <Card |
| 125 | + title="Docs" |
| 126 | + href="/docs/integrations/haystack/get-started" |
| 127 | + icon={<BookOpen />} |
| 128 | + /> |
| 129 | + <Card |
| 130 | + title="Coookbook" |
| 131 | + href="/docs/integrations/haystack/example-python" |
| 132 | + icon={<FileCode />} |
| 133 | + /> |
| 134 | + <Card title="Video" href="/guides/videos/haystack" icon={<Video />} /> |
| 135 | +</Cards> |
0 commit comments