diff --git a/docs/_static/imgs/trace-langsmith.png b/docs/_static/imgs/trace-langsmith.png new file mode 100644 index 000000000..ca120c654 Binary files /dev/null and b/docs/_static/imgs/trace-langsmith.png differ diff --git a/docs/howtos/applications/index.md b/docs/howtos/applications/index.md index aa10ae334..8a43beb32 100644 --- a/docs/howtos/applications/index.md +++ b/docs/howtos/applications/index.md @@ -11,4 +11,5 @@ compare_embeddings compare_llms custom_prompts use_prompt_adaptation +tracing ``` diff --git a/docs/howtos/applications/tracing.md b/docs/howtos/applications/tracing.md new file mode 100644 index 000000000..3a581067b --- /dev/null +++ b/docs/howtos/applications/tracing.md @@ -0,0 +1,41 @@ + +# Logging and tracing + +Logging and tracing results from llm are important for any language model-based application. This is a tutorial on how to do tracing with Ragas. Ragas provides `callbacks` functionality which allows you to hook various tracers like Langmsith, wandb, etc easily. In this notebook, I will be using Langmith for tracing + +To set up Langsmith, we need to set some environment variables that it needs. For more information, you can refer to the [docs](https://docs.smith.langchain.com/) + +```{code-block} bash +export LANGCHAIN_TRACING_V2=true +export LANGCHAIN_ENDPOINT=https://api.smith.langchain.com +export LANGCHAIN_API_KEY= +export LANGCHAIN_PROJECT= # if not specified, defaults to "default" +``` + +Now we have to import the required tracer from langchain, here we are using `LangChainTracer` but you can similarly use any tracer supported by langchain like [WandbTracer](https://python.langchain.com/docs/integrations/providers/wandb_tracing) + +```{code-block} python +# langsmith +from langchain.callbacks.tracers import LangChainTracer + +tracer = LangChainTracer(project_name="callback-experiments") +``` + +We now pass the tracer to the `callbacks` parameter when calling `evaluate` + +```{code-block} python +from datasets import load_dataset +from ragas.metrics import context_precision +from ragas import evaluate + +dataset = load_dataset("explodinggradients/amnesty_qa","english") +evaluate(dataset["train"],metrics=[context_precision],callbacks=[tracer]) +``` + +``` +{'context_precision': 1.0000} +``` +![](./../../_static/imgs/trace-langsmith.png) + + +You can also write your own custom callbacks using langchain’s `BaseCallbackHandler`, refer [here](https://www.notion.so/Docs-logging-and-tracing-6f21cde9b3cb4d499526f48fd615585d?pvs=21) to read more about it. \ No newline at end of file