Skip to content

Commit f856b51

Browse files
marcklingenmarcklingen
committed
docs: query traces and observations via sdks
1 parent be7cea2 commit f856b51

File tree

8 files changed

+250
-44
lines changed

8 files changed

+250
-44
lines changed

‎next.config.mjs‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -201,6 +201,7 @@ const nonPermanentRedirects = [
201201
["/changelog/2024-03-03-posthog-integration", "/docs/analytics/posthog"],
202202
["/guides/videos/2-min", "/guides/videos/introducing-langfuse-2.0"],
203203
["/tos", "/terms"],
204+
["/docs/export-and-fine-tuning", "/docs/query-traces"],
204205

205206
// Reorder Tracing section
206207
["/docs/tracing/overview", "/docs/tracing"],

‎pages/changelog/2024-07-04-query-traces-via-sdks.mdx‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
---
2+
date: 2024-07-04
3+
title: Query Traces via SDKs
4+
description: The Langfuse Python and JS SDKs now support fetching traces and observations from the Langfuse API. Use them for custom analysis, few-shotting or fine-tuning workflows.
5+
ogImage: /images/changelog/2024-07-04-query-traces-via-sdks.png
6+
showOgInHeader: false
7+
author: Marc
8+
---
9+
10+
import { ChangelogHeader } from "@/components/changelog/ChangelogHeader";
11+
12+
<ChangelogHeader />
13+
14+
Since the very beginning, all Langfuse Tracing data has been open and accessible via the Langfuse API. Today, we're excited to finally release the SDK methods to conveniently query traces and observations directly from the Langfuse API.
15+
16+
<Tabs items={["Python", "JS/TS"]}>
17+
<Tab>
18+
19+
```bash
20+
pip install langfuse
21+
```
22+
23+
```python
24+
from langfuse import Langfuse
25+
langfuse = Langfuse()
26+
27+
# Fetch list of traces, supports filters and pagination
28+
traces = langfuse.fetch_traces()
29+
30+
# Fetch a single trace by ID
31+
trace = langfuse.fetch_trace("traceId")
32+
33+
# Fetch list of observations, supports filters and pagination
34+
observations = langfuse.fetch_observations()
35+
36+
# Fetch a single observation by ID
37+
observation = langfuse.fetch_observation("observationId")
38+
```
39+
40+
</Tab>
41+
<Tab>
42+
43+
```bash
44+
npm install langfuse
45+
```
46+
47+
```typescript
48+
import { Langfuse } from "langfuse";
49+
const langfuse = new Langfuse();
50+
51+
// Fetch list of traces, supports filters and pagination
52+
const traces = await langfuse.fetchTraces();
53+
54+
// Fetch a single trace by ID
55+
const trace = await langfuse.fetchTrace("traceId");
56+
57+
// Fetch list of observations, supports filters and pagination
58+
const observations = await langfuse.fetchObservations();
59+
60+
// Fetch a single observation by ID
61+
const observation = await langfuse.fetchObservation("observationId");
62+
```
63+
64+
</Tab>
65+
</Tabs>
66+
67+
See [docs](/docs/query-traces) to learn more.

‎pages/docs/_meta.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,14 +15,15 @@
1515
"tracing-features": "Features",
1616
"sdk": "SDKs",
1717
"integrations": "Integrations",
18+
"query-traces": "Query Traces",
1819

1920
"-- Develop": {
2021
"type": "separator",
2122
"title": "Develop"
2223
},
2324
"prompts": "Prompt Management",
2425
"playground": "Playground",
25-
"export-and-fine-tuning": "Export & Fine-tuning",
26+
"fine-tuning": "Fine-tuning",
2627

2728
"-- Monitor": {
2829
"type": "separator",

‎pages/docs/export-and-fine-tuning.mdx‎

Lines changed: 0 additions & 39 deletions
This file was deleted.

‎pages/docs/fine-tuning.mdx‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
---
2+
description: Langfuse traces all your development and production LLM calls. You can export this data to train or fine-tune models at any time.
3+
---
4+
5+
# Fine-tuning
6+
7+
Langfuse is [open-source](/docs/open-source) and data tracked with Langfuse is open. You can easily [trace](/docs/tracing) your application, collect user feedback, and then use the data to fine-tune a model for your specific use case.
8+
9+
## Export generations
10+
11+
The generations table in Langfuse allows you to export all LLM calls recorded with Langfuse. All filters applied to the table will be applied to the export, thus, you can e.g. filter for a specific model or minimum evaluation threshold and then export the generations.
12+
13+
Available export formats:
14+
15+
- CSV
16+
- JSON
17+
18+
<Frame>
19+
![Download generations in Langfuse
20+
UI](https://static.langfuse.com/changelog-2023-09-18-export-generations.gif)
21+
</Frame>
22+
23+
## SDKs/API
24+
25+
All data collected in Langfuse is also available programmatically via the API and SDKs (Python, JS/TS). Refer to the [API reference](https://api.reference.langfuse.com/) and [tracing docs (_query traces_)](/docs/query-traces) for more information.
26+
27+
## Trouble exporting?
28+
29+
If you're having trouble exporting data, please [let us know](#contact) and we'll help you out.

‎pages/docs/query-traces.mdx‎

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
---
2+
description: Conveniently fetch your LLM Observability traces for few-shotting, fine-tuning or further analysis.
3+
---
4+
5+
# Query Traces
6+
7+
Langfuse is [open-source](/docs/open-source) and data tracked with Langfuse is open. You can query/export data via: [SDKs](#sdks), [API](#api), and [Langfuse UI](#ui).
8+
9+
Common use cases:
10+
11+
- Train or [fine-tune](/docs/fine-tuning) models on the production traces in Langfuse. E.g. to create a small model after having used a large model in production for a specific use case.
12+
- Collect few-shot examples to improve quality of output.
13+
- Programmatically create [datasets](/docs/datasets).
14+
15+
Want to learn more about the Langfuse data model? Check out our [introduction to tracing in Langfuse](/docs/tracing).
16+
17+
## SDKs [#sdks]
18+
19+
Via the [SDKs](/docs/sdk/overview) for Python and JS/TS you can easily query the API without having to write the HTTP requests yourself.
20+
21+
<Tabs items={["Python", "JS/TS"]}>
22+
<Tab>
23+
24+
```bash
25+
pip install langfuse
26+
```
27+
28+
```python
29+
from langfuse import Langfuse
30+
langfuse = Langfuse(
31+
secret_key="sk-lf-...",
32+
public_key="pk-lf-...",
33+
host="https://cloud.langfuse.com" # 🇪🇺 EU region
34+
# host="https://us.cloud.langfuse.com" # 🇺🇸 US region
35+
)
36+
37+
# Fetch list of traces, supports filters and pagination
38+
traces = langfuse.fetch_traces()
39+
40+
# Fetch a single trace by ID
41+
trace = langfuse.fetch_trace("traceId")
42+
43+
# Fetch list of observations, supports filters and pagination
44+
observations = langfuse.fetch_observations()
45+
46+
# Fetch a single observation by ID
47+
observation = langfuse.fetch_observation("observationId")
48+
```
49+
50+
Python SDK reference:
51+
52+
- [`fetch_traces()`](https://python.reference.langfuse.com/langfuse/client#Langfuse.fetch_traces)
53+
- [`fetch_trace()`](https://python.reference.langfuse.com/langfuse/client#Langfuse.fetch_trace)
54+
- [`fetch_observations()`](https://python.reference.langfuse.com/langfuse/client#Langfuse.fetch_observations)
55+
- [`fetch_observation()`](https://python.reference.langfuse.com/langfuse/client#Langfuse.fetch_observation)
56+
57+
</Tab>
58+
<Tab>
59+
60+
```bash
61+
npm install langfuse
62+
```
63+
64+
```typescript
65+
import { Langfuse } from "langfuse";
66+
const langfuse = new Langfuse({
67+
secretKey: "sk-lf-...",
68+
publicKey: "pk-lf-...",
69+
baseUrl: "https://cloud.langfuse.com", // 🇪🇺 EU region
70+
// baseUrl: "https://us.cloud.langfuse.com", // 🇺🇸 US region
71+
});
72+
73+
// Fetch list of traces, supports filters and pagination
74+
const traces = await langfuse.fetchTraces();
75+
76+
// Fetch a single trace by ID
77+
const trace = await langfuse.fetchTrace("traceId");
78+
79+
// Fetch list of observations, supports filters and pagination
80+
const observations = await langfuse.fetchObservations();
81+
82+
// Fetch a single observation by ID
83+
const observation = await langfuse.fetchObservation("observationId");
84+
```
85+
86+
JS/TS SDK reference:
87+
88+
- [`fetchTraces()`](https://js.reference.langfuse.com/classes/langfuse.Langfuse.html#fetchTraces)
89+
- [`fetchTrace()`](https://js.reference.langfuse.com/classes/langfuse.Langfuse.html#fetchTrace)
90+
- [`fetchObservations()`](https://js.reference.langfuse.com/classes/langfuse.Langfuse.html#fetchObservations)
91+
- [`fetchObservation()`](https://js.reference.langfuse.com/classes/langfuse.Langfuse.html#fetchObservation)
92+
93+
</Tab>
94+
</Tabs>
95+
96+
## API [#api]
97+
98+
All data in Langfuse is available via the API. Refer to the [API reference](https://api.reference.langfuse.com/) for more information.
99+
100+
Example routes:
101+
102+
```bash
103+
GET /api/public/traces
104+
GET /api/public/traces/:traceId
105+
106+
GET /api/public/observations
107+
GET /api/public/observations/:observationId
108+
109+
GET /api/public/scores
110+
GET /api/public/scores/:scoreId
111+
```
112+
113+
## Export via UI [#ui]
114+
115+
<Callout type="info">
116+
This is currently only available for the `generations` table in the UI. We're
117+
working on adding more export options for other objects.
118+
</Callout>
119+
120+
All filters applied to the table will be applied to the export.
121+
122+
Available export formats:
123+
124+
- CSV
125+
- JSON
126+
127+
<Frame>
128+
![Download generations in Langfuse
129+
UI](https://static.langfuse.com/changelog-2023-09-18-export-generations.gif)
130+
</Frame>
131+
132+
## Trouble exporting?
133+
134+
If you're having trouble exporting data, please [let us know](#contact) and we'll help you out.

‎pages/docs/tracing.mdx‎

Lines changed: 17 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -10,13 +10,13 @@ LLM applications use increasingly complex abstractions, such as chains, agents w
1010
videoId="be2a8276c589a0de39ab846d505cf731"
1111
aspectRatio={1.516}
1212
gifStyle
13-
className='block dark:hidden'
13+
className="block dark:hidden"
1414
/>
1515
<CloudflareVideo
1616
videoId="835959b1fbe097cd396e7b20cd5cf91e"
1717
aspectRatio={1.516}
1818
gifStyle
19-
className='hidden dark:block'
19+
className="hidden dark:block"
2020
/>
2121
_Example trace of our [public demo](/docs/demo)_
2222

@@ -66,7 +66,7 @@ classDiagram
6666

6767
</div>
6868

69-
<div className="border rounded py-4 text-center bg-background block dark:hidden">
69+
<div className="border rounded py-4 text-center bg-background block dark:hidden">
7070

7171
**Example trace in Langfuse UI**
7272

@@ -109,7 +109,14 @@ import { Rocket, Joystick } from "lucide-react";
109109

110110
You can extend the tracing capabilities of Langfuse by using the following features:
111111

112-
import { Users, Tag, MessagesSquare, Braces, Globe } from "lucide-react";
112+
import {
113+
Users,
114+
Tag,
115+
MessagesSquare,
116+
Braces,
117+
Globe,
118+
Database,
119+
} from "lucide-react";
113120

114121
<Cards num={3}>
115122
<Card
@@ -137,6 +144,12 @@ import { Users, Tag, MessagesSquare, Braces, Globe } from "lucide-react";
137144
icon={<Globe />}
138145
arrow
139146
/>
147+
<Card
148+
title="Export traces"
149+
href="/docs/query-traces"
150+
icon={<Database />}
151+
arrow
152+
/>
140153
</Cards>
141154

142155
## Enable/disable tracing

‎public/images/changelog/2024-07-04-query-traces-via-sdks.png‎

62.1 KB
Loading

0 commit comments

Comments
 (0)