Migration from LangSmith to Self-Hosted Langfuse (Prompts, Datasets, Traces)

[hiteshbalapanuru11]

Describe your question

Hello,

I am currently using LangSmith and would like to migrate my assets to a self-hosted Langfuse instance.

Specifically, I’m looking to migrate:

• Prompts (including versions, variables, and metadata)
• Datasets (test cases, inputs/outputs, evaluation results)
• Traces (runs, spans, logs, feedback, and metadata)

I would appreciate guidance on best practices for:

1. Exporting data from LangSmith in a structured and complete format
2. Mapping LangSmith objects (runs, projects, datasets, etc.) to their Langfuse equivalents
3. Preserving relationships between prompts, traces, and datasets
4. Migrating historical traces at scale (performance considerations, batching, API limits)
5. Handling evaluation results and feedback during migration
6. Any existing tools, scripts, or recommended workflows for such migrations

If there are recommended schemas, APIs, or migration utilities available, examples would be very helpful.

Thanks in advance for your guidance!

Langfuse Cloud or Self-Hosted?

Self-Hosted

If Self-Hosted

No response

If Langfuse Cloud

No response

SDK and integration versions

No response

Pre-Submission Checklist

• I have checked for existing issues/discussions and consulted Langfuse AI.

[inkeep-ai-assistant[bot]]

While the available documentation doesn't specifically cover migrating from LangSmith to Langfuse, there is a comprehensive guide for migrating data between Langfuse projects that demonstrates the core patterns you'll need(1).

Migration Approach

The documented migration cookbook provides Python code for transferring prompts, traces, and datasets between Langfuse instances using the Public API(1). This same approach can be adapted for LangSmith migration by:

1. Exporting from LangSmith - Use LangSmith's export APIs to extract your data
2. Transforming to Langfuse format - Map LangSmith objects to Langfuse equivalents
3. Importing via Langfuse API - Use the patterns shown in the migration guide

Key Migration Components

Prompts Migration

The documentation shows how to migrate prompts including version history(1). The code handles both chat and text prompt types and preserves labels, configuration, and tags(1).

Traces Migration

For traces, the guide demonstrates transforming trace data into ingestion batch events(1). The key pattern involves:

• Preserving original trace IDs
• Generating new IDs for observations within traces
• Mapping parent/child relationships between observations
• Supporting spans, generations, events, and scores(1)

Datasets Migration

Dataset migration includes dataset items with inputs, expected outputs, and metadata(1).

Important Considerations

Rate Limits: Be mindful of API rate limits on your Langfuse instance. Large migrations might require implementing back-off logic or running in batches(1).

Prerequisites: You'll need access to both source and destination projects with appropriate API keys(1).

Time Filtering: The migration code supports optional time filters to migrate data from specific date ranges(1).

Recommended Next Steps

1. Review the complete migration cookbook at the provided source(1)
2. Study the data transformation functions, particularly transform_trace_to_ingestion_batch which shows how to structure trace data for the Langfuse ingestion API(1)
3. Adapt the patterns for your LangSmith export format
4. Test with a small subset of data before full migration

For the complete working code examples including batch processing and error handling, refer to the migration guide(1).

📚 Sources:

1. Migrating Data from One Langfuse Project to Another

---

Have another question? Just tag @inkeep.

[jannikmaierhoefer]

Hi @hiteshbalapanuru11, thanks. Unfortunately, we don't have any guides on migrating LangSmith data to Langfuse at this point.

[hiteshbalapanuru11]

Thanks for the clarification.I appreciate the response.

Are there any plans to publish documentation or guidance for migrating data from LangSmith to LangFuse in the future?

[jannikmaierhoefer]

@hiteshbalapanuru11, there are no concrete plans for this at the moment. I would assume that Claude Code (or similar) could give you a good overview of how data should be mapped between the two providers given ther documentations.