Skip to content

fix(ci): run Langfuse v4 in dual-write mode - #1795

Merged
hassiebp merged 3 commits into
mainfrom
hassiebbot/v4-ready-test-suite
Jul 30, 2026
Merged

fix(ci): run Langfuse v4 in dual-write mode#1795
hassiebp merged 3 commits into
mainfrom
hassiebbot/v4-ready-test-suite

Conversation

@hassiebp

@hassiebp hassiebp commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

What changed

  • run the Langfuse v4 web and worker containers with LANGFUSE_MIGRATION_V4_WRITE_MODE=dual
  • keep the existing SDK implementation and test assertions unchanged

Why

Langfuse platform v4 is now pulled by default in CI, where the default write mode is events_only. The Python SDK test suite still intentionally exercises APIs backed by the legacy trace and observation tables, including the experiment runner's dataset-run-item POST request.

Dual-write mode populates both the v4 events tables and the legacy tables, allowing CI to validate the current SDK contract against the v4 deployment while the remaining platform compatibility issue is fixed separately.

Scope

This is only a CI deployment configuration change. It does not migrate SDK ingestion, batch evaluation, experiment handling, support helpers, or test expectations to v4-native APIs.

Validation

  • confirmed the Compose override resolves LANGFUSE_MIGRATION_V4_WRITE_MODE: dual for both langfuse-web and langfuse-worker
  • ran an isolated Langfuse v4 stack with the override
  • passed 7 representative E2E tests from the original CI failures:
    • trace ingestion and reads
    • observation listing
    • local and server-backed experiment runs
    • multiple experiments on one dataset
    • persisted experiment scores

CI is the authoritative full-suite validation.

Linear

LFE-14622: Run Python SDK CI against Langfuse v4 in dual-write mode

@hassiebp
hassiebp marked this pull request as ready for review July 30, 2026 11:11
@github-actions

Copy link
Copy Markdown

@claude review

Comment thread langfuse/batch_evaluation.py Outdated
Comment thread langfuse/batch_evaluation.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e63e5cbb9f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread langfuse/batch_evaluation.py Outdated
Comment thread langfuse/batch_evaluation.py Outdated
Comment thread langfuse/batch_evaluation.py Outdated
Comment thread langfuse/batch_evaluation.py Outdated
Comment thread langfuse/batch_evaluation.py Outdated
@hassiebp
hassiebp requested a review from a team as a code owner July 30, 2026 11:20
@hassiebp hassiebp changed the title fix(ci): make test suite compatible with Langfuse v4 fix(ci): run Langfuse v4 in dual-write mode Jul 30, 2026
@hassiebp
hassiebp merged commit f857aea into main Jul 30, 2026
20 checks passed
@hassiebp
hassiebp deleted the hassiebbot/v4-ready-test-suite branch July 30, 2026 11:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant