Skip to content

feat: replace Langfuse with local-first tracing system#183

Merged
anandgupta42 merged 4 commits intomainfrom
feat/tracing
Mar 16, 2026
Merged

feat: replace Langfuse with local-first tracing system#183
anandgupta42 merged 4 commits intomainfrom
feat/tracing

Conversation

@anandgupta42
Copy link
Contributor

What does this PR do?

Replaces the Langfuse SDK integration with a vendor-neutral, exporter-based tracing system. Every CLI and TUI session now captures full traces (generations, tool calls, tokens, cost, timing) as local JSON files with an interactive HTML viewer.

Core additions:

  • Tracer class with FileExporter (local JSON) and HttpExporter (remote endpoints)
  • 4-mode HTML trace viewer (Waterfall, Tree, Chat, Log) via Bun.serve
  • altimate-code trace list and altimate-code trace view CLI commands
  • Data engineering semantic attributes for warehouse/SQL/dbt/quality/cost
  • Config schema: tracing.enabled, tracing.dir, tracing.maxFiles, tracing.exporters

Key design decisions:

  • Crash-safe: writeFileSync in signal handlers, atomic tmp+rename snapshots
  • Config-aware: TUI and CLI both honor tracing.* config settings
  • Fail-safe: all tracing methods wrapped in try-catch, never crashes the app
  • Localhost-only: trace viewer server binds to 127.0.0.1
  • Session finalization: TUI traces end on session.status=idle

Type of change

  • New feature (non-breaking change which adds functionality)

Issue for this PR

Closes #182

How did you verify your code works?

  • 380 tests across 13 test files (unit, integration, adversarial, e2e)
  • TypeScript type-check passes (0 new errors)
  • Manual testing of trace list, trace view, and TUI /trace command
  • 6-model code review (Claude, GPT 5.2 Codex, Gemini 3.1 Pro, Kimi K2.5, MiniMax M2.5, GLM-5) with convergence round

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

kulvirgit and others added 3 commits March 15, 2026 19:56
Instrument `altimate run` with Langfuse observability (activated via
LANGFUSE_* env vars) so benchmark runs capture tool calls, tokens, cost,
and timing as traces. After evaluation, traces are updated with
benchmark_pass scores for end-to-end visibility in the Langfuse dashboard.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Generations previously had empty input (no context) and empty output
when the model only produced tool calls. Now:
- Input shows tool results from the preceding step
- Output falls back to "[tool calls: read, write, ...]" when no text

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the Langfuse SDK integration with a vendor-neutral, exporter-based
tracing system that captures full session traces locally by default.

Core changes:
- New `Tracer` class with `FileExporter` (local JSON) and `HttpExporter` (remote)
- 4-mode HTML trace viewer (Waterfall, Tree, Chat, Log) served via `Bun.serve`
- `altimate-code trace list` and `altimate-code trace view` CLI commands
- Data engineering semantic attributes (`de-attributes.ts`) for warehouse/SQL/dbt
- Tracing config schema: `tracing.enabled`, `tracing.dir`, `tracing.maxFiles`, `tracing.exporters`
- Integrated into both `run` command and TUI worker with per-session tracers

Key design decisions:
- Crash-safe: `writeFileSync` in signal handlers, atomic tmp+rename snapshots
- Config-aware: TUI and CLI both honor `tracing.*` config settings
- Fail-safe: all tracing methods wrapped in try-catch, never crashes the app
- Localhost-only: trace viewer server binds to `127.0.0.1`
- Session finalization: TUI traces properly end on `session.status=idle`

Tests: 380 tests across 13 files (unit, integration, adversarial, e2e)
Docs: `docs/docs/configure/tracing.md` + CLI docs updated

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
const row = [
formatDate(trace.startedAt).padEnd(13),
formatTimestamp(trace.startedAt).padEnd(10),
status.padEnd(10),

This comment was marked as outdated.

`padEnd(10)` was called on strings containing ANSI color codes,
causing the invisible escape characters to be counted toward the
padding length. This broke column alignment for non-"ok" statuses.

Fix: pad the visible text first, then wrap with color codes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Comment on lines +69 to +79
if (tc?.exporters) {
for (const exp of tc.exporters) {
exporters.push(new HttpExporter(exp.name, exp.endpoint, exp.headers))
}
}
tracingExporters = exporters
tracingMaxFiles = tc?.maxFiles
} catch {
// Config failure should not prevent TUI from working
}
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: A race condition in loadTracingConfig sets tracingConfigLoaded to true before async operations complete, causing subsequent calls to use uninitialized tracing exporters.
Severity: MEDIUM

Suggested Fix

Move the tracingConfigLoaded = true assignment to after the await Config.get() call has successfully completed. Alternatively, use a Promise-based loading mechanism instead of a simple boolean flag to ensure subsequent callers wait for the configuration to be fully loaded before proceeding.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.

Location: packages/opencode/src/cli/cmd/tui/worker.ts#L61-L79

Potential issue: The `loadTracingConfig` function sets the `tracingConfigLoaded` flag to
`true` immediately upon entry, before an asynchronous `Config.get()` operation
completes. If a second call to `loadTracingConfig` occurs during this async window
(e.g., from a rapid `setWorkspace` call), it will see the flag as true and return
prematurely. This causes the second call's context to proceed with an `undefined`
`tracingExporters` variable, leading to the use of default tracer settings instead of
the user's configured exporters. Traces may be written to the wrong directory or sent to
incorrect endpoints.

@anandgupta42 anandgupta42 merged commit 15d2264 into main Mar 16, 2026
8 checks passed
anandgupta42 added a commit that referenced this pull request Mar 17, 2026
* feat: add Langfuse tracing for CLI and Spider2-DBT benchmark

Instrument `altimate run` with Langfuse observability (activated via
LANGFUSE_* env vars) so benchmark runs capture tool calls, tokens, cost,
and timing as traces. After evaluation, traces are updated with
benchmark_pass scores for end-to-end visibility in the Langfuse dashboard.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: populate generation input/output in Langfuse traces

Generations previously had empty input (no context) and empty output
when the model only produced tool calls. Now:
- Input shows tool results from the preceding step
- Output falls back to "[tool calls: read, write, ...]" when no text

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: replace Langfuse with local-first tracing system

Replace the Langfuse SDK integration with a vendor-neutral, exporter-based
tracing system that captures full session traces locally by default.

Core changes:
- New `Tracer` class with `FileExporter` (local JSON) and `HttpExporter` (remote)
- 4-mode HTML trace viewer (Waterfall, Tree, Chat, Log) served via `Bun.serve`
- `altimate-code trace list` and `altimate-code trace view` CLI commands
- Data engineering semantic attributes (`de-attributes.ts`) for warehouse/SQL/dbt
- Tracing config schema: `tracing.enabled`, `tracing.dir`, `tracing.maxFiles`, `tracing.exporters`
- Integrated into both `run` command and TUI worker with per-session tracers

Key design decisions:
- Crash-safe: `writeFileSync` in signal handlers, atomic tmp+rename snapshots
- Config-aware: TUI and CLI both honor `tracing.*` config settings
- Fail-safe: all tracing methods wrapped in try-catch, never crashes the app
- Localhost-only: trace viewer server binds to `127.0.0.1`
- Session finalization: TUI traces properly end on `session.status=idle`

Tests: 380 tests across 13 files (unit, integration, adversarial, e2e)
Docs: `docs/docs/configure/tracing.md` + CLI docs updated

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: pad status text before wrapping with ANSI codes in `trace list`

`padEnd(10)` was called on strings containing ANSI color codes,
causing the invisible escape characters to be counted toward the
padding length. This broke column alignment for non-"ok" statuses.

Fix: pad the visible text first, then wrap with color codes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: kulvirgit <kulvirgithub@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@anandgupta42 anandgupta42 deleted the feat/tracing branch March 17, 2026 00:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: local-first tracing system to replace Langfuse

2 participants