feat: builder prompt rewrite + dbt skills consolidation + altimate-dbt CLI

[suryaiyer95]

Summary

• Rewrite builder.txt from flat tool list to skills-first architecture with altimate_core offline SQL analysis tools surfaced
• Add altimate-dbt CLI (packages/dbt-tools/) — 16 commands wrapping @altimateai/dbt-integration for one-shot dbt operations
• Consolidate 8 dbt skills into 5 with progressive references/ system for lean AI routing
• Fix dbt-tools CLI error handling in columns, init, and main dispatch

Changes

Builder Prompt (packages/opencode/src/altimate/prompts/builder.txt)

• Skills-first architecture: organized tables for dbt (5), SQL (3), Training (3) skills
• Surface 6 altimate_core tools (validate, semantics, lint, column_lineage, correct, grade) — previously invisible to the agent
• Structured 5-phase workflow: Explore → Plan → Analyze → Execute → Validate
• Common Pitfalls section from benchmark failure analysis
• Removed verbose pre-execution protocol and FinOps tools

altimate-dbt CLI (packages/dbt-tools/)

• 16 commands: init, doctor, info, compile, build, run, test, execute, columns, graph, deps, etc.
• Config at ~/.altimate-code/dbt.json, auto-detected via altimate-dbt init
• JSON output to stdout, logs to stderr
• Python bridge fix for altimate_python_packages bundling

Skills Consolidation (.opencode/skills/)

• New: dbt-develop, dbt-test, dbt-troubleshoot, dbt-analyze, dbt-docs (5 focused skills)
• Deleted: dbt-cli, model-scaffold, generate-tests, yaml-config, incremental-logic, medallion-patterns, impact-analysis (7 merged)
• Each skill has lean SKILL.md + references/ directory for progressive context

Test plan

• Verify altimate-dbt init detects dbt project and creates config
• Verify altimate-dbt build --model <name> compiles and runs models
• Verify builder agent uses skills (dbt-develop, dbt-troubleshoot) when appropriate
• Verify altimate_core_* tools appear in agent's tool list
• Run Spider2-DBT benchmark to compare pass rate vs main

🤖 Generated with Claude Code

[sentry]

Bug: The children and parents commands are missing async/await, causing them to return a Promise instead of data, which results in empty {} output.
_{Severity: HIGH}

Suggested Fix

Add the async keyword to the children and parents function declarations in graph.ts. Then, add await before the adapter method calls (getChildrenModels and getParentModels) within those functions. Finally, add await to the calls for these commands in index.ts.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.

Location: packages/dbt-tools/src/index.ts#L175-L178

Potential issue: The `children` and `parents` commands are not defined as `async`
functions and do not `await` the results from their respective adapter methods
(`getChildrenModels` and `getParentModels`). All other commands and adapter interactions
in the codebase use `async`/`await`. This inconsistency will cause the commands to
return a Promise object instead of the resolved data. Consequently, when the result is
stringified to JSON for output, it will produce an empty object `{}` instead of the
expected model graph, and any errors during execution will lead to unhandled promise
rejections.

_{Did we get this right? 👍 / 👎 to inform future reviews.}

[sentry]

Bug: The children and parents commands are missing await, which can cause a race condition where the adapter is disposed before the async operation completes, leading to runtime errors.
_{Severity: HIGH}

Suggested Fix

Mark the children and parents functions in graph.ts as async. Then, add await to the calls to these functions in index.ts for the children and parents cases to ensure the operations complete before the adapter is disposed.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.

Location: packages/dbt-tools/src/index.ts#L174-L178

Potential issue: The `children` and `parents` commands in the main switch statement do
not `await` the result of their respective functions. These functions call
`adapter.getChildrenModels` and `adapter.getParentModels`, which likely return Promises,
based on the usage pattern of other adapter methods. The `finally` block disposes of the
`adapter` immediately after the command is dispatched. This creates a race condition
where the adapter can be disposed of before the asynchronous operation completes,
leading to a runtime error when the pending Promise attempts to access the disposed
adapter's resources.

[dev-punia-altimate]

🤖 Behavioral Analysis — 4 Finding(s)

🔴 Critical (1)

• F1 packages/dbt-tools/src/commands/graph.ts:17 [Missing async/await — Promise returned as data]
  Both children and parents are declared as plain synchronous functions but call async adapter methods without await. They return a Promise instead of resolved data. In index.ts the results are also not awaited (lines 4116-4121). Effect 1: result is a Promise object — JSON.stringify(Promise) produces {} — both commands output an empty object instead of the model graph. Effect 2: the finally { await adapter.dispose() } block fires immediately after the switch, disposing the adapter while the Promise is still pending. When the Promise eventually resolves it tries to use a closed Python bridge, causing a crash caught by the unhandledRejection handler. All other commands in the switch use result = await ...; this is the only place the pattern is broken. Note: flagged twice by sentry bot but not yet acknowledged by the author. — Auto-fixable: In graph.ts add async to both function declarations and await the adapter calls. In index.ts add await before both .children(adapter, rest) and .parents(adapter, rest) calls.

🟡 Warnings (1)

• F2 packages/opencode/src/tool/registry.ts:218 [DbtRunTool removed from registry despite description updated for fallback use]
  The PR simultaneously (a) updates dbt-run.ts description to 'Use this tool only as a fallback when altimate-dbt is unavailable' and (b) removes DbtRunTool from both the re-export in altimate/index.ts and the tool registry. The updated description implies the tool should remain available as a last resort, but removing it from the registry makes it completely inaccessible to the agent. The builder.txt prompt also contradicts the updated tool description: the prompt says 'Never call raw dbt directly' while the description says 'use when altimate-dbt is unavailable'. In environments where altimate-dbt has not been initialized (fresh installs, CI), the agent now has zero dbt execution path. The file dbt-run.ts still exists with an unreachable description, which will confuse future maintainers.

🔵 Nits (2)

• F3 packages/dbt-tools/src/commands/execute.ts:8 — Two edge cases silently fall through to unlimited execution: (1) --limit 0: raw is '0' (truthy string), parseInt returns 0, if (0) is falsy so unlimited SQL runs instead of returning empty; (2) --limit abc or when the next argv is another flag: parseInt returns NaN, if (NaN) is falsy so unlimited SQL runs with no warning. Neither path signals to the caller that the limit was ignored.

• F4 packages/dbt-tools/src/commands/columns.ts:36 — The values() function calls adapter.getColumnValues(model, col) without a try/catch. The sibling columns() and source() functions both wrap their adapter calls in try/catch and return structured { error: '...actionable message...' } objects. If getColumnValues throws, the error propagates to bail() which emits a generic 'Run: altimate-dbt doctor' message instead of the actionable context the other commands provide.

_{Analysis run | Powered by QA Autopilot}

[sentry]

Bug: parseInt on the --limit option can return NaN. The subsequent truthiness check if (limit) fails silently, ignoring the user's invalid input instead of raising an error.
_{Severity: MEDIUM}

Suggested Fix

After parsing the --limit value with parseInt, add a validation step using Number.isNaN() to check if the result is NaN. If it is, throw an error to inform the user that they have provided an invalid value for the limit. This aligns with the existing validation pattern in packages/opencode/src/session/retry.ts.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.

Location: packages/dbt-tools/src/commands/execute.ts#L8-L9

Potential issue: When a non-numeric value is passed to the `--limit` option, `parseInt`
returns `NaN`. The code only performs a truthiness check on the result (`if (limit)`).
Since `NaN` is falsy, the condition fails, and the code silently falls back to executing
the SQL without a limit via `immediatelyExecuteSQL`. This contradicts the user's intent
and happens without any error or warning. The expected behavior is to validate the
parsed number and inform the user if their input is invalid, a pattern already
established elsewhere in the codebase.

[sentry]

Bug: The format function incorrectly uses the presence of stderr content to detect errors, rather than checking the exit_code. This can cause successful commands to be reported as failures.
_{Severity: HIGH}

Suggested Fix

Modify the format function to determine success or failure based on the result.exit_code property. An exit_code of 0 indicates success. The presence of stderr should not be treated as a definitive error condition, as shown in the implementation for dbt-run.ts.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.

Location: packages/dbt-tools/src/commands/build.ts#L39-L41

Potential issue: The `format` function in `build.ts` and `deps.ts` incorrectly
determines if a dbt command failed. It checks if `result.stderr` has content, but dbt
often writes non-error information like warnings and progress logs to stderr. The
function should instead check `result.exit_code === 0` to determine success, which is
the reliable indicator. This incorrect logic will cause successful dbt operations (like
`build`, `test`, or `deps`) to be reported as failures if dbt writes anything to stderr,
leading to user confusion and unnecessary retries.

[Copilot]

Pull request overview

This PR introduces a new altimate-dbt CLI package (Bun/TS) for dbt operations and adds a Spider2-DBT benchmarking/evaluation harness, while updating Altimate prompts/tools to prefer altimate-dbt over raw dbt execution and patching python-bridge for runtime compatibility.

Changes:

• Add packages/dbt-tools (CLI + adapter wrapper around @altimateai/dbt-integration) with Bun tests and workspace wiring.
• Add experiments/spider2_dbt benchmark runner, evaluator, report generator, and SQLite tracker for results.
• Update Altimate prompts/tool exports/registry to de-emphasize or remove the dbt_run tool, and patch python-bridge@1.1.0.

Reviewed changes

Copilot reviewed 64 out of 66 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
patches/python-bridge@1.1.0.patch	Patch python-bridge child_process import to avoid promisify behavior.
packages/opencode/src/tool/registry.ts	Removes DbtRunTool import/registration from tool registry.
packages/opencode/src/altimate/tools/dbt-run.ts	Updates tool description to prefer altimate-dbt (tool now appears orphaned).
packages/opencode/src/altimate/prompts/builder.txt	Rewrites builder prompt to emphasize altimate-dbt workflow and principles.
packages/opencode/src/altimate/index.ts	Removes barrel export for dbt-run.
packages/dbt-tools/tsconfig.json	Adds strict TS config for the new dbt-tools package.
packages/dbt-tools/test/config.test.ts	Adds basic tests around config handling/JSON structure.
packages/dbt-tools/test/cli.test.ts	Adds CLI behavior tests via bun spawnSync.
packages/dbt-tools/src/index.ts	Implements CLI routing, formatting, error diagnosis, adapter lifecycle.
packages/dbt-tools/src/config.ts	Implements config file read/write under ~/.altimate-code/dbt.json.
packages/dbt-tools/src/commands/init.ts	Adds init command to discover project + Python and write config.
packages/dbt-tools/src/commands/info.ts	Adds info command wrapper.
packages/dbt-tools/src/commands/graph.ts	Adds children/parents DAG wrappers.
packages/dbt-tools/src/commands/execute.ts	Adds SQL execution wrapper with optional limit.
packages/dbt-tools/src/commands/doctor.ts	Adds doctor command to report prerequisite status.
packages/dbt-tools/src/commands/deps.ts	Adds deps installation and package add wrappers.
packages/dbt-tools/src/commands/compile.ts	Adds model/query compile wrappers.
packages/dbt-tools/src/commands/columns.ts	Adds model/source/values column inspection wrappers.
packages/dbt-tools/src/commands/build.ts	Adds build/run/test/project wrappers.
packages/dbt-tools/src/check.ts	Adds prerequisite checking and validation messaging.
packages/dbt-tools/src/adapter.ts	Creates and initializes DBTProjectIntegrationAdapter.
packages/dbt-tools/script/copy-python.ts	Copies bundled Python packages from @altimateai/dbt-integration into dist.
packages/dbt-tools/package.json	Defines new workspace package, bin entry, build/test scripts.
packages/dbt-tools/bin/altimate-dbt	Node bin shim that imports the built CLI entry.
package.json	Adds packages/dbt-tools workspace and patches python-bridge@1.1.0.
experiments/spider2_dbt/tracker.py	Adds SQLite-backed run/task tracking and comparison commands.
experiments/spider2_dbt/setup_spider2.py	Adds one-time setup (clone/download/extract/verify) for Spider2-DBT.
experiments/spider2_dbt/schema_introspect.py	Adds DuckDB schema summarization for prompt context.
experiments/spider2_dbt/run_benchmark.py	Adds benchmark runner with retries, parallelism, caching, and result aggregation.
experiments/spider2_dbt/requirements.txt	Adds Python deps for benchmark tooling.
experiments/spider2_dbt/report.py	Adds single-file HTML report generator for evaluation results.
experiments/spider2_dbt/prompt_template.py	Builds task prompts including YAML model discovery and DuckDB schema summary.
experiments/spider2_dbt/evaluate_results.py	Evaluates benchmark outputs via Spider2 duckdb_match and writes summary JSON.
experiments/spider2_dbt/config.py	Centralizes benchmark config (paths, defaults, leaderboard data).
experiments/spider2_dbt/altimate-code-dev.sh	Adds local dev wrapper script (currently hardcoded path).
experiments/spider2_dbt/.gitignore	Ignores cloned repo/workspace/results artifacts for experiments.
bun.lock	Updates lockfile for new workspace/package deps and patched deps.
.opencode/skills/yaml-config/SKILL.md	Removes legacy skill doc.
.opencode/skills/model-scaffold/SKILL.md	Removes legacy skill doc.
.opencode/skills/medallion-patterns/SKILL.md	Removes legacy skill doc.
.opencode/skills/incremental-logic/SKILL.md	Removes legacy skill doc.
.opencode/skills/impact-analysis/SKILL.md	Removes legacy skill doc.
.opencode/skills/generate-tests/SKILL.md	Removes legacy skill doc.
.opencode/skills/dbt-troubleshoot/SKILL.md	Adds new dbt-troubleshoot skill doc using altimate-dbt.
.opencode/skills/dbt-troubleshoot/references/test-failures.md	Adds troubleshooting reference for test failures.
.opencode/skills/dbt-troubleshoot/references/runtime-errors.md	Adds troubleshooting reference for runtime errors.
.opencode/skills/dbt-troubleshoot/references/compilation-errors.md	Adds troubleshooting reference for compilation errors.
.opencode/skills/dbt-troubleshoot/references/altimate-dbt-commands.md	Adds altimate-dbt command reference (troubleshoot skill).
.opencode/skills/dbt-test/SKILL.md	Adds new dbt-test skill doc using altimate-dbt.
.opencode/skills/dbt-test/references/unit-test-guide.md	Adds dbt unit testing guide.
.opencode/skills/dbt-test/references/schema-test-patterns.md	Adds schema-test patterns reference.
.opencode/skills/dbt-test/references/custom-tests.md	Adds custom test patterns reference.
.opencode/skills/dbt-test/references/altimate-dbt-commands.md	Adds altimate-dbt command reference (test skill).
.opencode/skills/dbt-docs/SKILL.md	Updates dbt docs skill to be altimate-dbt-driven + adds references.
.opencode/skills/dbt-docs/references/documentation-standards.md	Adds documentation standards reference.
.opencode/skills/dbt-docs/references/altimate-dbt-commands.md	Adds altimate-dbt command reference (docs skill).
.opencode/skills/dbt-develop/SKILL.md	Adds new dbt-develop skill doc using altimate-dbt.
.opencode/skills/dbt-develop/references/yaml-generation.md	Adds YAML generation reference.
.opencode/skills/dbt-develop/references/medallion-architecture.md	Adds medallion architecture reference.
.opencode/skills/dbt-develop/references/layer-patterns.md	Adds dbt layering patterns reference.
.opencode/skills/dbt-develop/references/incremental-strategies.md	Adds incremental strategies reference.
.opencode/skills/dbt-develop/references/common-mistakes.md	Adds common mistakes reference.
.opencode/skills/dbt-develop/references/altimate-dbt-commands.md	Adds altimate-dbt command reference (develop skill).
.opencode/skills/dbt-analyze/SKILL.md	Adds new dbt-analyze impact analysis skill doc.
.opencode/skills/dbt-analyze/references/lineage-interpretation.md	Adds lineage interpretation reference.
.opencode/skills/dbt-analyze/references/altimate-dbt-commands.md	Adds altimate-dbt command reference (analyze skill).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

[dev-punia-altimate]

✅ Tests — All Passed

TypeScript — passed

Python — passed

_{Tested at cb8efd79 | Run log | Powered by QA Autopilot}