Integrate Droughty functionality into the Wire Framework

[markrittman]

Why we are considering this

Rittman Analytics currently maintains two separate delivery tools that serve overlapping but distinct purposes: Wire Framework and Droughty. Both are used on client engagements but they have their own workflows (SoP), require different installation and configuration steps and produce outputs that do not automatically connect to one another. A consultant working on an engagement where both tools are relevant must context-switch between them manually, copy outputs from one into the other by hand, and maintain two sets of configuration files.

This situation has emerged organically as Wire was envisaged as a way to define agentic workflows for the delivery lifecycle from requirements through to dashboards, while Droughty was built specifically to automate the creation and QA of warehouses already built using dbt. While each tool does its job well, the proposal is that combining them into a single, coherent framework would make consultants meaningfully more effective and the benefits from doing this outweigh the cost and complexity of the development work required along with staff retraining.

The impetus for this review is the stated goal of the Wire Framework: a single place to go, a single codebase, and a standardised way to deliver every type of project RA works on. Droughty addresses a class of engagement — existing warehouse migration, extension, or audit — that Wire does not currently handle well. If Wire is to be the single delivery tool, it needs to either incorporate or coherently reference Droughty's capabilities.

Potential benefits of combining the two tools

1. One tool, one workflow, one place to go.
Consultants would open one tool (Claude Code with the Wire plugin), run one set of commands, and produce all engagement artifacts in one place — regardless of whether the project is a greenfield build, an existing warehouse migration, or a discovery sprint. No separate Droughty installation, no separate configuration, no context-switching.

2. Droughty outputs flow automatically into Wire's artifact structure.
Today, a consultant running droughty dbml gets a DBML file in a lookml/ directory that has no relationship to Wire's .wire/releases/<release>/artifacts/ structure. If Wire manages the call to Droughty, the ERD, quality report, and field documentation land in the right place, carry the right metadata, and are available to Wire's AI when generating subsequent artifacts — the problem definition can reference the ERD, the requirements artifact can embed the schema inventory, the LookML generation can be seeded from Droughty's base views.

3. Schema evidence available to Wire AI throughout the lifecycle.
When Wire generates a problem definition or requirements document today, the AI works from whatever documents the consultant has provided. On engagements with an existing warehouse, the most authoritative source of truth is the schema itself — table names, column types, row counts, null rates, PK coverage. If Wire can access this evidence by running Droughty at the right lifecycle points, every downstream artifact is grounded in the actual data landscape, not just what the SoW describes.

4. Validate/review lifecycle applied to schema-driven artifacts.
Droughty generates LookML, dbt tests, and staging models, but has no concept of validation or stakeholder review. If these artifacts are produced inside Wire, they automatically inherit Wire's validate/review/status tracking lifecycle — the same quality gates applied to every other deliverable. Droughty-generated LookML goes through /wire:semantic_layer-validate before being presented for review; Droughty-generated tests go through /wire:dbt-validate. This means clients get consistently quality-assured deliverables regardless of how the artifact was generated.

5. Standardised delivery methodology across project types.
Wire encodes RA's delivery methodology as executable specifications. Droughty solves a technical code-generation problem but carries no delivery methodology — it does not know about stakeholder reviews, sprint planning, or sign-off. Bringing Droughty inside Wire means the methodology applies uniformly: schema-introspection engagements follow the same lifecycle rigour as designed-platform engagements.

6. Reduced cognitive load and onboarding overhead.
A new RA consultant needs to understand one framework, not two. Documentation, training, and QA all converge on a single system. The decision of when to use Wire vs Droughty is eliminated — Wire decides internally, based on engagement type and warehouse availability.

7. Jira, Linear, and document store integration for schema-driven work.
Wire's integrations with Jira, Linear, Confluence, and Notion currently apply only to Wire-managed artifacts. Droughty-generated outputs exist outside these systems. Bringing Droughty inside Wire means schema introspection tasks, ERD reviews, and data quality sign-offs can be tracked in the same project management and document tools as the rest of the engagement.

---

Background

Wire Framework is a top-down, document-driven delivery system. It starts from a Statement of Work and stakeholder interviews, works through requirements and design, then uses AI to generate code artifacts — dbt models, LookML, dashboards. The workflow assumes the data platform is being designed and built.

Droughty is a bottom-up, schema-driven toolkit. It connects to a live BigQuery or Snowflake warehouse, reads the actual table and column metadata, and generates artifacts from what exists — LookML views, dbt tests, DBML entity-relationship diagrams, staging model SQL, AI-generated field documentation, and a LangChain/LangGraph data quality agent.

Before mapping integration options, it is worth being precise about where the two toolsets overlap and where they do not — because the answer is more nuanced than "both generate LookML."

---

Capability map: overlap, complement, and gap

LookML generation

Both Wire and Droughty generate LookML, but they are doing fundamentally different things.

Wire's LookML generation (/wire:semantic_layer-generate + LookML authoring skill) is semantic and designed. The AI works from requirements documents — it knows what business questions need answering, what dimensions and measures matter to the client, how tables should be joined to reflect business entities, and what derived fields and filters express the client's logic. The output is a semantic layer that encodes business meaning, not just schema structure. It requires iteration and human judgement.

Droughty's LookML generation (droughty lookml) is mechanical and schema-derived. It reads column names and types from the warehouse and translates them directly into dimensions and measures, applies naming convention heuristics (PK/FK suffixes, date fields), and generates explores from configured join relationships. The output is accurate to what exists but does not encode business logic. It is fast but produces a baseline that typically needs refinement.

These are complementary, not competing. Droughty produces the base layer from what exists; Wire refines it, adds business logic, and designs the explores. For a client with 80 existing warehouse tables, generating base views by hand is days of work that Droughty eliminates.

dbt test generation

Wire's dbt generation (/wire:dbt-generate) has the AI write schema.yml tests based on requirements context — it infers what should be tested from what it knows about the data model design. Tests are semantically appropriate but may miss coverage gaps in the actual schema.

Droughty's dbt generation (droughty dbt) reads the live schema and applies deterministic rules: columns named with _pk suffix get uniqueness + not-null tests; date/datetime columns get valid-range tests; columns with high cardinality get accepted-values tests. Coverage is guaranteed across every column in every table, but it is pattern-based rather than semantically informed.

Again these are complementary: Droughty provides exhaustive coverage via pattern rules; Wire adds semantically meaningful tests for business logic.

Testing, QA, and AI capabilities

This is where Droughty has no Wire equivalent at all:

• droughty qa — a LangChain/LangGraph agent that runs data quality checks against the live warehouse and produces a structured quality report. Wire has no equivalent.
• droughty docs — uses OpenAI to generate intelligent field descriptions for every dimension and measure, drawing on column names, types, and surrounding context. Wire has no field documentation generation command.
• droughty dbml — generates a DBML entity-relationship diagram from the live schema. Wire produces no visual data model documentation.

These three capabilities are the most distinctive value Droughty brings, and the most immediately useful in client-facing delivery — field documentation and ERDs are deliverables clients expect, and data quality validation is a standard part of every discovery sprint.

Summary

Capability	Wire	Droughty	Relationship
LookML views (base, from schema)	—	✓	Droughty generates base; Wire extends
LookML explores + business logic	✓	Partial	Wire primary; Droughty seeds joins
LookML Period-over-Period	—	✓	Droughty only
dbt models (staging, integration, warehouse)	✓	Staging only	Wire primary for model design
dbt schema.yml tests (pattern-based)	—	✓	Droughty only
dbt schema.yml tests (semantic)	✓	—	Wire only
Staging layer SQL + sources.yml	—	✓	Droughty only
Field documentation (AI)	—	✓	Droughty only
Data quality agent	—	✓	Droughty only
DBML / ERD	—	✓	Droughty only
Requirements from documents	✓	—	Wire only
Conceptual + data model design	✓	—	Wire only
Dashboard generation	✓	—	Wire only
Delivery lifecycle (validate/review/status)	✓	—	Wire only

---

The three engagement contexts

Context 1 — Greenfield platform build (RA's typical engagement)

No pre-existing analytics warehouse. RA designs and builds the data platform from scratch: requirements, data model, dbt transformation layer, semantic layer, dashboards. This is the dominant engagement type at RA.

The critical clarification for Droughty: Droughty does not require an externally-provided legacy schema. It requires a warehouse with tables to introspect — but in a greenfield engagement, dbt creates those tables. Once the dbt models have been built and deployed to BigQuery or Snowflake, Droughty can introspect the dbt-generated schema and produce LookML base views, pattern-based tests, field documentation, and a QA report against those tables. Droughty therefore sits inside the standard Wire delivery lifecycle — after dbt, before the semantic layer — not as an alternative to it.

This is the primary integration context. Droughty is not a special-case tool for legacy warehouses; it is a post-dbt acceleration step in RA's standard greenfield delivery workflow.

Context 2 — Existing or legacy warehouse

The client has pre-existing BigQuery or Snowflake tables — raw source data, partial transformations, possibly a legacy BI tool being replaced. Less common for RA, but this context also fits Droughty well: the schema exists regardless of how it was created, and Droughty can introspect it before dbt work begins to map the data landscape and seed the staging layer.

Context 3 — Discovery sprint on an existing landscape

The client's data landscape is undocumented or unknown in scope. The engagement is to map it, assess quality, and produce a platform recommendation. Droughty's ERD generation, data quality agent, and field documentation capabilities are directly useful here, producing evidence that feeds Wire's problem definition and pitch artifacts.

---

Integration options

Option 1 — Parallel command namespace

Add a self-contained /wire:droughty-* namespace. The existing Wire lifecycle is unchanged; consultants run Droughty commands when they have a live warehouse to work from.

Commands:

/wire:droughty-setup         — Generate profile.yaml + droughty_project.yaml from Wire context
/wire:droughty-introspect    — Schema inventory, row counts, PK/FK coverage report
/wire:droughty-dbml          — DBML entity-relationship diagram
/wire:droughty-docs          — AI field descriptions for all columns
/wire:droughty-qa            — LangChain/LangGraph data quality agent
/wire:droughty-stage         — Staging model SQL + sources.yml
/wire:droughty-dbt-tests     — schema.yml tests from schema patterns
/wire:droughty-lookml        — Base LookML views, explores, measures, Period-over-Period
/wire:droughty-generate      — All of the above in sequence

Workflow — Context 3 (discovery sprint):

/wire:new                              ← engagement_type: discovery
/wire:droughty-setup                   ← configure Droughty from context.md
/wire:droughty-dbml                    ← ERD of all schemas in scope
/wire:droughty-docs                    ← field descriptions for every column
/wire:droughty-qa                      ← data quality report
/wire:problem-definition-generate      ← AI synthesises from Droughty outputs + interviews
/wire:pitch-generate
/wire:sprint-plan-generate

Workflow — Context 1 (standard greenfield, Droughty after dbt deploy):

/wire:new                              ← engagement_type: full_platform
... requirements, data model, dbt phases as normal ...
                                       ← consultant runs: dbt run
/wire:droughty-setup
/wire:droughty-dbt-tests               ← augment schema.yml from deployed schema
/wire:droughty-lookml                  ← base LookML views from dbt-created tables
/wire:droughty-docs                    ← field descriptions
/wire:droughty-qa                      ← quality validation
/wire:semantic_layer-generate          ← Wire extends base views with business logic
/wire:semantic_layer-validate
/wire:semantic_layer-review

Pros:

• Existing Wire commands completely unchanged
• Clear mental model: Droughty = schema-driven artifacts, Wire = document/design-driven artifacts
• Straightforward to implement — thin subprocess wrappers
• Consultants have explicit control over when Droughty runs

Cons:

• Consultants must understand two parallel namespaces and when to blend them
• No enforced sequencing — a consultant could run droughty-lookml before requirements-generate and the outputs would not be connected
• Droughty-generated artifacts bypass Wire's validate/review lifecycle unless the consultant explicitly runs Wire's validation commands afterwards
• Requires Python/pip and Droughty installed in the engagement environment

Development effort: 9 command specs + 1 setup spec. S

---

Option 2 — Droughty enrichment within existing Wire commands

Droughty runs transparently inside existing Wire commands when a warehouse connection is configured. The consultant never invokes Droughty directly — Wire detects the connection and calls Droughty internally at appropriate lifecycle points.

Modified commands:

Command	What changes
/wire:problem-definition-generate	Calls droughty dbml and droughty qa if warehouse configured; embeds ERD and quality report in artifact
/wire:requirements-generate	Calls droughty introspect to seed data landscape section with actual table inventory and column counts
/wire:dbt-generate	Calls droughty dbt for existing tables; AI generates tests for new/designed tables; merges outputs
/wire:semantic_layer-generate	Calls droughty lookml for existing tables as base views; AI extends with business logic and writes explores
/wire:semantic_layer-validate	Calls droughty docs to validate field descriptions exist for all exposed dimensions

Workflow — Context 1 (standard greenfield, Droughty enrichment after dbt deploy):

/wire:new                              ← configure warehouse connection once
... requirements, data model, dbt phases as normal ...
                                       ← consultant runs: dbt run
/wire:dbt-generate                     ← internally: calls droughty dbt to augment tests
/wire:semantic_layer-generate          ← internally: calls droughty lookml for base views,
                                          droughty docs for field descriptions;
                                          AI extends with business logic
/wire:semantic_layer-validate
/wire:semantic_layer-review

Pros:

• Single workflow — no new commands to learn, no mental model split
• Droughty outputs are always integrated into Wire artifacts and go through validate/review
• Discovery sprint deliverables are automatically richer with no extra effort from the consultant

Cons:

• Significant complexity added to existing, working command specs
• Each command needs two execution paths (warehouse present / warehouse absent), doubling the test surface
• Failure modes are harder to diagnose — if droughty dbml fails silently, the problem definition artifact is incomplete with no obvious signal
• Reduces consultant visibility into what was generated how
• Risk of breaking existing greenfield workflows during implementation

Development effort: Modifications to 5 existing command specs, each with conditional logic and merged output handling. L — the day count is moderate but the regression risk in existing commands inflates the true cost.

---

Option 3 — Droughty as defined post-dbt steps in the standard delivery lifecycle

Rather than creating a new engagement type or a parallel command namespace, Droughty is incorporated as a set of explicit, named steps in the existing Wire delivery lifecycle — specifically positioned after dbt validation and before semantic layer generation. This reflects how Droughty actually works: it introspects a schema that dbt has already created, so it can only run once the dbt models have been built and deployed to the warehouse.

No new engagement type is needed. The standard full_platform, pipeline_only, and dbt_development engagement types gain a new optional phase: a Droughty phase between dbt and the semantic layer. Consultants can skip it if they prefer to write LookML from scratch; they run it when they want Droughty to generate the base layer from the deployed dbt schema.

New commands (inserted into existing delivery workflow):

/wire:droughty-setup <release>         — One-time config: generate profile.yaml + droughty_project.yaml
                                         from Wire context; pip install droughty if absent
/wire:droughty-lookml <release>        — Introspect dbt-created tables; generate base LookML views,
                                         explores, measures, and Period-over-Period where configured
/wire:droughty-dbt-tests <release>     — Augment existing schema.yml with pattern-based tests derived
                                         from the deployed table schema (PK, not-null, date ranges)
/wire:droughty-docs <release>          — Generate AI field descriptions for all deployed dimensions
/wire:droughty-qa <release>            — Run LangChain/LangGraph data quality agent against
                                         deployed tables; produce structured quality report

Where these sit in the standard delivery lifecycle:

/wire:requirements-generate <release>
/wire:data_model-generate <release>
/wire:data_model-validate <release>
/wire:data_model-review <release>

/wire:dbt-generate <release>
/wire:dbt-validate <release>
/wire:dbt-review <release>

                                       ← dbt models deployed to warehouse by consultant
                                          (dbt run / dbt Cloud job)

/wire:droughty-setup <release>         ← NEW: configure Droughty once per engagement
/wire:droughty-dbt-tests <release>     ← NEW: augment tests from deployed schema
/wire:droughty-lookml <release>        ← NEW: generate base LookML views from dbt tables
/wire:droughty-docs <release>          ← NEW: field descriptions for all exposed columns
/wire:droughty-qa <release>            ← NEW: data quality validation report

/wire:semantic_layer-generate <release>   ← Wire extends Droughty base views with explores,
                                             business logic, derived dimensions and measures
/wire:semantic_layer-validate <release>
/wire:semantic_layer-review <release>

Workflow — Context 1 (standard greenfield, with Droughty phase):

/wire:new                              ← engagement_type: full_platform (no change)
... requirements, data model, dbt phases as normal ...
                                       ← consultant runs: dbt run (deploys models to BigQuery/Snowflake)
/wire:droughty-setup 01-platform       ← configure warehouse connection
/wire:droughty-dbt-tests 01-platform   ← pattern-based tests added to schema.yml
/wire:droughty-lookml 01-platform      ← base views generated from dbt model tables
/wire:droughty-docs 01-platform        ← field descriptions for all dimensions
/wire:droughty-qa 01-platform          ← quality check against deployed data
/wire:semantic_layer-generate 01-platform  ← Wire adds explores, joins, business logic
/wire:semantic_layer-validate 01-platform
/wire:semantic_layer-review 01-platform

Workflow — Context 3 (discovery sprint — Droughty for evidence gathering):

For discovery engagements where a warehouse exists (Context 2 or 3), droughty-lookml and droughty-dbt-tests are less relevant (dbt hasn't run yet), but the evidence-gathering commands still apply:

/wire:new                              ← engagement_type: discovery
/wire:droughty-setup 01-discovery
/wire:droughty-qa 01-discovery         ← data quality report on existing schema
/wire:droughty-docs 01-discovery       ← field descriptions for existing tables
/wire:problem-definition-generate 01-discovery  ← AI works from QA report + docs as evidence
/wire:pitch-generate 01-discovery
/wire:sprint-plan-generate 01-discovery

Pros:

• Correct mental model — Droughty runs after dbt, not instead of it or before it
• No new engagement type — standard workflows gain an optional Droughty phase
• Existing Wire commands completely unchanged
• Droughty-generated LookML flows directly into /wire:semantic_layer-generate; the handoff is explicit and sequenced
• All Droughty-generated artifacts land in Wire's artifact structure and are subject to validate/review
• Jira/Linear sync and document store integration work without any additional plumbing
• Consultants can skip the Droughty phase entirely on any engagement without affecting anything else

Cons:

• Consultant must manually trigger dbt run between /wire:dbt-review and /wire:droughty-setup — Wire cannot do this for them; the deployment step is outside Wire's control
• Droughty-generated base LookML views and Wire-extended explores will coexist in the same LookML project — a clear file organisation convention is needed (e.g. views/generated/ for Droughty output, views/extended/ for Wire extensions)
• droughty-dbt-tests augments the existing schema.yml generated by /wire:dbt-generate — merge logic needs to be defined to avoid duplication or conflict

Development effort:

• 5 new command specs (droughty-setup, droughty-lookml, droughty-dbt-tests, droughty-docs, droughty-qa)
• Update existing delivery status templates to include optional Droughty phase steps
• Update CLAUDE.md / consultant handbook with new phase guidance
• M

---

Option 4 — Do nothing; keep them separate

Wire and Droughty remain independent tools. Consultants install and run Droughty standalone when they need schema-driven artifacts, and use Wire for the document-driven delivery lifecycle. No integration work is done.

Workflow — Context 3 (discovery sprint):

# In terminal, outside Wire:
pip install droughty
droughty dbml                          ← ERD generated to lookml/ directory
droughty qa                            ← quality report printed to terminal or file
droughty docs                          ← field descriptions written to configured path

# Separately, in Claude Code with Wire:
/wire:new
/wire:problem-definition-generate      ← consultant manually pastes/attaches Droughty outputs
/wire:pitch-generate
/wire:sprint-plan-generate

Workflow — Context 2 (extending existing warehouse):

# In terminal:
droughty lookml                        ← base views written to lookml/
droughty dbt                           ← schema.yml written to dbt project

# Separately, in Claude Code with Wire:
/wire:dbt-generate                     ← consultant tells Claude about existing Droughty-generated tests
/wire:semantic_layer-generate          ← consultant references existing Droughty views
/wire:semantic_layer-validate
/wire:semantic_layer-review

Pros:

• Zero development effort
• No risk to existing Wire commands or workflows
• Droughty continues to evolve independently on its own release cadence without Wire needing to track it
• Consultants who are comfortable with the CLI have full, direct control over Droughty's configuration and output
• No Wire–Droughty version compatibility concerns

Cons:

• Two tools, two installation steps, two configuration files (profile.yaml, droughty_project.yaml alongside Wire's .wire/ structure)
• Droughty outputs land in Droughty's own directories (lookml/, stage_files/, etc.) not in Wire's artifact structure — consultant must manually move or reference files
• No status tracking, validate/review gates, or Jira/Linear sync for Droughty-generated artifacts
• Context produced by Droughty (ERD, quality report, field docs) is not automatically available to Wire's AI when generating subsequent artifacts — consultant must manually copy and paste evidence
• Cognitive load on the consultant: two mental models, two workflows, no guidance on sequencing
• The goal of "one place to go" is explicitly not met

Development effort: None. XS

---

Recommended approach

Option 3 is the right architecture, and its effort is now M rather than XL because there is no new engagement type to define — just five new command specs that slot into an existing workflow at a well-defined point (after dbt deploy, before semantic layer).

Phase 1 — Ship the evidence-gathering commands first.

Start with the two commands that have no Wire equivalent and no dependency on dbt having run:

1. /wire:droughty-qa — data quality report against any accessible schema
2. /wire:droughty-docs — AI field descriptions for any accessible schema

These are immediately useful on discovery sprints (Context 3) and as a post-go-live audit step on greenfield engagements. Total effort: S. They validate whether Droughty integration is worth extending further.

Phase 2 — Add the post-dbt delivery commands.

Once Phase 1 is in use, add the commands that depend on dbt tables existing:

3. /wire:droughty-setup — warehouse connection config (prerequisite for all)
4. /wire:droughty-dbt-tests — pattern-based test augmentation
5. /wire:droughty-lookml — base LookML generation from dbt tables

These complete the Droughty phase in the standard delivery lifecycle and unlock the full value of the integration.

Option 2 (transparent enrichment) is not recommended — the conditional complexity it adds to existing specs creates regression risk not justified by the marginal UX benefit of fewer explicit commands.

Option 1 (parallel namespace) is a reasonable fallback if time is constrained, but it gives consultants no guidance on sequencing relative to the Wire lifecycle and allows Droughty to be run at the wrong point (e.g. before dbt has deployed).

Option 4 (do nothing) is a legitimate choice if the team does not currently encounter the pain points integration would solve — manually writing LookML base views from dbt tables, producing field documentation, or running data quality checks. The integration is only worth building if those tasks represent real time spent on current engagements.

---

Open questions

1. Droughty as pip dependency vs. vendored: Wire commands install Droughty at runtime (pip install droughty) or it can be pinned as a repo dependency. Runtime install is simpler but adds latency on first run; pinning gives version stability. Python >=3.9, <3.12.4 must be documented as a prerequisite.

2. Credential handling: Droughty's profile.yaml overlaps with Wire's existing BigQuery MCP credentials. The droughty-setup command (needed by all options) should check for Wire's MCP config first and offer to derive profile.yaml from it rather than prompting for credentials again.

3. LookML handoff convention: For Context 2 engagements, the consultant will have Droughty-generated base views and Wire-generated explores in the same LookML project. A naming convention is needed — e.g. Droughty produces view: stg_orders files in views/generated/, Wire extends them with view: +stg_orders refinement files in views/extended/.

4. The dbt deploy gap: Wire cannot trigger dbt run or a dbt Cloud job — that step sits outside Claude Code. The /wire:droughty-setup command needs to explicitly prompt the consultant to confirm that dbt models have been deployed before proceeding, and surface a clear error if Droughty cannot find the expected tables in the warehouse.

5. QA agent scope: Droughty's qa agent uses LangChain/LangGraph to query the live warehouse. This is powerful but requires an OpenAI key and may produce non-deterministic results. The schema-qa command should document its assumptions and the consultant should review the output critically, not treat it as an automated pass/fail gate.

[olivierdupuis]

The recommended approach does make sense to me. What I think will be important is to keep some sort of hygiene around the proliferation of commands. Better yet, reduce commands to a minimum and covert the rest to skills, so that Claude is the judge as to when to use specific skills based on the next step in a release's workflow.

[markrittman]

Thanks, and. that's a good point (re skills).