[Proposal] Add Dep (Dependency) syntax to DBML for data lineage

[TeaNguyen]

Hi everyone — we’re exploring a DBML language extension to capture data lineage / transformation dependencies directly in DBML, alongside the existing ER relationships (Ref). We’d love feedback from the community before we formalize and implement it.

Motivation

DBML today is decent for schema modeling (tables, columns, refs). However, in many analytics / data engineering workflows (ELT/ETL, marts, semantic layers), we also need to describe:

• Column-level lineage (e.g. raw_orders.user_id → stg_orders.user_id)
• Table-level dependencies (e.g. stg_orders feeds fct_orders)
• Business meaning of a dependency edge (join key, aggregation, dedupe, normalization, etc.)

This information is usually stored elsewhere (SQL, docs, YAML, pipeline tools), which makes lineage hard to keep in sync with the canonical model.

We want DBML to optionally describe data dependencies without conflating them with foreign key relationships.

---

Goals

• Provide a first-class syntax to express data dependency edges between tables/fields.
• Support both table-level and column-level dependencies.
• Allow attaching a short note to explain semantics.
• Keep Dep separate from Ref (FK relationships) to avoid overloading meaning.

Non-goals (for the initial version)

• Executable transformation logic (SQL, expressions, macros)
• Full DAG/pipeline orchestration semantics (schedules, materializations, runtime)
• Guaranteeing correctness of lineage (it’s descriptive, not validated against SQL)
• Advanced attributes (tags, owners, severity, freshness, tests) — we can iterate later

---

Proposed syntax

The syntax will contains two parts:

• Lineage levels: table-level vs column-level
• Syntax forms: short, long, inline

Lineage levels

1) Table-level dependency

Declares a high-level DAG edge between entities:

Dep: upstream_table -> downstream_table

2) Column-level dependency

Declares a mapping from one output column to one or more input columns.

The simplest form is a single mapping:

Dep: source.col -> target.col

For multi-source transforms (many-to-one), you declare multiple mappings that share the same downstream table (typically in long form; see below).

Syntax forms

1) Short form

Best for quickly sketching table-level dependencies:

Dep: upstream_table -> downstream_table

// Optional settings (prototype)
Dep: raw_orders -> stg_orders [materialized: view, schema: staging, note: 'Raw → staging cleaning']

// Column-level single-mapping 
Dep: raw_users.id -> stg_users.user_id [note: 'Cast + rename']

2) Long form

Long form is useful when:

• A downstream entity depends on multiple upstream sources (multi-source / many-to-one)
• You want to define multiple column-level mappings together & attach edge metadata as body-level attributes (note, custom attributes, etc.)

// Multi-source, column-level lineage
Dep {
  stg_users.user_id -> dim_users.user_id
  stg_users.email -> dim_users.email
  stg_orders.ordered_at -> dim_users.first_order_at
  stg_orders.order_id -> dim_users.total_orders
  stg_orders.amount -> dim_users.total_spent

  note: 'Joins user profile with order aggregates.'
}

// Table-level lineage
Dep {
  raw_users -> stg_users
  note: 'Raw → staging cleaning'
  materialized: view
  schema: staging
}

3) Inline form (on tables/columns)

For teams that prefer keeping lineage near the schema definition, column-level lineage can be expressed inline on the target side (using <-):

Table stg_users {
  user_id int [pk, dep: <- raw_users.id]
  email varchar [dep: <- raw_users.email]
}

Or inline on the source side (using ->):

Table raw_users {
  id int [pk, dep: -> stg_users.user_id]
  email varchar [dep: -> stg_users.email]
}

Inline forms don't support note or custom attributes, similar to Ref.

Notes

Dep: raw_users.email -> stg_users.email [note: 'Lowercase + validate email']

// Multi-line note
Dep {
  raw_users -> stg_users
  note: '''
  Raw → staging cleaning
  '''
}

---

Compatibility & impact

• This is an additive language feature. It can stay in a new module of DBML called DBML Lineage.
• Existing DBML files continue to parse unchanged. Tools that don’t understand Dep can either ignore it (best-effort) or treat as unknown (depending on implementation choice; we’ll aim for graceful ignoring where possible).

---

Open questions for feedback

1. Naming
   • Should the keyword be Dep, Dependency, or something else?
2. Direction operator
   • We are currently sketching Dep with -> for data flow (upstream → downstream). What operator(s) would you prefer for Dep, and why? (e.g. stick with ->, or alternatives such as =>, ~>, etc.)
3. Custom / additional attributes
   • Rich lineage metadata (tags, materialization, owners, etc.) overlaps with a broader topic: user-defined custom attributes on DBML elements.
   • That direction is discussed separately here: DBML discussion #676.
   • For Dep in particular: is there anything you need on top of that thread for a first version, or should Dep stay minimal until custom attributes land?

---

💬 We’d love your thoughts — especially from folks using DBML for analytics engineering or maintaining large transformation graphs.

[mrMartinGrosse]

I think the direction of travel looks sound! dep with ->- looks like an easy way to use. It looks analogous to the use of ref.

could the operator be as simple as >though?

[TeaNguyen]

Thanks! Re operator: we deliberately avoided reusing > because in DBML today it strongly reads as FK / ER relationship (Ref), and we worry people will mentally interpret A > B as “A references B” rather than “data flows A → B”.

That said, I agree the “as simple as possible” instinct is good. If you (or others) feel > is still the best choice, could you share if mixing Ref and Dep in the same file would ever feel ambiguous to you?

We’ll pick the operator based on what’s most legible in real DBML files that contain both Ref and Dep.

[yachal-upson]

End of day here, but I'll try. I like the concept so long as it's additive.
I've been venting this week about the lack of comprehensive multi-layer systems tooling, again, and building out a pseudo dependency tree for a client.. adjacent to the ER data model. So coming from that, makes sense.

I think there's some non-trivial semantics to consider:
Are you describing the DAG flow (ie. Upstream -> Downstream)
Or a dependency edge/relation? "B Depends on A" (ie. B -[DEPEENDS_ON]-> A)

Consider coming from Data Ops, vs coming from systems mapping or network graph.

Reflection: I personally do like the char arrow syntax.
But I don't like the DAG framing - I think that says more about the hammer I'm holding right now.

[TeaNguyen]

Great call-out — and I think the answer is: it depends on how teams want to use it.

In this proposal, the arrow already encodes the dependency semantics: we treat A -> B as “B depends on A” (and, equivalently, “A is upstream of B / contributes to B”). So we’re not trying to introduce a second, competing interpretation—just making the direction explicit in the syntax.

On the “DAG” wording: we use DAG as an example mental model (a common way people visualize lineage), but we’re not framing everything as orchestration or runtime semantics. Dep is purely descriptive: users can model dependency edges at whatever granularity they need—table-level for coarse flow, or column-level (including multi-source) for detailed lineage—depending on their demands.

[chris2v]

Love this! A lot of time dependencies are strictly inferred from the FKs to be on the safe side, which isn't always necessary for processing order dependencies, especially in non-enforced FK environments, creating unnecessary delays.

If I could have my pick of syntax, I would use ">>" and "<<"; I feel it is a better indication of flow/movement. "upstream" >> "downstream". And if I really want to be pedantic, "->" is one modified key (requires shift) and one regular key (First world problems, I know 😄).

Are there any planned visualizations for dependencies? E.g., Dotted lines?
When exporting to PDF, will a separate page generate indicating the dependencies?

[TeaNguyen]

Thanks a lot — and +1 on the “FK-inferred deps” pain; that’s one of the motivations for keeping Dep separate from Ref.

• Operator: I like the >> / << idea — it reads very “flow / movement”. We’re still collecting feedback on the final operator, but this is a strong candidate.

• Planned visualization? Yes. If/when we visualize Dep, we’ll make it visually distinct from Ref (e.g. dotted/dashed lines, different color, or other styling) so users can immediately tell “lineage dependency” vs “ER relationship”.

• PDF export / separate page? Yes — the intention is to have a Dependencies-only view/page (and therefore a separate page in PDF export) so you can review lineage without the ER refs cluttering the diagram.

[rdillinger]

Naming
Should the keyword be Dep, Dependency, or something else? - DEP is easy enough, I would keep it syntactically simple.

Direction operator
We are currently sketching Dep with -> for data flow (upstream → downstream). What operator(s) would you prefer for Dep, and why? (e.g. stick with ->, or alternatives such as =>, ~>, etc.) - Simple Arrows are also fine, as would be << or >> mentioned above.

Custom / additional attributes

For Dep in particular: is there anything you need on top of that thread for a first version, or should Dep stay minimal until custom attributes land? I need to show the data flow, including any calculation or aggregation that occurred between a source table and a dependent table (the transformation and/or filters). In fact, I'm attempting to use DBDiagram.io for this exact purpose to keep a better programmatic version of lineage, and would use this type of capability immediately.

[numa1031k-sketch]

r

[moodgorning]

Naming: Dep is fine
Direction operator: I would vote for >>/<<, easier to type than ->/<-
Custom / additional attributes: Echoing @rdillinger's suggestions. Being able to add some information on what transformations are happening for this dependency would be a really nice addition. Maybe a link tag beyond the note to explicitly point to documentation? In the same way, if the transformation is running inside DBT or something similar, it would be cool to be able to point to the exact model or implementation file with an implementation ('impl' maybe) element.

[LenaBagh]

Naming

I am not sure Dep / Dependency is the best term. a) It raises the question what depends on what (and it is not what -> suggests). b) It is not necessarily about dependencies. It is often more about source/destination, i.e. about data flow.
Direction operator
-> is fine, however > or >> is better as it is easier to type. Either way, with the keyword describing the relationships I don't feel any of the options would be confusing.
Custom / additional attributes
I agree with @rdillinger : being able to document data transformation/filtering would be very helpful. The same functionality could be used for adding comments on table/column connectors on the diagram.
As a "nice to have" I would like to request functionality that allows highlighting all connectors and columns/tables that are upstream for the selected column/table. As a next step, exporting this selection as a script would be awesome.

My background is data integration. I tried using dbdiagram for data flow diagrams and mostly failed. Having dependency functionality would make a huge difference for me