RFC: drivers syntax for metric trees — quantitative relationships between measures

[rsyi]

Proposal: drivers annotation on measures

The semantic layer today encodes what metrics are (dimensions, measures, expressions) but not how they relate. This RFC proposes a drivers field on measures that declares quantitative and qualitative relationships, enabling automated sensitivity analysis, impact prediction, root-cause explanation, and opportunity sizing.

The syntax

measures:
  - name: revenue
    type: number
    expr: "{{orders.total_commission}}"
    drivers:
      # Linear: each additional order adds ~$12 revenue
      - measure: orders.total_orders
        coefficient: 12.0
        form: linear
        description: "Each incremental order adds ~$12 at current AOV and take rate"

      # Log-log (elasticity): 1% increase in AOV → ~0.9% increase in revenue
      - measure: orders.aov
        coefficient: 0.9
        form: log-log
        description: "AOV elasticity — higher baskets grow revenue with slight take rate compression"

      # Log-linear: each 1pp increase in take rate → ~8% increase in revenue
      - measure: orders.take_rate
        coefficient: 0.08
        form: log-linear
        description: "Take rate is a multiplicative lever on the GMV base"

      # Qualitative: no coefficient, just direction/strength
      - measure: sellers.active_listings_total
        direction: positive
        strength: moderate
        confidence: medium
        description: "More listings improve selection, driving buyer conversion"

      # Linear-log: doubling traffic → ~150 additional orders (diminishing returns)
      - measure: traffic.sessions
        coefficient: 150.0
        form: linear-log
        lag: 7
        description: "Traffic drives orders with diminishing returns"

Two types of edges in the metric tree

1. Component edges (implicit) — extracted automatically from type: number expressions containing {{view.measure}} references. These represent mathematical identity (e.g., aov = gmv / orders). Edge signs (+/-) inferred from expression context.

2. Driver edges (explicit) — declared via drivers. These represent correlative or causal business relationships.

Quantitative forms

Form	Relationship	Use case
linear	Δy = c × Δx	Volume drivers (orders → revenue)
log-log	%Δy = c × %Δx (elasticity)	Price/rate sensitivity (AOV → revenue)
log-linear	%Δy = c × Δx	Rate levers (take rate → revenue)
linear-log	Δy = c × ln(1 + Δx/x)	Diminishing returns (traffic → orders)
(qualitative)	direction + strength + confidence	When you know the direction but not the magnitude

Operations enabled by this

• sensitivity <measure> — rank all drivers by influence magnitude
• predict --if measure=delta [--if ...] — propagate hypothetical changes through the tree
• explain <measure> --time <dim> --current start:end --previous start:end — recursive root-cause analysis decomposing a metric change into (component, segment) pairs
• opportunity <measure> --time <dim> --period start:end — find underperforming segments, size the upside, propagate through drivers

Worked examples

Prototype branch: feature/metric-tree (PR #24)

Three examples demonstrating different use cases:

• SaaS revenue (examples/metric-tree/) — ARR → MRR → churn/expansion/new, with marketing drivers
• E-commerce marketplace (examples/metric-tree-ecommerce/) — orders × AOV × take rate → revenue, cross-view drivers between orders/sellers/traffic, all four quantitative forms
• Airbnb funnel (examples/metric-tree-funnel/) — host onboarding funnel (HLP → LYS → Amenities → Address → ... → Active Listing), opportunity sizing finding that Android+India has 0% address conversion

Interactive visualization

airlayer visualize generates a standalone HTML file with a force-directed graph of the full metric tree — no external dependencies, works offline.

airlayer visualize                        # full tree
airlayer visualize --root revenue.arr     # subtree rooted at a specific measure
airlayer visualize --output my-tree.html  # custom output path

The visualization encodes relationship types visually:

• Node shapes: Root/North Star (gold border), composite metrics (blue border), atomic metrics (gray)
• Edge types: solid black = component (math identity), solid green = positive driver, solid red = negative driver
• Confidence: dashed lines = low confidence drivers
• Click a node to see its detail panel — view, type, derivation expression, and all inputs (component + driver) with descriptions and reference links
• Double-click to focus on a subgraph (only connected nodes), click again to unfocus
• Drag, pan, zoom to explore large trees

The detail panel (shown on the right in the screenshot) surfaces the drivers metadata — direction, strength, descriptions, and reference URLs — making the metric tree explorable by anyone on the team, not just whoever wrote the YAML.

inspect --metric-tree provides the same information as CLI text output:

airlayer inspect --metric-tree              # full tree
airlayer inspect --metric-tree revenue.arr  # subtree
airlayer inspect --metric-tree --json       # machine-readable

Explain: Adtributor-style root-cause analysis

airlayer explain decomposes a metric change into the smallest (component, segment) pairs that explain it. The algorithm uses two types of splits:

1. Component splits — follow mathematical identities (e.g., revenue = commission). Exact decomposition.
2. Dimension splits — find which segment (region, platform, cohort) changed the most unexpectedly.

Dimension ranking uses surprise, not raw delta. Instead of "which segment had the biggest change?", explain asks "which segment's share of the metric shifted the most?" This is based on the Adtributor algorithm (Bhagwan et al., NSDI 2014):

• For each dimension, compute prior (previous period share) and posterior (current period share) per element
• Per-element surprise = Jensen-Shannon divergence contribution: S = 0.5 × (p × log(2p/(p+q)) + q × log(2q/(p+q)))
• Dimension surprise = sum of surprises from elements with |EP| ≥ 5%
• The most surprising dimension wins; within it, the element with the highest explanatory power is recursed into

This means a segment that represents 5% of volume but drops 80% will be flagged over a segment representing 50% of volume that drops 30% proportionally — because the small segment's behavior is more unexpected given its historical share.

Driver attribution is included when the target measure has declared drivers. After the component decomposition, explain queries each driver's change across periods and estimates its impact using the declared coefficient and functional form:

airlayer explain funnel.new_active_listings \
  --time funnel.event_date \
  --current 2024-02-01:2024-02-29 \
  --previous 2024-01-01:2024-01-31

Output shows both the component decomposition (what changed) and driver attribution (why it matters):

  funnel.new_active_listings 22 → 10 -12 (-54.5%)

  └── funnel.active_listings  -12  100% of total
      ├── country=India  -7  58% of total
      ...

  Driver attribution
  ├── funnel.amenities_to_address_rate  0.81 → 0.68  -0.13  estimated impact: -3.5 (log-log, coeff: 1)
  ├── funnel.hlp_to_lys_rate  0.63 → 0.54  -0.09  estimated impact: -2.9 (log-log, coeff: 0.95)
  └── funnel.hlp_visitors  187 → 187  +0  no impact

Open questions

1. Where should drivers live? Currently inline on measures via a drivers field. Alternatives:

   • Separate .driver.yml files
   • A top-level relationships section in the view
   • On the target measure rather than the source

2. Are four quantitative forms the right set? Should we add exponential, polynomial, or custom expressions?

3. Static vs dynamic coefficients? Current approach is explicit YAML declaration. Future work could auto-fit coefficients from historical data.

4. Cross-view drivers: Drivers can reference measures in other views (e.g., orders.revenue driven by sellers.active_listings_total). This is powerful but creates implicit coupling. Should we require explicit joins or keep it flexible?

5. Lag semantics: The lag field (in days) declares a delayed effect. Should this affect query time ranges automatically, or is it purely documentation?

Looking for feedback on the syntax design and which operations are most valuable.

[rsyi]

Feedback from @paullevchuk (here: https://www.linkedin.com/feed/update/urn:li:activity:7449468907913912320/?dashCommentUrn=urn%3Ali%3Afsd_comment%3A%287449497096832925696%2Curn%3Ali%3Aactivity%3A7449468907913912320%29):

Link to his medium post: https://medium.com/@paul.levchuk/the-metric-tree-trap-4280405fd35e

[rsyi]

My response here: https://www.linkedin.com/feed/update/urn:li:activity:7449468907913912320/?dashCommentUrn=urn%3Ali%3Afsd_comment%3A%287449497096832925696%2Curn%3Ali%3Aactivity%3A7449468907913912320%29&dashReplyUrn=urn%3Ali%3Afsd_comment%3A%287449559214169145345%2Curn%3Ali%3Aactivity%3A7449468907913912320%29

Paul Levchuk This was a fantastic read, thanks for sharing.

A couple of comments, would love to hear your thoughts:

• Re: 1 & 3: to address the analytical/definitional mix, perhaps it'd make more sense to have analytical/probabilistic drivers as independent files? E.g. .view.yml as semantic definitions, .driver.yml as analytical correlations.

• Re: 2: The way I've gotten around this is by defining the algorithms that sit on top so they look at dimensions and driver cuts simultaneously. Of course, dealing with non-deterministic drivers can make this difficult esp with heterogeneity, but I think it's a valid null hypothesis to assume homogeneity to start.

E.g. below is an image of what a root cause analysis might look like, traversing programmatically down the tree through dimensions + drivers, essentially just searching for the splits by contribution (using a KL-divergence-like calculation to just compare this week to the previous week)

[rsyi]

Paul's response: https://www.linkedin.com/feed/update/urn:li:activity:7449468907913912320/?dashCommentUrn=urn%3Ali%3Afsd_comment%3A%287449497096832925696%2Curn%3Ali%3Aactivity%3A7449468907913912320%29&dashReplyUrn=urn%3Ali%3Afsd_comment%3A%287449564392649793537%2Curn%3Ali%3Aactivity%3A7449468907913912320%29

Robert Yi
1/ On view.yml vs driver.yml: separating them is cleaner, but the hard problem remains: drivers have different factors, timeframes, and mechanisms than the metric definitions they'd attach to. A driver that matters for monthly cohort analysis may be irrelevant for quarterly board reporting of the same metric. It's not just a file organization problem.
2/ On the screenshot: I think this actually illustrates the concern. "EMEA enterprise churned_mrr explains 99%" is a WHERE answer, not a WHY answer. This is descriptive decomposition, essentially a programmatic pivot table. The hard part that metric trees promise but don't deliver is: why did those customers churn, was it preventable, and what should we prioritize? That requires feature importance, counterfactual reasoning, and cost-benefit: none of which live in the tree structure.
3/ "Assume homogeneity to start" is also risky, heterogeneity across segments is usually exactly where actionable insight hides.

[rsyi]

My response: https://www.linkedin.com/feed/update/urn:li:activity:7449468907913912320/?dashCommentUrn=urn%3Ali%3Afsd_comment%3A%287449497096832925696%2Curn%3Ali%3Aactivity%3A7449468907913912320%29&dashReplyUrn=urn%3Ali%3Afsd_comment%3A%287449566955797127168%2Curn%3Ali%3Aactivity%3A7449468907913912320%29

Paul Levchuk Great points. It's funny - I said exactly the same things when Joseph Moon first proposed the idea of deterministic RCA/opportunity sizing/etc. about a year ago to me. Whenever I personally had to do this kind of work, it ended up being 10% of what amounted to metric tree traversal, and 90% of careful thought around causality / scraping through slack to figure out if anything happened recently.

Perhaps a satisfactory implementation here to start should be spare, accepting that the full job can't yet be automated, but at least the more redundant aspects can (building the "programmatic pivot table" as you call it).

Aside: I imagine this may be more useful in that regard for opportunity sizing vs. RCA. RCA requires knowledge of what happened, but the issue is that that may just be an engineering accidentally a bug to prod or an exec deciding to turn off the landing page. But opportunity sizing can largely be done programmatically, without such knowledge

[rsyi]

Algorithm Reference: Explain, Opportunity, Sensitivity, Predict

This comment provides a precise description of each algorithm as implemented in PR #24.

---

1. Explain (Root-Cause Analysis)

Fast Mode (default): Greedy Adtributor Search

The fast mode is a greedy recursive algorithm based on the Adtributor paper (Bhagwan et al., NSDI 2014). At each level, it evaluates two types of splits and picks whichever has the highest concentration (signed fraction of the parent's delta explained by the child):

Component splits follow mathematical identities in the metric tree. For net_mrr = total_mrr + expansion_mrr + new_mrr - churned_mrr, the algorithm queries each child measure independently and computes how much of the parent's delta each child accounts for. Edge signs are inferred from the expression (e.g., churned_mrr is preceded by -, so its sign is -1). An increase in churned_mrr contributes to a decrease in net_mrr. The algorithm normalizes by total_attributed = Σ(child_delta × edge_sign) to strip scaling factors (e.g., the ×12 in arr = net_mrr * 12).

Dimension splits use Adtributor's surprise-based ranking. For each available dimension:

1. Query the measure broken down by dimension value, extracting per-element (previous, current) pairs
2. Compute per-element Explanatory Power (EP): EP_i = delta_i / delta_total
3. Compute per-element JSD surprise: for shares p_i = prev_i/prev_total and q_i = curr_i/curr_total, the surprise is 0.5 × (p × ln(2p/(p+q)) + q × ln(2q/(p+q)))
4. Rank dimensions by total surprise (sum of per-element JSD contributions where |EP| ≥ 5%)
5. Within the winning dimension, rank elements by concentration (EP)

This means surprise selects which dimension to split on, while concentration selects which element within that dimension. A small segment with a large proportional shift beats a large segment with a modest shift, because its behavior is more unexpected given its historical share.

Recursion strategy:

• Top level: emit the single best split, recurse, accumulate coverage
• Each level: evaluate all component + dimension candidates, pick the type with the highest top-element concentration
• Siblings: non-recursed candidates shown for context (all components; top-5 dimension values)

Stopping criteria:

Parameter	Default	Purpose
coverage_threshold	80%	Stop adding top-level splits once this fraction of the root delta is explained
max_depth	10	Maximum recursion depth
min_concentration	5%	Stop when best child's concentration drops below this
min_root_fraction	0.5%	Stop when cascaded root fraction drops below this (prevents 0.8^N decay)
Positive concentration	—	Stop when no remaining candidate has positive concentration

Detection heuristics (always checked):

• Simpson's Paradox: all dimension elements move opposite to the aggregate (flags when every segment improved but the overall metric declined)
• Opposing Offsets: two component deltas that substantially cancel (masking ratio > 0.3)

Driver attribution: After the component decomposition, if the target has declared drivers, the algorithm queries each driver's change and estimates its impact using the declared coefficient and functional form (linear, log-log, log-linear, linear-log).

---

Deep Mode (--deep): Multi-Strategy Beam Search

Activated with --deep, this runs a beam search with 4 competing scoring strategies in parallel, producing ranked alternative explanations with statistical significance.

Phase 1: Decompose to searchable measures. Walk component edges from the target downward, collecting leaf measures (no component children) and intermediate composites that have their own dimensions. Each searchable measure carries a cumulative_sign (product of edge signs from root) used to compute leaf_share = (measure_delta × cumulative_sign) / target_delta.

Phase 2: Per-measure beam search. For each searchable measure with non-zero delta and available dimensions:

1. Seed the beam: evaluate all (dimension, strategy) combinations, keep top beam_width candidates (default 10)
2. Expand: at each depth, for each beam entry with remaining dimensions and root_fraction ≥ min_root_fraction:
   • Query the filtered measure to get the current delta
   • Evaluate all remaining dimensions with all 4 strategies
   • Emit the current path as a completed alternative
   • Add extensions to the next beam
3. Prune: deduplicate by (measure, filter_set), keep the highest root_fraction, truncate to beam_width
4. Terminate: when beam is empty or max_depth reached; remaining entries become completed paths

Each path's root_fraction is scaled by leaf_share so alternatives across different measures are comparable.

The 4 strategies (each picks one element per dimension per level):

#	Strategy	Element selection	Dimension ranking
1	max_concentration	Highest signed delta / parent_delta	By that element's concentration
2	topk_concentration	Highest `	delta / parent_delta
3	jsd_smoothed	Highest Laplace-smoothed JSD surprise (filtered by `	EP
4	iv_woe	Highest `	WOE

Strategies 1 and 2 diverge when opposing elements exist (some segments rising, others falling). Strategy 1 picks the largest same-direction contributor; Strategy 2 picks the largest absolute contributor. Strategies 3 and 4 diverge from 1/2 when share distributions shift in ways not captured by raw delta — e.g., a segment that was 1% of volume growing to 5% has high JSD/WOE but may have modest absolute delta.

Laplace smoothing (strategies 3 and 4): ε = 1 / (total_prev + total_curr), applied to all shares as (value + ε) / (total + ε × n). This handles zero-share elements without introducing NaN/Inf in log computations.

Adaptive EP threshold: 0.05 / √n (where n = number of elements). Prevents high-cardinality dimensions from filtering out all elements in uniform degradation scenarios.

Uniform degradation detection: When no single element has |EP| ≥ 5% but collectively they explain >50% of the parent delta, emit a UniformDegradation split instead of drilling into individual elements.

Phase 3: Cross-cutting detection. After all per-measure searches complete, scan all paths for the same (dimension_name, value) appearing across multiple measures from different views. E.g., ads.region=EMEA and subs.region=EMEA both explaining drops → emit a CrossCutting annotation with combined root_fraction.

Phase 4: Rank and truncate. Sort all paths by root_fraction descending, deduplicate by terminal (measure, filter_set), keep top max_alternatives (default 5).

Phase 5: Statistical significance. For each top-K path:

1. Query 12 months of monthly data before the previous period for the terminal (measure, filters) combination
2. Compute period-over-period deltas from the monthly values
3. Run a two-tailed t-test: t = (current_delta - mean) / (std / √n), p-value from Student's t-distribution with n-1 degrees of freedom
4. Requires ≥6 historical periods; returns None otherwise
5. Deduplication: historical queries are cached by (measure, filter_set) key so paths sharing terminal nodes don't trigger redundant queries

---

2. Opportunity Sizing

Answers: "Where should we focus to grow this metric?"

Input: target measure, time dimension, period range.

Step 1: Query overall value. Execute the target measure aggregated over the period (no dimension breakdown).

Step 2: Discover dimensions. Find all non-time dimensions (string, number, boolean) in the target's view.

Step 3: Per-dimension gap analysis. For each dimension:

1. Query the measure broken down by dimension value over the period
2. Compute the benchmark:
   • Additive measures (count, sum, count_distinct, avg, min, max): benchmark = overall_value / n_segments (fair share)
   • Ratio measures (type: number): benchmark = overall_value (the true weighted average)
3. For each segment below the benchmark:
   • gap = benchmark - current_value
   • segment_share = 1 / n_segments
   • weighted_gap = gap × segment_share
4. Sum weighted gaps → total_weighted_gap for the dimension
5. Sort dimensions by total_weighted_gap descending

Step 4: Downstream propagation. Take the top dimension's total_weighted_gap and propagate it through the metric tree via predict(), showing the estimated impact on upstream metrics.

Output: per-dimension opportunities (sorted by potential), per-segment detail within each, and downstream impacts.

---

3. Sensitivity Analysis

Answers: "What influences this metric?"

Algorithm: Backward BFS from the target through the metric tree.

1. Build reverse adjacency: target_measure → [edges pointing to it]
2. Seed BFS with all direct inputs (component and driver edges)
3. For each edge traversed:
   • Effective coefficient = product of edge coefficients along the path (chain rule). Component edges have coefficient 1.0. If any edge in the chain is qualitative (no coefficient), the effective coefficient is None.
   • Direction: inferred from coefficient sign (positive coefficient → positive direction, negative → negative). Falls back to declared direction for qualitative edges.
   • Strength: inferred from |coefficient| (≥0.5 → strong, ≥0.1 → moderate, else weak). Falls back to declared strength.
4. Collect all reachable drivers with their paths, effective coefficients, forms, lags, and descriptions
5. Sort: quantitative drivers first (by |effective_coefficient| descending), then qualitative (by strength rank)

Result: an ordered list of all direct and transitive drivers of the target, each with its influence path and magnitude.

---

4. Predict (What-If Analysis)

Answers: "What would happen if this metric changed by X?"

Algorithm: Forward BFS from each input change through the metric tree.

1. For each input (measure, delta):
   • Follow outgoing edges (component and driver) to parent measures
   • Component edges: pass delta through exactly (confidence: "exact")
   • Driver edges with coefficient: output_delta = coefficient × input_delta (linear approximation for all forms; confidence: "estimated")
   • Driver edges without coefficient: output_delta = 0 (qualitative-only, can't quantify)
   • Continue propagating upward through the tree
2. Diamond accumulation: each input gets its own visited set, so the same target reached from two different inputs accumulates both impacts. Impacts at the same node from multiple paths are summed.
3. Confidence tracking: if any edge in the path is "estimated", the entire path is "estimated". Only pure component-edge paths are "exact".
4. Lag accumulation: lags from edges along the path are summed (total days before effect manifests)
5. Collapse: one impact per target measure (sum of deltas from all paths), sorted by |estimated_delta| descending

Output: list of all impacted measures with estimated deltas, confidence levels, propagation paths, forms, and cumulative lags.

---

Implementation Stats

• 304 tests total (76 for metric tree operations alone)
• 52 explain tests covering: pathological cases (hidden multi-dimension drops, JSD/IV-overblown signals), uniform degradation, Simpson's paradox, opposing offsets, cross-cutting patterns, significance testing, beam search convergence
• 8 sensitivity tests: coefficient chains, mixed quant/qualitative, diamond graphs, zero coefficients
• 7 predict tests: forward propagation, diamond accumulation, zero-coefficient pruning, confidence tracking, deep chains
• 10 opportunity tests: additive/ratio benchmarks, downstream propagation, gap sizing, edge cases