While implementing #58 we landed on a precedence rule of:
runtime kwarg > stage.variables > outer query.variables > model.query_variables
with the runtime kwarg threaded from the outermost engine.execute(...) call winning at every nesting level. This was the simplest rule that satisfied the immediate requirements, but several edge cases warrant a closer look:
-
Lockable stage variables. A stage author may want a variable baked into the saved query that no caller can override (e.g., a tenant scope, a hard cutoff date). The current regime lets the runtime kwarg trump anything; there's no way to express "this stage variable is locked."
-
Defaults vs stage values. If model.query_variables has a default and the same key appears in a stage's variables, the stage wins. This is the right call for "stage-specific override," but it means a model-level default cannot bind a stage that already specifies the variable. Worth confirming this is desired across all use cases.
-
Typed / parameterised variables. Today variables are literal-substituted into SQL strings ({var} → value). This conflates type-coercion with substitution and forces save-time placeholder hacks (literal '0' for dry-run). A typed variable system with bind parameters would be cleaner but bigger.
-
Save vs execute behaviour for unresolved variables. Save substitutes literal '0' and proceeds; execute raises a clear error. This asymmetry means a model can save successfully and fail at first run for variables that don't accept '0' (e.g., date columns). Worth deciding whether save should also raise for type-incompatible placeholders, or whether execute should adopt the same forgiving substitution.
-
Unknown-kwarg handling. Kwargs not referenced anywhere are silently ignored. We may want a strict mode that warns or errors, especially for orchestrators where a typo is more costly than a missing var.
No urgency — the regime in #58 is workable. Filing for visibility.
While implementing #58 we landed on a precedence rule of:
with the runtime kwarg threaded from the outermost
engine.execute(...)call winning at every nesting level. This was the simplest rule that satisfied the immediate requirements, but several edge cases warrant a closer look:Lockable stage variables. A stage author may want a variable baked into the saved query that no caller can override (e.g., a tenant scope, a hard cutoff date). The current regime lets the runtime kwarg trump anything; there's no way to express "this stage variable is locked."
Defaults vs stage values. If
model.query_variableshas a default and the same key appears in a stage'svariables, the stage wins. This is the right call for "stage-specific override," but it means a model-level default cannot bind a stage that already specifies the variable. Worth confirming this is desired across all use cases.Typed / parameterised variables. Today variables are literal-substituted into SQL strings (
{var}→ value). This conflates type-coercion with substitution and forces save-time placeholder hacks (literal'0'for dry-run). A typed variable system with bind parameters would be cleaner but bigger.Save vs execute behaviour for unresolved variables. Save substitutes literal
'0'and proceeds; execute raises a clear error. This asymmetry means a model can save successfully and fail at first run for variables that don't accept'0'(e.g., date columns). Worth deciding whether save should also raise for type-incompatible placeholders, or whether execute should adopt the same forgiving substitution.Unknown-kwarg handling. Kwargs not referenced anywhere are silently ignored. We may want a strict mode that warns or errors, especially for orchestrators where a typo is more costly than a missing var.
No urgency — the regime in #58 is workable. Filing for visibility.