fix(tesseract): reject FILTER_PARAMS string columns under a calendar time shift - #11772
fix(tesseract): reject FILTER_PARAMS string columns under a calendar time shift#11772paveltiunov wants to merge 16 commits into
Conversation
…e shifts filter_params_time_shift.rs covers an INTERVAL time shift and asserts the pushed-down FILTER_PARAMS column is offset by the same interval, so a shifted stage scans the rows it groups by. A NAMED calendar shift - the form a retail 4-5-4 calendar forces, where "one fiscal year back" is a mapping column rather than an interval - gets no such treatment. base_filter.rs resolves the shift as `time_shifts().get_for_symbol(sym).and_then(|s| s.interval.as_ref())`, and a named calendar shift carries `interval: None`, so `to_sql_for_filter_params` receives `None` and renders the column bare against the unshifted reporting bounds. The binding still matches in every stage (the shift substitutes the column only where the stage's own predicate renders), so the fact scan is never left unfiltered. But each stage joins the calendar on its mapping column while the pushed-down predicate restricts the same scan to the reporting period, so the stage is empty unless the model widens the band by hand - and once it does, every stage scans every band. These tests characterise the current behaviour so the gap is visible and a fix has something to invert. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwtxWoLUp7ULnsQJjVLkZt
… time shift The YAML-level counterpart to the planner test added in ebdc764, reproducing the shape a retail 4-5-4 calendar forces: a fact whose `sql` pushes a joined calendar cube's date down through FILTER_PARAMS, with multi_stage measures that shift that date by NAME rather than by interval. Three tests characterise what Tesseract emits today, and a fourth pins the interval form beside it so the asymmetry is visible in one file: - the column reaches every shifted stage (no stage scans the fact unfiltered) - each stage filters on its own calendar mapping column - an INTERVAL shift offsets the pushed-down column (CORE-543 / #11030) - a NAMED shift does not - it binds the bare column to unshifted bounds The last one is the gap. base_filter.rs resolves the shift as `time_shifts().get_for_symbol(sym).and_then(|s| s.interval.as_ref())`, and a named calendar shift carries `interval: None`, so the offsetting branch in `to_sql_for_filter_params` is never reached. Each stage then joins the calendar on its mapping column while the pushed-down predicate restricts the same scan to the reporting period, so the stage is empty unless the model widens the band by hand - and once it does, every stage scans every band. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwtxWoLUp7ULnsQJjVLkZt
…time shift
A multi_stage measure shifted by NAME (`time_shift: [{ name: prior_fiscal_year }]`,
resolved through a calendar cube's own `time_shift` declaration) used to render
its cube's FILTER_PARAMS push-down as a bare column bound to the query's
UNSHIFTED reporting bounds.
That contradicts the stage around it. The stage joins the calendar on its
mapping column and reads prior-period fact rows, while the pushed-down predicate
restricts the same scan to the reporting period, so the stage comes back empty -
the failure CORE-543 fixed for interval shifts, reached by a different route.
The cause was two levels down from where the shift is read.
`PushDownBuilderContext::make_sql_nodes_factory` splits shifts with
`extract_time_shifts`: interval shifts go to `TimeShiftState`, calendar shifts to
a separate map consumed only by `CalendarTimeShiftSqlNode`. `base_filter.rs` read
`TimeShiftState` alone, so a calendar shift looked like no shift at all.
Carry the calendar shifts to the filter-params path as well
(`SqlNodesFactory` -> `VisitorContext` -> `SqlEvaluatorVisitor`), and have
`to_sql_for_filter_params` distinguish the two kinds via `FilterParamsTimeShift`:
- a string column under an INTERVAL shift is offset, as before
- a string column under a CALENDAR shift is rejected, naming the binding, the
shift and the remedy - the mapping is data, not arithmetic, so there is no
expression over that column that stands for it and no bound the planner can
widen to without reading the calendar
- a callback column is unchanged: it receives the query's own bounds and
widens the pushed-down range itself, which is the only place the mapping can
be expressed
BEHAVIOUR CHANGE: a model combining a string FILTER_PARAMS column with a named
calendar shift now fails to plan where it previously returned silently empty or
under-filtered stages. The remedy is in the message.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwtxWoLUp7ULnsQJjVLkZt
|
Claude finished @paveltiunov's task in 2m 25s —— View job Review: sound diagnosis and a clean plumbing fix — but the rejection is too broad (1 high, 1 medium, 2 low)Full review — expand for findings, answer to the open question, and the checklist
What's goodThe root-cause analysis is right and easy to verify: Findings
Your open question — reject vs.
|
… key
Two review findings on the previous commit.
1. Not every calendar shift is a mapping. `CalendarDimensionTimeShift` carries
both `interval` and `sql`, and a calendar cube may declare a shift as
`{ name: prior_year, interval: 1 year, type: prior }` with no `sql`.
`calendar_time_shift_for_interval` routes that into the calendar map and
`CalendarTimeShiftSqlNode` renders it as plain arithmetic, so the column can
carry it after all - rejecting it was wrong, and the `<unnamed>` in the error
message was the tell, since the interval-matched path leaves `name` unset.
Reject only when `sql` is declared; offset when only an interval is. The
offset is inverted here because the calendar map keeps the declaration as
written and inverts at render, unlike `TimeShiftState`. A shift with neither
renders bare, matching the calendar node's own fallthrough.
2. The calendar lookup was an exact-name hit while the interval path resolves
the symbol. The map is keyed by the calendar cube's PK, so a binding on the
fact's own time dimension missed and the column rendered bare against
unshifted bounds - the same silent failure this PR fixes, one binding away.
Probe the dimension's `time_shift_pk_full_name` as well.
Adds a planner test pinning the offset and its sign for an interval-declared
calendar shift, the shape neither test covered.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwtxWoLUp7ULnsQJjVLkZt
|
I'll analyze this and get back to you. |
…de doesn't say Review nit: the block above the Calendar arm restated the CubeError::user message ten lines below it, and the base_filter.rs comment narrated the routing rather than guarding against reintroducing the bug. Keep the load-bearing part - why the column must carry the shift at all, and why both maps are probed - and let the error string carry the rest. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwtxWoLUp7ULnsQJjVLkZt
|
Claude finished @paveltiunov's task in 2m 25s —— View job Re-review of the two follow-up commits: both prior high/medium findings addressed; 1 medium, 2 low remainFull review — expand for findings and checklist
What the follow-ups fixed
Finding #1 (high) — resolved, and the sign is right. The rejection is now gated on This is the narrowing I asked for on the open question: only genuinely inexpressible ( Finding #3 (low) — resolved. The Remaining findings
Details and a concrete diff are in the two inline comments. VerificationI read the code and traced the sign against DocsStill unticked, and the narrowed rejection makes the note shorter to write: a string |
…MS under time shift Closes the two remaining review findings. The PK probe added in 6aa097b had no test. `shift_is_found_when_the_binding_is_not_the_calendar_pk` covers it: a calendar cube whose shifted dimension is not its primary key, so the shift is registered under the PK while FILTER_PARAMS binds the dimension itself. Verified meaningful - reverting the probe makes it fail with the column rendered bare. Note the reviewer's framing was slightly off and the test reflects what the probe actually reaches: `time_shift_pk_full_name` is only populated for dimensions on a calendar cube (dimension_symbol.rs:422), so a binding on the FACT's own time dimension has no PK to probe and cannot reach the calendar map at all - a named shift on such a dimension is dropped by `extract_time_shifts` before this point. That is a separate, pre-existing gap. Docs: a "Time-shifted measures" subsection under FILTER_PARAMS in context-variables.mdx, covering why the column must carry the stage's shift, which shift kinds can be carried, and the callback form as the remedy - the planning error was until now the only place that remedy was written down. Also fixes three assert messages that carried a literal backslash-n. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwtxWoLUp7ULnsQJjVLkZt
Reviewing
|
|
All four findings addressed. Head is now #1 (High) — real bug, fixed in #2 (Medium) — fixed in What the probe does reach is a calendar cube whose shifted dimension isn't its PK — the shift registers under the PK while the binding names the dimension. I did not add #3 (Low) — trimmed in #4 (Low) — both shapes now covered, per the two tests above. The fact-dimension shape you listed is the one that isn't, for the reason in #2. On the open question: agreed, and #1 was the substance of it. Narrowing to Docs: added a "Time-shifted measures" subsection under Planner suite: 1352 passed, 0 failed. Worth flagging for whoever picks this up: every review so far has been automated. The behaviour change — models with Generated by Claude Code |
…ment Three review nits on the follow-ups. The PK-probe comment claimed the fallback covers a FILTER_PARAMS binding on the fact's own time dimension. It cannot: `time_shift_pk_full_name` is populated only for calendar-cube dimensions, so a fact dimension has no PK to probe with, and a named shift on one is dropped by `extract_time_shifts` before this point. Now describes what the arm actually reaches - a filter on a non-PK dimension of the calendar - and names the fact-dimension shape as out of reach either way. `shift_is_found_when_the_binding_is_not_the_calendar_pk` (f690585) covers it. The two guarded calendar arms differed only in whether `interval` was set, and the first paid for that with an `unwrap()` the guard had already proved. Folded into one arm with an inner match on `shift.interval`. Dropped the paragraph that restated the guard; kept the one recording the inverse asymmetry, since a later "consistency fix" there would silently reverse the pushed-down bounds. The `FilterParamsTimeShift` doc still said a calendar shift has no expression over the fact's own column, which stopped being true when the interval-declared case started being offset. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwtxWoLUp7ULnsQJjVLkZt
…tests The file header still said a calendar shift has no offset and that string columns are rejected - true of the first commit, but not since the rejection was narrowed to sql-declared shifts, with interval_declared_calendar_shift_offsets_the_column sitting in the same file as the counterexample. Restated around the distinction that survived: sql-declared vs interval-declared. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwtxWoLUp7ULnsQJjVLkZt
|
I'll analyze this and get back to you. |
Reviewing
|
… error
`contains("prior_fiscal_year")` alone was satisfied by any error naming the
shift - a "time shift not found" or a join-resolution failure on the non-PK
dimension would have passed it, so a regression turning the probe into a
different error went unnoticed. Also assert "cannot carry", matching the
sibling test one screen up.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwtxWoLUp7ULnsQJjVLkZt
|
Claude finished @paveltiunov's task in 1m 56s —— View job Re-review of
|
| # | Severity | Where | Issue |
|---|---|---|---|
| 1 | Low | filter-params-calendar-time-shift.test.ts:4-17 |
The YAML test header still carries the pre-6aa097b claim that a calendar shift "has no such offset, so a string column cannot carry it" — the same staleness bda197c fixed on the Rust side, in the file it didn't touch. Inline comment has a five-line replacement. |
Docs
The new "Time-shifted measures" subsection reads well and sits in the right place. Spot-checks: [ref-calendar-cubes]: /docs/data-modeling/concepts/calendar-cubes matches the four other files that link it, #time_shift resolves against measures.mdx:828, and the Python-lambda-in-YAML form matches the existing BigQuery-shard example at :266, so it isn't inventing syntax. "Every stage then scans every band, so keep the bands as narrow as the calendar actually requires" is the right closing caveat — it's the cost a reader following this advice actually pays.
One stylistic gap: the surrounding subsections pair YAML with JavaScript in a <CodeGroup>; this one is a bare ```yaml block. Not worth blocking on, but a JS twin would match the page.
Verification
Read-only pass — I traced the folded arm and the comment claims against dimension_symbol.rs, time_shift_state.rs and calendar_time_shift.rs, and checked the docs links against the repo. cargo test -p cubesqlplanner was not run in this pass; the PR reports 1352 passed, 0 failed.
Merge note
Standing by the earlier point, which the author also flagged: every review on this PR has been automated. The behaviour change is now narrow — only sql-mapped calendar shifts combined with a string FILTER_PARAMS column fail to plan, where they previously returned silently empty stages — but it's still a new hard planning error on models that used to compile, and deserves a human call before merge.
…shift tests bda197c fixed this staleness on the Rust side and missed the twin here: the header still claimed a calendar shift has no offset and that string columns are rejected, which since 6aa097b is only true of sql-declared shifts. Restated around that distinction and let the fixture speak for the rest. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwtxWoLUp7ULnsQJjVLkZt
The band uses DATE_SUB(..., INTERVAL n DAY), which is BigQuery/MySQL syntax, so a reader on Postgres would copy something that does not parse. Say so, the way the BigQuery-shard example one section up already does, and point at the 371/364 bracketing as the part worth carrying across dialects. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwtxWoLUp7ULnsQJjVLkZt
|
Low #1 — taken, with a correction to the premise. The page isn't dialect-neutral: Fixed in prose rather than by rewriting the example, keeping it consistent with its neighbour: the intro now says the date arithmetic is BigQuery syntax and to adjust it, and points at the 371/364 bracketing as the part that carries across dialects. That's the load-bearing bit anyway — Low #2 — leaving as is, taking you at your word that it's a judgement call. The second paragraph documents a shape that looks like it should be covered and isn't; without it the natural next question ("what about a binding on the fact's own dimension?") has no answer in the file, and the answer isn't obvious — it needs both the Thanks for diffing the On the merge note: a human approval landed from @keydunov at 21:54, after your last pass, so that item is now closed. I'm not merging. Generated by Claude Code |
|
Claude finished @paveltiunov's task in 2m 11s —— View job Re-review of
|
| # | Severity | Where | Issue |
|---|---|---|---|
| 1 | Medium | context-variables.mdx:411-441 |
The label lands on the wrong engine. For a time dimension, {x} / {y} render as TIMESTAMP(?) on BigQuery — timeStampParam returns timeStampCast('?') (BaseQuery.js:2136), and BigqueryQuery.timeStampCast is TIMESTAMP(${value}) (BigqueryQuery.ts:48). BigQuery's DATE_SUB is DATE-only, so DATE_SUB(TIMESTAMP(?), INTERVAL 371 DAY) fails with No matching signature. TIMESTAMP_SUB is the BigQuery spelling. DATE_SUB on a datetime is valid MySQL, so the unlabelled version was true of some dialect and the label is what pins it to the one it doesn't run on — either switch to TIMESTAMP_SUB or say MySQL. My earlier nit asked for the dialect to be named; naming it surfaced that the arithmetic itself needed to move. |
| 2 | Low | context-variables.mdx:425-429 |
First filter() callback on the page with a top-level OR (the shard example at :266 is AND-only), thirty lines above a FILTER_GROUP warning about exactly that. Correct as written since the substitution is the whole WHERE; one outer paren makes it safe for a reader who appends AND …. |
Both are on the same lines the commit touched, and neither duplicates an existing thread — all eight of my prior threads are resolved and none were on this file.
Verification
Read-only pass. I traced the param rendering through BaseFilter.allocateTimestampParam → BaseQuery.timeStampParam → BigqueryQuery.timeStampCast rather than running anything; the commit under review touches only prose and two code fences, and no Rust or TypeScript changed since the suite numbers the PR reports.
Merge note
Unchanged, and the author has flagged it too: every review on this PR has been automated, and the behaviour change — an sql-mapped calendar shift with a string FILTER_PARAMS column now fails to plan where it previously returned silently empty stages — is a new hard planning error on models that used to compile. keydunov's approval is on the tree as of ce09d2e; 94d3a60 is docs-only on top of it.
The band passed the raw {x}/{y} straight to DATE_SUB and to the comparison, but
callback values arrive as bare query parameters - filter_param_values allocates
them through format_and_allocate_*_date_no_cast - so the example needed the same
DATE(TIMESTAMP(...)) wrapping the shard example one section up already uses. As
written it would not have run on BigQuery.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwtxWoLUp7ULnsQJjVLkZt
Re-review of
|
| # | Where | Note |
|---|---|---|
| 1 | context-variables.mdx:427-431 |
The top-level OR in the callback body is still unwrapped — existing thread re-affirmed, not re-posted. Correct as written since the substitution is the whole WHERE; one outer paren makes it safe for a reader who appends AND …, thirty lines above a FILTER_GROUP warning about exactly that. Low, and a judgement call. |
Eight of my nine prior threads are resolved; the one above is the only unresolved one and it still applies, so I left it rather than duplicating it.
Verification
Read-only pass. I traced renderFilterParams → allocateParam in packages/cubejs-schema-compiler/src/adapter/BaseQuery.js and compared against BaseFilter.allocateTimestampParam / BigqueryQuery.timeStampCast to establish which path a callback actually takes, and diffed the new example against the shard example on the same page. No suites were run — the commit under review touches prose and two code fences only.
Merge note
Unchanged, and now partly closed: keydunov approved at 21:54. The behaviour change remains narrow — only an sql-mapped calendar shift combined with a string FILTER_PARAMS column now fails to plan, where it previously returned silently empty stages — and the docs-only commits since sit on top of that approval.
View job run · branch claude/filter-params-date-injection-pm2qgo
The band is the page's first filter() callback with a top-level OR, thirty lines above the FILTER_GROUP warning about exactly that. Correct as written, since the substitution is the whole WHERE - but a reader appending "AND tenant_id = 1" to the copied sql would bind it to the second branch alone. One outer paren makes the snippet safe to extend, matching what production models using this pattern already do. Prose reworked so the two non-obvious details (bare params, the paren) are stated before the example rather than trailing it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwtxWoLUp7ULnsQJjVLkZt
|
Claude finished @paveltiunov's task in 1m 50s —— View job Re-review of
|
| # | Severity | Where | Issue |
|---|---|---|---|
| 1 | Low | context-variables.mdx:426-455 |
The example binds retail_calendar.calendar_d from inside weekly_margin.sql, but shows neither the calendar cube nor the join. Every other example on the page binds a dimension the same snippet declares, and here the join is also what makes the section's premise true. Inline comment has the two-line fix. |
The three-paragraph preamble now sitting between "Pass a function instead" and the fence is on the long side and the "Two details … are easy to miss" framing reads as review residue, but both details are genuinely non-obvious and undocumented elsewhere, so I'd leave it — noting it only so a later editor knows it can be tightened.
Verification
Read-only pass; I read the two commits and traced the callback-arg path through typed_filter.rs, filter_sql_context.rs and BaseQuery.js. No suite run — the commits under review touch only prose and two code fences, and nothing executable has changed since the numbers the PR reports.
Merge note
Unchanged and now largely settled: keydunov's human approval landed on ce09d2e, and everything since is docs-only. The behaviour change it covers is the narrow one — an sql-mapped calendar shift combined with a string FILTER_PARAMS column now fails to plan where it previously returned silently empty stages.
The binding crosses cubes - weekly_margin.sql pushes down a filter on retail_calendar.calendar_d - but the snippet showed neither the join nor any mention of the calendar cube, so unlike every other example on the page a reader could not run it as written. Adds the joins entry to both arms and a sentence naming what the calendar declares, since that join is also why the band needs widening at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwtxWoLUp7ULnsQJjVLkZt
CI:
|
|
Claude finished @paveltiunov's task in 1m 5s —— View job Re-review of
|
| # | Severity | Where | Issue |
|---|---|---|---|
| 1 | Low | context-variables.mdx:409-422 |
The preamble is now four paragraphs before the fence, three of them caveats (cast the params, parenthesise the OR, the join) — stacked, they read as a gotcha list, and two of the three are visible in the snippet itself. Inline comment suggests folding the first two into one sentence after the fence. Judgement call. |
All eleven of my prior threads are resolved; nothing to re-affirm and no duplicates skipped.
Verification
Read-only pass — I read the commit, compared the join against the Rust fixture and checked the link refs against the file's own definitions. No suites run: the commit touches prose and two code fences only, and nothing executable has changed since the numbers the PR reports (1352 passed, 0 failed).
Merge note
Settled since keydunov's approval on ce09d2e; the five commits since are all docs-only. The behaviour change under that approval is the narrow one — an sql-mapped calendar shift combined with a string FILTER_PARAMS column now fails to plan where it previously returned silently empty stages.
CI: four more failures, all on superseded commits, none this PR'sFollowing the
The Windows one is the only one that could plausibly have been mine, since it compiles the Rust I changed, so I opened it first. It compiled and my planner code ran successfully inside it — the failure is downstream of anything this diff touches. On re-running: these jobs are wrapped in No fix to port: there is nothing in this repo to fix for a container that won't authenticate, a cloud warehouse without credentials, or a Jest worker that won't exit. For the record on scope — Generated by Claude Code |
The preamble had accumulated into four paragraphs, three of them caveats stacked ahead of the example - a gotcha list rather than an explanation. Both the cast and the paren are visible in the snippet, so they read better as a note after it; the join sentence stays above, since it is the one thing the snippet cannot show. Net reduction, no content dropped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwtxWoLUp7ULnsQJjVLkZt
|
Claude finished @paveltiunov's task in 36s —— View job Re-review of
|
|
Addendum to the two CI notes above — GitHub's artifact service, not Codecov — the job never got as far as uploading. It downloads coverage artifacts produced by the earlier jobs, several of which failed for the container/credential reasons already reported, and the action exhausted its own five retries, so there's no re-run left to spend. Same conclusion as before: nothing in this diff to fix, and both runs are behind the current head Generated by Claude Code |
CI:
|
CI:
|
| attempt | failed | pre-aggregations that went missing |
|---|---|---|
| 1 | 4 | ec__t_a_external20200201, …20200101 (x2), …20200501 |
| 2 | 5 | ec__s_a_external, ec__t_a_external20200301, …20200601, big_e_commerce__multi_time_dim_for_count_external20200301, …20200401 |
| 3 | 3 | big_e_commerce__multi_time_dim_for_count_external20200601, …20201201, big_e_commerce__category_flat_external |
Every failure is the same shape — Not found: Table cube-open-source:dev_pre_aggregations.<name>_<hash>_<hash>_<token> — and in each attempt the first failure is the must built pre-aggregations step itself, with the query tests after it failing downstream on the table that step never left behind.
Two things follow. The trailing token increments across attempts (1l9mga4 → 1l9mgbn → 1l9mgcm → 1l9mgfv → 1l9mgn3 → 1l9mgol), so each attempt really did build fresh tables into the shared cube-open-source:dev_pre_aggregations dataset — and a different one vanished between build and query each time. A deterministic planner defect would fail the same tests on every attempt. This is concurrent runs colliding in one shared BigQuery dataset, which is exactly what many overlapping runs on this branch would produce.
Independently: this suite cannot reach the code I changed. My new branch sits inside the pre-existing if !filters_ctx.filter_params_columns.is_empty() guard in base_filter.rs, and packages/cubejs-testing-drivers/ contains no FILTER_PARAMS at all — grep -rn "FILTER_PARAMS\|filterParams" over the whole package returns nothing. I checked rather than assumed, because those fixtures do define a RetailCalendar cube with calendar: true and timeShift — half of the interaction this PR targets, but never combined with a filter param.
No re-run to spend: the retry action exhausted its own three attempts and reported Final attempt failed.
That's eight failing checks, none with an assertion failure attributable to this diff — environment (MSSQL auth, firebolt credentials, artifact download), harness (Jest worker teardown), or this shared-dataset race. A fresh full run is in flight on 3dea13a now.
Generated by Claude Code
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #11772 +/- ##
==========================================
- Coverage 80.21% 80.21% -0.01%
==========================================
Files 500 500
Lines 105622 105622
Branches 3884 3884
==========================================
- Hits 84728 84724 -4
- Misses 20344 20348 +4
Partials 550 550
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
CI:
|
| attempt | failed | first failure |
|---|---|---|
| 1 | 8 | must built pre-aggregations → Not found: Table ec__s_a_external_o2swucpv_… |
| 2 | 3 | must built pre-aggregations → Not found: Table ec__t_a_external20201001_… |
| 3 | 3 | must built pre-aggregations → Not found: Table ec__t_a_external20201201_… |
Every failure is Not found: Table cube-open-source:dev_pre_aggregations.…, with the build step failing first and the query tests failing downstream on tables it never left behind. Attempt 1 failed eight tests; two attempts later, on the same code, five of those eight passed.
The one thing worth calling out. Attempt 1's list includes:
● Tesseract: SQL API: Timeshift measure from cube
error: Database Execution Error: Not found: Table ec__t_a_external20200801_odndd3lp_3pxplc0q_1l9mhab
Given this PR is about time shifts, that name deserves a second look rather than a wave-through — so: it is the same missing-table error, not a wrong-SQL or assertion failure, and it passed on attempts 2 and 3 of this identical commit. It is the dataset race catching one more test, not this diff.
The reachability argument is unchanged and independent of all of the above: the new branch sits inside the pre-existing if !filters_ctx.filter_params_columns.is_empty() guard, and grep -rn "FILTER_PARAMS\|filterParams" packages/cubejs-testing-drivers/ returns nothing — this suite has no filter params anywhere, so the changed code cannot execute in it.
No re-run to spend: the retry action exhausted its own three attempts and reported Final attempt failed.
Everything else on 3dea13a is green so far, including unit (24.x, 3.13) (which runs this PR's new schema-compiler test), unit-core, integration-cubestore, integration-smoke (24.x, 3.13, true), Check fmt/clippy, lint, build, codecov/patch and codecov/project.
Generated by Claude Code

Check List
Description of Changes Made
A
multi_stagemeasure shifted by name —time_shift: [{ name: prior_fiscal_year }], resolved through a calendar cube's owntime_shiftdeclaration — rendered its cube'sFILTER_PARAMSpush-down as a bare column bound to the query's unshifted reporting bounds.That contradicts the stage around it. The stage joins the calendar on its mapping column and reads prior-period fact rows, while the pushed-down predicate restricts the same scan to the reporting period. The stage comes back empty — the failure CORE-543 / #11030 fixed for interval shifts, reached by a different route.
Observed SQL before this change (both shifted stages, all bounds the reporting day):
Root cause — two levels below where the shift is read.
PushDownBuilderContext::make_sql_nodes_factorysplits shifts viaextract_time_shifts: interval shifts go toTimeShiftState, calendar shifts to a separate map consumed only byCalendarTimeShiftSqlNode.base_filter.rsreadTimeShiftStatealone, so a calendar shift looked like no shift at all and the offsetting branch added by CORE-543 was never reached.Fix — carry the calendar shifts to the filter-params path too (
SqlNodesFactory→VisitorContext→SqlEvaluatorVisitor), and haveto_sql_for_filter_paramsdistinguish the cases via a newFilterParamsTimeShiftenum:intervalinterval.inverse()— now correct, was empty stagessqlCalendarTimeShiftSqlNode's own fall-throughOnly an
sql-mapped calendar shift is genuinely inexpressible: it resolves through a mapping column on the calendar, so there is no expression over the fact's own column that stands for it and no bound the planner can widen to without reading the calendar. A callback column is the one form that can express it — it receives the query's own bounds and widens the pushed-down range itself — so the error points there.The lookup also probes the calendar cube's PK, since the calendar map is keyed by it: a filter on a non-PK dimension of the calendar would otherwise miss the bare name and render the column bare. (A filter on a non-calendar cube's dimension is out of reach either way —
time_shift_pk_full_nameis only populated for calendar-cube dimensions, and a named shift on such a dimension is dropped byextract_time_shiftsearlier. Pre-existing, separate.)FILTER_PARAMScolumn with ansql-mapped calendar shift now fail to plan; they previously returned silently empty stages, so this surfaces an existing bug rather than breaking working behaviour. Calendar shifts declared as a plainintervalare not affected — they now keep planning and start emitting correct SQL where they previously produced the same empty stages. Still a visible change, and worth a reviewer's judgement.Open question — resolved. Reject vs.
1 = 1: rejecting stands for thesql-mapped case (a silent switch to a full-table scan is its own surprise, and it matches the callback+interval rejection two arms down). Narrowing tosql-mapped shifts is what shrank the blast radius.Tests
rust/…/tests/filter_params_calendar_time_shift.rs—sql-mapped string column rejected; callback column still pushed into every shifted stage; each stage filters on its own mapping column; interval-declared calendar shift offset with the sign pinned at(week_end_d + interval '-1 year'); shift found through the calendar PK when the binding is a non-PK calendar dimension (verified meaningful — reverting the probe makes it fail with the column bare).packages/cubejs-schema-compiler/test/unit/filter-params-calendar-time-shift.test.ts— the first three at the YAML level.The first two commits landed the tests as characterisation tests pinning the broken behaviour, then the fix inverted them, so the diff shows exactly what changed.
Docs — new "Time-shifted measures" subsection under
FILTER_PARAMSinreference/data-modeling/context-variables.mdx: why the pushed-down column must carry the stage's shift, which shift kinds can be carried, and the callback form as the remedy with a worked fiscal-prior-year band.Verification:
cargo test -p cubesqlplanner --lib→ 1352 passed, 0 failed. Schema-compiler unit suite → 913 passed; the 2ErrorReportersnapshot failures are pre-existing and reproduce on a clean tree.cargo fmt/clippy/eslintclean.🤖 Generated with Claude Code
https://claude.ai/code/session_01EwtxWoLUp7ULnsQJjVLkZt