fix(schema-compiler): remove AthenaQuery.convertTz override that produces wrong timestamps on engine v3#10779
Conversation
…uces wrong timestamps on engine v3 On Athena engine v3 (Trino-based), CAST(timestamp WITH TIME ZONE AS TIMESTAMP) returns the UTC instant's wall-clock fields, not the local wall-clock at the specified zone. The AthenaQuery.convertTz override used this pattern, silently returning UTC wall-clock instead of the target timezone's wall-clock. Remove the override so AthenaQuery inherits PrestodbQuery.convertTz, which manually shifts by timezone_hour + timezone_minute offsets before casting. Add a unit test to verify AthenaQuery.convertTz uses the manual offset calculation rather than the broken simplified cast. Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>
|
Claude finished @paveltiunov's task in 2m 5s —— View job Code Review: PR #10779
Overall: This is a correct and well-scoped fix. The change is minimal, the problem is clearly explained, and the inherited What looks good
Issues and suggestions
|
Add a testQueries test case that queries ECommerce.count by month with timezone 'Asia/Kolkata'. This exercises the convertTz code path in a full Cube API stack to verify that non-UTC timezones produce correct monthly time dimension bucketing on Athena (and all other drivers). Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>
Codecov Report✅ All modified and coverable lines are covered by tests.
Additional details and impacted files@@ Coverage Diff @@
## master #10779 +/- ##
===========================================
- Coverage 78.77% 57.99% -20.79%
===========================================
Files 466 215 -251
Lines 92087 16775 -75312
Branches 3389 3388 -1
===========================================
- Hits 72546 9728 -62818
+ Misses 19049 6554 -12495
- Partials 492 493 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Claude finished @paveltiunov's task in 2m 4s —— View job Code Review: PR #10779 — fix(schema-compiler): remove AthenaQuery.convertTz override
Overall: This is a correct, well-scoped fix. The root cause analysis is sound, the change is minimal, and the inherited What looks good
Issues
|
…ation mismatch The non-UTC timezone test was failing because ECommerce.count with orderDate matched the MultiTimeDimForCount pre-aggregation, which was only built for UTC. Switch to customOrderDateNoPreAgg time dimension which has no pre-aggregation, so the query goes directly to the source database and exercises the convertTz code path. Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>
…non-UTC timezone (Asia/Kolkata)' test Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>
MySQL returns count values as numbers, not strings. Update the snapshot to match MySQL's actual output format. Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>
Check List
Description of Changes Made
On Athena engine v3 (Trino-based),
CAST(timestamp WITH TIME ZONE AS TIMESTAMP)returns the UTC instant's wall-clock fields, not the local wall-clock at the specified zone. TheAthenaQuery.convertTzoverride used this exact pattern:This caused every
convertTzcall on engine v3 to silently return UTC wall-clock instead of the target timezone's wall-clock, leading to incorrect time dimension bucketing (e.g., monthly buckets shifted by the timezone offset).Fix: Remove the
convertTzoverride fromAthenaQueryso it inheritsPrestodbQuery.convertTz, which correctly handles this by manually shifting usingtimezone_hour+timezone_minuteoffsets before casting:Changes:
packages/cubejs-schema-compiler/src/adapter/AthenaQuery.ts— removed theconvertTzoverride and its stale commentpackages/cubejs-schema-compiler/test/unit/athena-query.test.ts— added unit test verifyingAthenaQuery.convertTzuses the manual offset calculationpackages/cubejs-testing-drivers/src/tests/testQueries.ts— added E2E test case querying withtimezone: 'Asia/Kolkata'to verify non-UTC monthly bucketing works correctly across all drivers including Athena