-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
fix(cubestore): 'unsorted data' assertion with high-precision timestamps
CubeStore used to truncate timestamps to millisecond precision when writing to parquet, but sort the data with nanosecond precision. This led to 'unsorted data in merge' assertions. Ensure we truncate before we sort the data. Increasing the storage precision is another option, but that involves backward and forward compatibility issues and requires more planning. So stick with the current behavior for now. If you see 'unmerged data' assertion in the logs, you have to manually drop the tables where this happens, e.g. by rebuilding the rollups in CubeJS.
- Loading branch information
1 parent
9c0d4fe
commit 58a8cb4
Showing
2 changed files
with
41 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters