Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(sql): allow list of timestamps of higher precision than required in alter table attach/detach/drop partition #3002

Merged
merged 16 commits into from
Feb 27, 2023

Conversation

marregui
Copy link
Contributor

@marregui marregui commented Feb 18, 2023

Closes #2879

The approach is to let the compiler only process the chars required to understand a partition dir name, while ignoring the rest, for each of the items in the comma separated list following the LIST token.

Example, for a table partitioned by month:

alter table x drop partition list '2023-03-29T23', ... ;

2023-03-29T23 is a valid partition name now for month granularity because the compiler knows that the patter for partition by month is yyyy-MM, which means it can parse a date up to month, thus -29T23 is ignored. Incidentally, this would work too alter table x drop partition list '2023-03-a-banana' ;. We are trying to make life easier for users who might use a full timestamp (select max(ts) from table) to drop a partition (the last one in this example).

WHERE is not supported, no value, and harder a problem to solve.

The case of partition by week is complicated. We first try to parse the partition name candidate with format YYYY-Www, and if it fails we assume the user sent a string that looks more like a date, or a timestamp, and we parse it with the day pattern yyyy-MM-dd, then we translate it to the timestamp of the day the week starts at.

@marregui marregui added New feature Feature requests SQL Issues or changes relating to SQL execution labels Feb 18, 2023
@marregui marregui self-assigned this Feb 18, 2023
@marregui marregui changed the title feat(sql): allow list of timestamps of higher precision than required int alter table attach/detach/drop partition feat(sql): allow list of timestamps of higher precision than required in alter table attach/detach/drop partition Feb 21, 2023
Copy link
Contributor

@puzpuzpuz puzpuzpuz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for implementing this feature!

@ideoma
Copy link
Collaborator

ideoma commented Feb 27, 2023

[PR Coverage check]

😍 pass : 78 / 80 (97.50%)

file detail

path covered line new line coverage
🔵 io/questdb/cairo/PartitionBy.java 49 51 96.08%
🔵 io/questdb/griffin/engine/functions/date/WeekOfYearFunctionFactory.java 12 12 100.00%
🔵 io/questdb/std/datetime/microtime/TimestampFormatUtils.java 10 10 100.00%
🔵 io/questdb/griffin/SqlCompiler.java 7 7 100.00%

@bluestreak01 bluestreak01 merged commit 5148cd6 into master Feb 27, 2023
@bluestreak01 bluestreak01 deleted the ma/drop-partition-of-higher-resolution branch February 27, 2023 13:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
New feature Feature requests SQL Issues or changes relating to SQL execution
Projects
No open projects
Status: Done
Development

Successfully merging this pull request may close these issues.

ALTER TABLE DROP PARTITION should automatically truncate the given timestamp to the partition timestamp
4 participants