add time_bucket feature - #794
Conversation
4f01e43 to
1b6214e
Compare
|
This is a proposal to implement a window function similar to time_bucket. From my preliminary understanding, only TigerGraph supports this feature at present. Discussion for development is welcome. Thank you. |
|
Will the duckdb variant work for you? |
Either the streaming engine style or the DuckDB style works for me. What matters is that Ladybug supports this capability. So I need to implement this feature following either the DuckDB syntax or the streaming style, right? |
|
Yes, the feature looks valuable. Let's use the duckdb style for consistency with the rest of UDFs. |
Are there any existing syntax implementations aligned with DuckDB or streaming engines in the current project? I'd like to refer to them. Could you provide an example? |
|
is an example where we implemented functionality to be compatible with duckdb. It was more semantics than syntax. |
1b6214e to
d6e9dba
Compare
Signed-off-by: ericyuanhui <285521263@qq.com>
d6e9dba to
f0f6a92
Compare
follow duckdb time_bucket add ladybug feature. thank you |
|
@ericyuanhui the PR description still says |
update |
|
@ericyuanhui - broke the build on windows. Fixing now. |
I use this project under Linux and have never built it on Windows. Am I right that you’re fixing the Windows build issue now? |
|
@ericyuanhui don't worry about it. Taken care of. This was for future reference. Linux only CI is a conscious decision for dev velocity |
Required Syntax
The first argument uses Ladybug's existing
interval('...')expression andmust describe a non-null positive interval. It must support the units accepted
by Ladybug's current interval parser: years, quarters, months, weeks, days,
hours, minutes, seconds, and smaller units. The function is semantically
similar to DuckDB; it does not need to reproduce DuckDB's literal spelling.
Equivalent bucket-start usage for the existing daily IPv4 feature is:
The first
WITHis only needed when the caller wants to reuse the bucketstart.
window_endis not generated bytime_bucket; callers compute it asan ordinary expression when required.
Complete example:
Expected result:
Edge-table / relationship-property example:
Expected result:
Interval Reuse Requirements
time_bucketmust directly reuse Ladybug'sinterval('...')syntax, existinginterval parser, and its constant-folded result. It must not add a second unit
parser or reparse the input string. An
INTERVALis represented by threeindependent components,
months,days, andmicros; the implementation mustpreserve all three.
weeks, days, hours, minutes, seconds, milliseconds, microseconds, and their
currently supported aliases.
months == 0,daysandmicroscan be treated as a fixed bucketwidth.
months != 0, bucketing must use calendar year/month arithmetic. Itmust not call a helper that approximates months as 30 days or years as 365
days.
interval('5 months'),interval('2 years'),interval('3 weeks'), andcomposite intervals are valid
time_bucketinputs. The function binder mayvalidate constantness, non-nullness, positivity, and representability in the
output type, but must not narrow Ladybug's supported interval units.
Required Semantics
INTERVALvalueproduced by
interval('...'). It must not reject a value merely because ithas a month or year component:
interval('5 months'),interval('2 years'),interval('3 weeks'),interval('13 days'), andcurrent parser-supported hour/minute/second units are in scope.
DATE,TIMESTAMP,TIMESTAMP_SEC,TIMESTAMP_MS,TIMESTAMP_NS, orTIMESTAMP_TZ. The implementation must not arbitrarily omit a type thatthe current type system supports.
DATEcountsdays from
1970-01-01, andTIMESTAMPcounts microseconds from1970-01-01 00:00:00. Fixed widths floor directly from that epoch; calendarwidths with a month component advance from
1970-01-01using Ladybug'sinterval_tcalendar addition.TIMESTAMP_TZuses its represented instant;interval('1 day')means exactly 24 hours, not a session-local civil day.they do not restart for the input year. For example,
time_bucket(interval('5 months'), date('2024-02-29'))returns2023-10-01:the interval from
2023-10-01through2024-03-01contains that date. Thismatches Ladybug's single calendar addition of
interval_t * nto a timestamp.DATErequires a whole-day width,TIMESTAMP_SECa whole-second width, andTIMESTAMP_MSa whole-millisecondwidth. The other current types accept fixed widths representable by their
storage precision.
floor(t / width) * width.A width containing a month component uses calendar year/month arithmetic
from
1970-01-01and must not turn months or years into a fixed number ofdays or microseconds. Results are bucket starts with half-open membership
[start, next_start).columns are produced.
TIMESTAMP_NSvalue with a sub-microsecond remainder is normalized towardnegative infinity before bucket arithmetic.