Skip to content

feat(duckdb): don't translate ops.First() to ops.FirstValue() in windows #11308

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

NickCrews
Copy link
Contributor

Fixes #11307

Probably a bunch of other backends will fail this test, when I see which ones I'll add them.

Maybe I could solve this in a better way, but at least I get it working for duckdb here.

@github-actions github-actions bot added tests Issues or PRs related to tests sql Backends that generate SQL labels Jun 6, 2025
@NickCrews NickCrews force-pushed the any-agg-in-window branch from fc60c19 to 63a7981 Compare June 6, 2025 20:08
@NickCrews
Copy link
Contributor Author

Ah, I think the actual bug is that the order_by clause isn't properly transferred to the window Op from the First op. If you do t.mutate(x.first().over(order_by="y", group_by="x")) then it works.

@NickCrews
Copy link
Contributor Author

Also there is a separate bug where we should be forwarding the RESPECT NULLS/IGNORE NULLS to sqlglot depending on if our Operation has ignore_null` of True or False.

Do not replace first() with first_value() because, per
https://duckdb.org/docs/stable/sql/functions/window_functions#aggregate-window-functions
The first and last aggregate functions are shadowed by the respective
general-purpose window functions, with the minor consequence that the
FILTER clause is not available for these but IGNORE NULLS is.

After this change, the generated SQL is more canonical, and include_null actually works.

This probably should wait until ibis-project#11311 lands, which has the tests for this change.
@NickCrews
Copy link
Contributor Author

marking as WIP until #11314 is resolved, because that is a better implementation.

@NickCrews NickCrews marked this pull request as draft June 6, 2025 23:05
@NickCrews NickCrews changed the title feat(duckdb): support t.group_by().mutate(x.first(order_by="y")) feat(duckdb): don't translate ops.First() to ops.FirstValue() in windows Jun 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sql Backends that generate SQL tests Issues or PRs related to tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug: order_by is not preserved in .first() aggregate when used in window functions
1 participant