Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refactor(everything): port sql backends to sqlglot; split relations; fix a large number of bugs #8310

Merged
merged 190 commits into from Feb 12, 2024
Merged

refactor(everything): port sql backends to sqlglot; split relations; fix a large number of bugs #8310

merged 190 commits into from Feb 12, 2024

Conversation

cpcloud
Copy link
Member

@cpcloud cpcloud commented Feb 12, 2024

This PR is the rebase of the-epic-split onto main.

@cpcloud cpcloud added this to the 9.0 milestone Feb 12, 2024
@cpcloud cpcloud added bug Incorrect behavior inside of ibis refactor Issues or PRs related to refactoring the codebase ux User experience related issues expressions Issues or PRs related to the expression API sql Backends that generate SQL docs-preview Add this label to trigger a docs preview ci-run-cloud Add this label to trigger a run of Bigquery and Snowflake in CI labels Feb 12, 2024
@ibis-docs-bot ibis-docs-bot bot removed the ci-run-cloud Add this label to trigger a run of Bigquery and Snowflake in CI label Feb 12, 2024
@cpcloud
Copy link
Member Author

cpcloud commented Feb 12, 2024 •
edited

The Docs PR job appears to be hitting the rate limit of clickhouse's playground.

@cpcloud cpcloud force-pushed the the-epic-split branch 2 times, most recently from fb90baa to 918a9b0 Compare February 12, 2024 15:09
@cpcloud cpcloud added the ci-run-cloud Add this label to trigger a run of Bigquery and Snowflake in CI label Feb 12, 2024
@cpcloud
Copy link
Member Author

cpcloud commented Feb 12, 2024

docs PR job issue should be fixed by #8312

@ibis-docs-bot ibis-docs-bot bot removed the ci-run-cloud Add this label to trigger a run of Bigquery and Snowflake in CI label Feb 12, 2024
@cpcloud cpcloud mentioned this pull request Feb 12, 2024
Closed
21 tasks
@cpcloud
Copy link
Member Author

cpcloud commented Feb 12, 2024

The docs_pr and benchmarks jobs will not run because they have been replace with Docs PR in the-epic-split, and benchmarks only runs against main.

@cpcloud
Copy link
Member Author

cpcloud commented Feb 12, 2024

I'm going to disable those jobs in required checks now.

@cpcloud
Copy link
Member Author

cpcloud commented Feb 12, 2024

@jcrist @kszucs @gforsyth Can I get an explicit +1 here before merging?

gforsyth
gforsyth approved these changes Feb 12, 2024
View reviewed changes
Copy link
Member

@gforsyth gforsyth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@cpcloud cpcloud added docs-preview Add this label to trigger a docs preview and removed docs-preview Add this label to trigger a docs preview labels Feb 12, 2024
@ibis-docs-bot ibis-docs-bot bot removed the docs-preview Add this label to trigger a docs preview label Feb 12, 2024
@gforsyth gforsyth added the ci-run-cloud Add this label to trigger a run of Bigquery and Snowflake in CI label Feb 12, 2024
@ibis-docs-bot ibis-docs-bot bot removed the ci-run-cloud Add this label to trigger a run of Bigquery and Snowflake in CI label Feb 12, 2024
@kszucs kszucs added the docs-preview Add this label to trigger a docs preview label Feb 12, 2024
@ibis-docs-bot ibis-docs-bot bot removed the docs-preview Add this label to trigger a docs preview label Feb 12, 2024
@ibis-docs-bot
Copy link

ibis-docs-bot bot commented Feb 12, 2024

Docs preview: https://pr-8310-918a9b02bc31bc58dced4bd43fb16b55c3ef40c1--ibis-quarto.netlify.app

cpcloud and others added 28 commits February 12, 2024 14:31
@cpcloud
test: remove outdated sqlize tests
f2b71e0
@cpcloud
test(pyspark): snapshot update
e8f6de2
@cpcloud
test(arrays): disregard row output order when testing array APIs (#8225)
9539308 
Cleans up more array tests to disregard row output order using set
comparison.
@kszucs @cpcloud
fix(api): restore and deprecate ir.Table.to_array() (#8227)
8ce0193 
Co-authored-by: Phillip Cloud <417981+cpcloud@users.noreply.github.com>
@kszucs @cpcloud
chore(ci): allow codecov uploads to fail
8666d94
@cpcloud
test(duckdb): enable previously incorrectly skipped geospatial tests
64b051d
@kszucs @cpcloud
refactor(ir): remove now obsolete ops.Named mixin (#8244)
04f3e5e 
Co-authored-by: Phillip Cloud <417981+cpcloud@users.noreply.github.com>
@jcrist @cpcloud
test(api): add test for Table.select on literals (#8258)
185028e 
Fixes #8255.
@kszucs @cpcloud
refactor(ir): accept any relation in ops.ExistsSubquery (#8264)
99b9281
@cpcloud
test(dot-sql): replace snapshot test with sqlglot compilation test to…
216834e 
… avoid clashing table names (#8259)

Addresses flaky testing observed in
https://github.com/ibis-project/ibis/actions/runs/7811784691/job/21307516496,
ultimately coming from two concurrent runs of the BigQuery backend test
suite trying to create the same table. Here I changed the test to
compare against an equivalent sqlglot expression instead of using a
snapshot test.
@kszucs @cpcloud
chore(sql): remove unused ops.Project compile method (#8265)
7a27727
@jcrist @cpcloud
refactor: consolidate rewrite rule implementations
87c46d9
@jcrist @cpcloud
refactor: remove duplicated rewrite_sample rule
fedc675
@jcrist @cpcloud
fix(pyarrow): support accepting pyarrow dictionary types as inputs (#…
492f766 
…8276)

Fixes #8207.
@cpcloud
refactor(sql): move dialects to always-importable location (#8279)
83532ed
@jcrist @cpcloud
fix(trino): re-enable native TABLESAMPLE support (#8284)
62b2b3c
@cpcloud
test(sql): remove xdist_group marker for .sql tests (#8295)
1e31e78
@cpcloud
fix(backends): make string concat-with-null behavior consistent acros…
4e3cd6a 
…s backends (#8305)

Fixes #8302.
@cpcloud
refactor(flink): port to sqlglot
f09aefe
@cpcloud
chore(deps): relock
c35b516
@cpcloud
test(duckdb): fix no-arg udf test
6a9cbd9
@cpcloud
chore(datafusion): enable xpassing uuid test
90ae0de
@cpcloud
chore(snowflake): regen snapshot queries using select 1
b3a82bb
@cpcloud
chore(snowflake): use max and min for cumulative boolean agg
a2bac56
@cpcloud
refactor(snowflake): adjust window frames to use _minimize_spec due…
7f4a6dc 
… to upstream snowflake changes
@cpcloud
test(snowflake): fix error types and explain the nth_value situation
c0a00d0
@cpcloud
ci: remove running on the-epic-split
333d8a6
@cpcloud
ci: disable doc publishing until geo blog can be fixed
e6e4a46
@kszucs kszucs merged commit 989ad4f into main Feb 12, 2024
84 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kszucs kszucs kszucs approved these changes

@gforsyth gforsyth gforsyth approved these changes

Assignees
No one assigned
Labels
bug Incorrect behavior inside of ibis expressions Issues or PRs related to the expression API refactor Issues or PRs related to refactoring the codebase sql Backends that generate SQL ux User experience related issues
Projects
None yet
Milestone
9.0
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@cpcloud @kszucs @gforsyth @jcrist