New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
dialect/sql/sqlgraph: replace id IN with EXISTS for HasNeighbors predicates #3492
Conversation
Borne out of slack discussion: https://gophers.slack.com/archives/C01FMSQDT53/p1681910810225949 In my case, I had two tables, tracking messages and their deliveries for a (google) pubsub emulator. I had ~160k rows in each, with one delivery row for each message row. The code in question was looking for messages with no remaining deliveries to clean them up in batches (at a high level: get messages older than a threshold age with no associated deliveries, with a limit). With the prior implementation, this query was often taking over 15 minutes to run (and never less than 6 minutes according to my metrics) due to the query having to do a seq scan on both tables, and being unable to use any indexes such as the one indexing the delivery->message FK field. With this implementation, the query takes 500ms or less -- it's somewhat variable depending on other system load and whether there are any rows for it to find -- finding rows results in shorter runtimes because the |
Thanks, @mgabeler-lee-6rs. I appreciate the detailed report 🙏 |
[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [entgo.io/ent](https://togithub.com/ent/ent) | require | patch | `v0.12.4` -> `v0.12.5` | --- ### Release Notes <details> <summary>ent/ent (entgo.io/ent)</summary> ### [`v0.12.5`](https://togithub.com/ent/ent/releases/tag/v0.12.5) [Compare Source](https://togithub.com/ent/ent/compare/v0.12.4...v0.12.5) #### What's Changed - doc: fix typos in paging.md by [@​a8m](https://togithub.com/a8m) in [ent/ent#3458 - entsql: added OnDelete annotation by [@​giautm](https://togithub.com/giautm) in [ent/ent#3459 - doc/md: update doc for OnDelete annotation by [@​giautm](https://togithub.com/giautm) in [ent/ent#3460 - entc/gen: avoid ambiguous column names when joining the same table by [@​a8m](https://togithub.com/a8m) in [ent/ent#3461 - entc/gen: rename <type>.Order to <type>.OrderOption by [@​a8m](https://togithub.com/a8m) in [ent/ent#3468 - dialect/sql: skip formatting qualified columns by [@​a8m](https://togithub.com/a8m) in [ent/ent#3469 - dialect/sql/sqlgraph: allow query edge fields with alias by [@​a8m](https://togithub.com/a8m) in [ent/ent#3470 - dialect/sql: add method for finding selection occurrences in queries by [@​a8m](https://togithub.com/a8m) in [ent/ent#3473 - doc: add entgql edge ordering by [@​a8m](https://togithub.com/a8m) in [ent/ent#3482 - cmd/ent: hide the --idtype flag from generate command by [@​a8m](https://togithub.com/a8m) in [ent/ent#3490 - dialect/sql/sqlgraph: replace id IN with EXISTS for HasNeighbors predicates by [@​a8m](https://togithub.com/a8m) in [ent/ent#3492 - .github/workflow: added atlas-ci for examples by [@​giautm](https://togithub.com/giautm) in [ent/ent#3496 - entc/gen: load referenced foreign-keys on eager-loading by [@​a8m](https://togithub.com/a8m) in [ent/ent#3497 - entc/gen: remove unnecessary type arguments by [@​a8m](https://togithub.com/a8m) in [ent/ent#3503 - doc/md/ci: fix broken link for ent/contrib/ci action's repo by [@​Arthur1](https://togithub.com/Arthur1) in [ent/ent#3504 - doc/md: edge ordering is supported by [@​masseelch](https://togithub.com/masseelch) in [ent/ent#3506 - entc/gen/template: fix intercept package imports by [@​masseelch](https://togithub.com/masseelch) in [ent/ent#3507 - doc: add schema inspection to getting-started page by [@​a8m](https://togithub.com/a8m) in [ent/ent#3510 - entc/load: report 'go mod' errors on schema loading by [@​a8m](https://togithub.com/a8m) in [ent/ent#3511 - .github/workflow: sync action by [@​giautm](https://togithub.com/giautm) in [ent/ent#3513 - dialect/sql/schema: allow ariga.io/atlas schema diff to be extended by [@​a8m](https://togithub.com/a8m) in [ent/ent#3514 - dialect/sql/schema: support passing schema diff options by [@​a8m](https://togithub.com/a8m) in [ent/ent#3516 - dialect/sql: add OrderByRand option by [@​a8m](https://togithub.com/a8m) in [ent/ent#3518 - ent/privacy: move some of the generated privacy helpers to ent/privacy by [@​a8m](https://togithub.com/a8m) in [ent/ent#3527 - entc/gen: avoid conflict between order by edge-count and fields end with \_count by [@​a8m](https://togithub.com/a8m) in [ent/ent#3534 - Fix typo in docs by [@​jibbolo](https://togithub.com/jibbolo) in [ent/ent#3539 - dialect/sql: expose the underlying builder of UpdateSet by [@​a8m](https://togithub.com/a8m) in [ent/ent#3541 - Fix go install instruction for atlas by [@​toshi0607](https://togithub.com/toshi0607) in [ent/ent#3526 - entc/gen: fix conflicts between fields named 'op' and Mutation.Op by [@​a8m](https://togithub.com/a8m) in [ent/ent#3550 - ent: document ent.Value returned by mutations by [@​a8m](https://togithub.com/a8m) in [ent/ent#3556 - entc/gen: avoid conflict between package and receiver names by [@​a8m](https://togithub.com/a8m) in [ent/ent#3563 - dialect/sql/sqljson: inline boolean values by [@​a8m](https://togithub.com/a8m) in [ent/ent#3570 - schema/fields: validate for slices builder by [@​masseelch](https://togithub.com/masseelch) in [ent/ent#3566 - entc/gen: quote index annotations in generated schemas by [@​a8m](https://togithub.com/a8m) in [ent/ent#3578 - entc/gen: allow naming ent/schemas as builder by [@​a8m](https://togithub.com/a8m) in [ent/ent#3593 - entc/integration: add missing license header by [@​a8m](https://togithub.com/a8m) in [ent/ent#3594 - dialect/sql/sqlgraph: expose standard modifier to eager-load N neighbors by [@​a8m](https://togithub.com/a8m) in [ent/ent#3603 - .github: bump go version by [@​a8m](https://togithub.com/a8m) in [ent/ent#3609 - go: update ariga.io/atlas to latest version by [@​a8m](https://togithub.com/a8m) in [ent/ent#3610 - dialect/sql: support capturing predicates in selectors by [@​a8m](https://togithub.com/a8m) in [ent/ent#3612 - dialect/sql: add predicate to check hasprefix between columns/fields by [@​a8m](https://togithub.com/a8m) in [ent/ent#3614 - doc/md: fix edge.Annotations method name typo by [@​Mozuha](https://togithub.com/Mozuha) in [ent/ent#3618 - doc: fix new project instructions in tutorial-setup.md by [@​vijal-patel](https://togithub.com/vijal-patel) in [ent/ent#3622 - entc/gen: allow catching nested tx attempt with errors.Is by [@​a8m](https://togithub.com/a8m) in [ent/ent#3636 - Remove useless wording by [@​willfaught](https://togithub.com/willfaught) in [ent/ent#3640 - mark field as cleared when clearing edges by [@​yasser-sobhy](https://togithub.com/yasser-sobhy) in [ent/ent#3651 - dialect/sql/schema: allow only table creation modification in automatic schema planning by [@​a8m](https://togithub.com/a8m) in [ent/ent#3660 - doc/website/blog: vis with atlas by [@​rotemtam](https://togithub.com/rotemtam) in [ent/ent#3688 - entc/gen: add support for MapBulkCreate by [@​a8m](https://togithub.com/a8m) in [ent/ent#3696 - Feat: added nil checks to sqlgraph constraint errors by [@​lucvankessel](https://togithub.com/lucvankessel) in [ent/ent#3701 - Update 2022-05-09-versioned-migrations-sum-file.md by [@​gedalyah-ariga](https://togithub.com/gedalyah-ariga) in [ent/ent#3685 - doc/ci - removed a reference to a hard coded go version by [@​dorav](https://togithub.com/dorav) in [ent/ent#3732 - go: update ariga.io/atlas to latest version by [@​a8m](https://togithub.com/a8m) in [ent/ent#3751 - doc/website: visualize ent/schema as erd by [@​a8m](https://togithub.com/a8m) in [ent/ent#3758 - doc/md: fix custom predicate example by [@​masseelch](https://togithub.com/masseelch) in [ent/ent#3764 - Fix example in docs/crud/#create-many by [@​adamzapasnik](https://togithub.com/adamzapasnik) in [ent/ent#3770 - entc/gen: nillable update setters for non-optional fields by [@​a8m](https://togithub.com/a8m) in [ent/ent#3788 - dialect/entsql: add helper function for creating table annotation by [@​a8m](https://togithub.com/a8m) in [ent/ent#3816 - doc/md: drop v10 support and add v15 for PostgreSQL by [@​oinume](https://togithub.com/oinume) in [ent/ent#3806 - dialect/entsql: add schema to package annotation by [@​a8m](https://togithub.com/a8m) in [ent/ent#3817 - dialect/sql/schema: add table schema by [@​a8m](https://togithub.com/a8m) in [ent/ent#3818 - dialect/atlas: initial work for multi-schema migration using atlas by [@​a8m](https://togithub.com/a8m) in [ent/ent#3821 - doc: add multi-schema migrations using atlas by [@​a8m](https://togithub.com/a8m) in [ent/ent#3825 **Full Changelog**: ent/ent@v0.12.0...v0.12.5 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/PingCAP-QE/ee-apps). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40Ni4wIiwidXBkYXRlZEluVmVyIjoiMzcuODEuMyIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [entgo.io/ent](https://togithub.com/ent/ent) | `v0.12.4` -> `v0.13.1` | [![age](https://developer.mend.io/api/mc/badges/age/go/entgo.io%2fent/v0.13.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/go/entgo.io%2fent/v0.13.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/go/entgo.io%2fent/v0.12.4/v0.13.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/go/entgo.io%2fent/v0.12.4/v0.13.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>ent/ent (entgo.io/ent)</summary> ### [`v0.13.1`](https://togithub.com/ent/ent/compare/v0.13.0...v0.13.1) [Compare Source](https://togithub.com/ent/ent/compare/v0.13.0...v0.13.1) ### [`v0.13.0`](https://togithub.com/ent/ent/compare/v0.12.5...v0.13.0) [Compare Source](https://togithub.com/ent/ent/compare/v0.12.5...v0.13.0) ### [`v0.12.5`](https://togithub.com/ent/ent/releases/tag/v0.12.5) [Compare Source](https://togithub.com/ent/ent/compare/v0.12.4...v0.12.5) #### What's Changed - doc: fix typos in paging.md by [@​a8m](https://togithub.com/a8m) in [ent/ent#3458 - entsql: added OnDelete annotation by [@​giautm](https://togithub.com/giautm) in [ent/ent#3459 - doc/md: update doc for OnDelete annotation by [@​giautm](https://togithub.com/giautm) in [ent/ent#3460 - entc/gen: avoid ambiguous column names when joining the same table by [@​a8m](https://togithub.com/a8m) in [ent/ent#3461 - entc/gen: rename <type>.Order to <type>.OrderOption by [@​a8m](https://togithub.com/a8m) in [ent/ent#3468 - dialect/sql: skip formatting qualified columns by [@​a8m](https://togithub.com/a8m) in [ent/ent#3469 - dialect/sql/sqlgraph: allow query edge fields with alias by [@​a8m](https://togithub.com/a8m) in [ent/ent#3470 - dialect/sql: add method for finding selection occurrences in queries by [@​a8m](https://togithub.com/a8m) in [ent/ent#3473 - doc: add entgql edge ordering by [@​a8m](https://togithub.com/a8m) in [ent/ent#3482 - cmd/ent: hide the --idtype flag from generate command by [@​a8m](https://togithub.com/a8m) in [ent/ent#3490 - dialect/sql/sqlgraph: replace id IN with EXISTS for HasNeighbors predicates by [@​a8m](https://togithub.com/a8m) in [ent/ent#3492 - .github/workflow: added atlas-ci for examples by [@​giautm](https://togithub.com/giautm) in [ent/ent#3496 - entc/gen: load referenced foreign-keys on eager-loading by [@​a8m](https://togithub.com/a8m) in [ent/ent#3497 - entc/gen: remove unnecessary type arguments by [@​a8m](https://togithub.com/a8m) in [ent/ent#3503 - doc/md/ci: fix broken link for ent/contrib/ci action's repo by [@​Arthur1](https://togithub.com/Arthur1) in [ent/ent#3504 - doc/md: edge ordering is supported by [@​masseelch](https://togithub.com/masseelch) in [ent/ent#3506 - entc/gen/template: fix intercept package imports by [@​masseelch](https://togithub.com/masseelch) in [ent/ent#3507 - doc: add schema inspection to getting-started page by [@​a8m](https://togithub.com/a8m) in [ent/ent#3510 - entc/load: report 'go mod' errors on schema loading by [@​a8m](https://togithub.com/a8m) in [ent/ent#3511 - .github/workflow: sync action by [@​giautm](https://togithub.com/giautm) in [ent/ent#3513 - dialect/sql/schema: allow ariga.io/atlas schema diff to be extended by [@​a8m](https://togithub.com/a8m) in [ent/ent#3514 - dialect/sql/schema: support passing schema diff options by [@​a8m](https://togithub.com/a8m) in [ent/ent#3516 - dialect/sql: add OrderByRand option by [@​a8m](https://togithub.com/a8m) in [ent/ent#3518 - ent/privacy: move some of the generated privacy helpers to ent/privacy by [@​a8m](https://togithub.com/a8m) in [ent/ent#3527 - entc/gen: avoid conflict between order by edge-count and fields end with \_count by [@​a8m](https://togithub.com/a8m) in [ent/ent#3534 - Fix typo in docs by [@​jibbolo](https://togithub.com/jibbolo) in [ent/ent#3539 - dialect/sql: expose the underlying builder of UpdateSet by [@​a8m](https://togithub.com/a8m) in [ent/ent#3541 - Fix go install instruction for atlas by [@​toshi0607](https://togithub.com/toshi0607) in [ent/ent#3526 - entc/gen: fix conflicts between fields named 'op' and Mutation.Op by [@​a8m](https://togithub.com/a8m) in [ent/ent#3550 - ent: document ent.Value returned by mutations by [@​a8m](https://togithub.com/a8m) in [ent/ent#3556 - entc/gen: avoid conflict between package and receiver names by [@​a8m](https://togithub.com/a8m) in [ent/ent#3563 - dialect/sql/sqljson: inline boolean values by [@​a8m](https://togithub.com/a8m) in [ent/ent#3570 - schema/fields: validate for slices builder by [@​masseelch](https://togithub.com/masseelch) in [ent/ent#3566 - entc/gen: quote index annotations in generated schemas by [@​a8m](https://togithub.com/a8m) in [ent/ent#3578 - entc/gen: allow naming ent/schemas as builder by [@​a8m](https://togithub.com/a8m) in [ent/ent#3593 - entc/integration: add missing license header by [@​a8m](https://togithub.com/a8m) in [ent/ent#3594 - dialect/sql/sqlgraph: expose standard modifier to eager-load N neighbors by [@​a8m](https://togithub.com/a8m) in [ent/ent#3603 - .github: bump go version by [@​a8m](https://togithub.com/a8m) in [ent/ent#3609 - go: update ariga.io/atlas to latest version by [@​a8m](https://togithub.com/a8m) in [ent/ent#3610 - dialect/sql: support capturing predicates in selectors by [@​a8m](https://togithub.com/a8m) in [ent/ent#3612 - dialect/sql: add predicate to check hasprefix between columns/fields by [@​a8m](https://togithub.com/a8m) in [ent/ent#3614 - doc/md: fix edge.Annotations method name typo by [@​Mozuha](https://togithub.com/Mozuha) in [ent/ent#3618 - doc: fix new project instructions in tutorial-setup.md by [@​vijal-patel](https://togithub.com/vijal-patel) in [ent/ent#3622 - entc/gen: allow catching nested tx attempt with errors.Is by [@​a8m](https://togithub.com/a8m) in [ent/ent#3636 - Remove useless wording by [@​willfaught](https://togithub.com/willfaught) in [ent/ent#3640 - mark field as cleared when clearing edges by [@​yasser-sobhy](https://togithub.com/yasser-sobhy) in [ent/ent#3651 - dialect/sql/schema: allow only table creation modification in automatic schema planning by [@​a8m](https://togithub.com/a8m) in [ent/ent#3660 - doc/website/blog: vis with atlas by [@​rotemtam](https://togithub.com/rotemtam) in [ent/ent#3688 - entc/gen: add support for MapBulkCreate by [@​a8m](https://togithub.com/a8m) in [ent/ent#3696 - Feat: added nil checks to sqlgraph constraint errors by [@​lucvankessel](https://togithub.com/lucvankessel) in [ent/ent#3701 - Update 2022-05-09-versioned-migrations-sum-file.md by [@​gedalyah-ariga](https://togithub.com/gedalyah-ariga) in [ent/ent#3685 - doc/ci - removed a reference to a hard coded go version by [@​dorav](https://togithub.com/dorav) in [ent/ent#3732 - go: update ariga.io/atlas to latest version by [@​a8m](https://togithub.com/a8m) in [ent/ent#3751 - doc/website: visualize ent/schema as erd by [@​a8m](https://togithub.com/a8m) in [ent/ent#3758 - doc/md: fix custom predicate example by [@​masseelch](https://togithub.com/masseelch) in [ent/ent#3764 - Fix example in docs/crud/#create-many by [@​adamzapasnik](https://togithub.com/adamzapasnik) in [ent/ent#3770 - entc/gen: nillable update setters for non-optional fields by [@​a8m](https://togithub.com/a8m) in [ent/ent#3788 - dialect/entsql: add helper function for creating table annotation by [@​a8m](https://togithub.com/a8m) in [ent/ent#3816 - doc/md: drop v10 support and add v15 for PostgreSQL by [@​oinume](https://togithub.com/oinume) in [ent/ent#3806 - dialect/entsql: add schema to package annotation by [@​a8m](https://togithub.com/a8m) in [ent/ent#3817 - dialect/sql/schema: add table schema by [@​a8m](https://togithub.com/a8m) in [ent/ent#3818 - dialect/atlas: initial work for multi-schema migration using atlas by [@​a8m](https://togithub.com/a8m) in [ent/ent#3821 - doc: add multi-schema migrations using atlas by [@​a8m](https://togithub.com/a8m) in [ent/ent#3825 **Full Changelog**: ent/ent@v0.12.0...v0.12.5 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/infratographer/x). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xMjEuMCIsInVwZGF0ZWRJblZlciI6IjM3LjI2OS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
No description provided.