Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NEW: ExpressionList has clear() method #2941

Merged
merged 1 commit into from
Feb 1, 2023

Conversation

rPraml
Copy link
Contributor

@rPraml rPraml commented Jan 23, 2023

We had an issue in our application, where we had a (complex) query and we wanted to get only a subset of models for paging in a web app, where we had the IDs. Here is a use case, why we need this and cannot rebuild the query.

Use case

class Pager<T> {
  final List<UUID> ids;
  final Query<T> query;
  final int PAGE_SIZE = 20;

  Pager(Query<T> query) {
    this.query = query;
    this.ids = query.findIds();  // fetch IDs in correct order;
  }
  
  List<T> getPage(int page) {
     return query.copy().where().idIn(ids.subList(page * PAGE_SIZE, PAGE_SIZE));
  }
}

// construct pager with a query for the user's search result (where & fetch)
Pager<Customer> pager = new Pager(DB.find(Customer.class).fetch("address").where().like("name", "%foo%").query())
// (expensive) searching is done one time in the constructor, the pager is then hold in the http-session
result = pager.getPage(0);
// subsequent web requests will retrive more pages, that should be found fast, because we already have the IDs
result = pager.getPage(1);
...

We have really slow queries here (complex searches, especially in non indexed fields could take up several seconds), but then we have the whole search result in memory (we store only the IDs, so not to run in an OOM), then we display the results in the browser.
In theory, the ID queries should be superfast, but not always, Especially on MariaDB, we discvered some cases, that the query-optimizer not always starts with the id in (?, ?) clause, but with some other conditions, which makes the query slow.

So we removed the old where conditions with

     return query.copy().clear().where().idIn(ids.subList(page * PAGE_SIZE, PAGE_SIZE));

and the query is fast again.
(Yes, we know, that data can change during paging, but this is not a problem in our use case)

@rbygrave rbygrave added this to the 13.11.3 milestone Feb 1, 2023
@rbygrave rbygrave merged commit 4aa3ca6 into ebean-orm:master Feb 1, 2023
@rPraml rPraml deleted the expressionlist-clear branch August 10, 2023 14:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants