Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DefaultQueryCache entrySet inefficient result building #14177

Closed
TomaszGaweda opened this issue Nov 27, 2018 · 3 comments
Closed

DefaultQueryCache entrySet inefficient result building #14177

TomaszGaweda opened this issue Nov 27, 2018 · 3 comments

Comments

@TomaszGaweda
Copy link
Contributor

@TomaszGaweda TomaszGaweda commented Nov 27, 2018

Hi,

DefaultQueryCache handles queries to query caches. However, it's performance is degradated because of current flow in case of index aware query:

  1. get results from index
  2. if not null -> check isEmpty (which traverse whole result set!)
  3. again ask for query.size() - here it's not so bad, because size result is cached after point 2
  4. again traverse result set and add to final HashSet

This should be much faster: if query is not null, isEmpty should look only if any element was found; what's more, we can at first construct resultingSet, without isEmpty and size calls, because this eager optimization causes in real life huge performance degradation.

@ahmetmircik
Copy link
Member

@ahmetmircik ahmetmircik commented Nov 28, 2018

Hi @TomaszGaweda,

if not null -> check isEmpty (which traverse whole result set!)

Can you please give reference to this issue? Where does it happen in the code?

@TomaszGaweda
Copy link
Contributor Author

@TomaszGaweda TomaszGaweda commented Nov 28, 2018

@ahmetmircik
Hi, thanks for the quick response :)

Please look at DefaultQueryCache.entrySet: https://github.com/hazelcast/hazelcast/blob/master/hazelcast/src/main/java/com/hazelcast/map/impl/querycache/subscriber/DefaultQueryCache.java#L364

Algorithm is currently as follows:

  1. indexes.query returns i.e. AndPredicate (most commonly used in a predicate tree root).
  2. if result is not null, we call query.isEmpty(), which falls back to AbstractCollection.isEmpty. This method is implemented as size() == 0.
  3. Size is implemented as follows: https://github.com/hazelcast/hazelcast/blob/master/hazelcast/src/main/java/com/hazelcast/query/impl/AndResultSet.java#L146
  4. ok, we have some entries - then we call size() again to create HashSet of some size, this time result is cached, so no big problem.
  5. Next we iterate using for-each loop over the result of indexes.query -> this will again create AndResultSet$It and iterate whole Query Cache.

In case of "big" Query Caches (~2000) entries and compound predicates it can consume a lot of time

@ahmetmircik
Copy link
Member

@ahmetmircik ahmetmircik commented Dec 10, 2018

should be fixed by #14229

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants