Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JAMES-3769 Allow defining search overrides #993

Merged
merged 10 commits into from
May 23, 2022

Conversation

chibenwa
Copy link
Contributor

@chibenwa chibenwa commented May 10, 2022

Motivation

IMAP SEARCH command is used by some clients to resynchronize data and are often performed queries.

The use of ElasticSearch pose several issues:

  • Long lasting queries: it's not uncommon to see some search hanged for 60+ seconds
  • Many results returned kicking in ES driver paging
  • ElasticSearch is not the primary data store and can get out of date, reindexed. Indexing is a background process. Acceptable for user search, not really for resynchronisation.

The scope of this ticket is to allow execution of well chosen search requests against existing Cassandra datastructures. Optional and extensible.

TODO

  • Unit tests for search overrides
  • Documentation
  • Unit test for search override loading
  • JIRA ticket and ADR

@chibenwa chibenwa self-assigned this May 10, 2022
@chibenwa chibenwa force-pushed the search-overrides branch 2 times, most recently from 6dc9ffa to a4acbc4 Compare May 17, 2022 11:04
@chibenwa chibenwa changed the title WIP Allow defining search overrides JAMES-3769 Allow defining search overrides May 18, 2022
@chibenwa chibenwa marked this pull request as ready for review May 19, 2022 06:30
@chibenwa
Copy link
Contributor Author

Screenshot from 2022-05-19 13-31-01

Waaaay less ElasticSearch business going on without introducing too much new Cassandra business.

I observed a 10% resource consumption on a 18.000 users gatling run.

No performance degradation in the gatling runs I saw so far.

Copy link
Contributor

@Arsnael Arsnael left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this empty file server/container/guice/elasticsearch/src/test/resources/recursive/.keep for btw?

src/adr/0054-elasticsearch-search-overrides.md Outdated Show resolved Hide resolved
@chibenwa
Copy link
Contributor Author

What is this empty file server/container/guice/elasticsearch/src/test/resources/recursive/.keep for btw?

Adding the folder into the git structure which would otherwise not be kept.

@chibenwa chibenwa merged commit 6fed452 into apache:master May 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants