Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

JAMES-3492 #328

Closed
wants to merge 29 commits into from
Closed

JAMES-3492 #328

wants to merge 29 commits into from

Conversation

andrevka
Copy link

includes #318
Use elasticsearch v7 modules everywhere

LanKhuat and others added 20 commits March 9, 2021 10:29
@LanKhuat
JAMES-3492 Adapt ElasticSearchIntegrationTest
6719dcc
@LanKhuat
JAMES-3492 Replace deprecated CreateIndexRequest
451af03
@LanKhuat
JAMES-3492 Remove mapping check
172e271 
The check returns status 200 in ElasticSearch 7, preventing the usage of James custom mapping when creating index.
@LanKhuat
JAMES-3492 Migrate to Java time date format
bffd4c4
@LanKhuat
JAMES-3492 Search no longer need to fetch individual fields
79f7181 
The fields are stored within the document and can be fetched with storedFields option.
@LanKhuat
JAMES-3492 Adapt other implementations of AbstractMessageSearchIndexTest
6821807
@LanKhuat
JAMES-3492 Adapt ElasticSearchSearcherTest
4fdf070
@LanKhuat
JAMES-3492 Mapping should be created at the same time as index
3a9ab02
@LanKhuat
JAMES-3492 Check Mapping existence by GetMappingsAPI
65f2a12
@LanKhuat
JAMES-3492 Add missing dependency
99be71d
@LanKhuat
JAMES-3492 Adapt ElasticSearchQuotaMailboxListenerTest
f8757dd
@LanKhuat
JAMES-3492 Use assertion with Awaitility in QuotaSearcherContract
63d9303
@LanKhuat
JAMES-3492 Remove NodeMappingFactory & tests
55d0299
JAMES-3492, switch to elasticsearch 7 in guice modules
20ffa84
Merge remote-tracking branch 'lankhuat/elastic-search-7-mailbox-test'…
0ee7303 
… into JAMES-3492-guice
JAMES-3492, switch to elasticsearch 7 in poms
047455f
JAMES-3492, disable failing tests
3a22278
JAMES-3492, change ElasticSearchStartUpCheck.java to use v7
e984390
JAMES-3492, fix tests, wait for elasticsearch flushes to complete for…
ca156cf 
… test assertions
JAMES-3492, applied sortpom:sort
ba2ada9
@mbaechler
Copy link
Member 

18:45:39,620 [INFO] Apache James :: Mailbox :: ElasticSearch ........... FAILURE [  8.768 s]

The module doesn't compile right now

LanKhuat
LanKhuat reviewed Mar 11, 2021
View reviewed changes
Copy link
Contributor

@LanKhuat LanKhuat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you squash all the commits from #318 along with 0ee7303 commit?

pom.xml Outdated Show resolved Hide resolved
...min/webadmin-mailbox/src/test/java/org/apache/james/webadmin/routes/UserQuotaRoutesTest.java Outdated Show resolved Hide resolved
...min/webadmin-mailbox/src/test/java/org/apache/james/webadmin/routes/UserQuotaRoutesTest.java Outdated Show resolved Hide resolved
...min/webadmin-mailbox/src/test/java/org/apache/james/webadmin/routes/UserQuotaRoutesTest.java Outdated Show resolved Hide resolved
...min/webadmin-mailbox/src/test/java/org/apache/james/webadmin/routes/UserQuotaRoutesTest.java Outdated Show resolved Hide resolved
...min/webadmin-mailbox/src/test/java/org/apache/james/webadmin/routes/UserQuotaRoutesTest.java Outdated Show resolved Hide resolved
...min/webadmin-mailbox/src/test/java/org/apache/james/webadmin/routes/UserQuotaRoutesTest.java Outdated Show resolved Hide resolved
...min/webadmin-mailbox/src/test/java/org/apache/james/webadmin/routes/UserQuotaRoutesTest.java Outdated Show resolved Hide resolved
...min/webadmin-mailbox/src/test/java/org/apache/james/webadmin/routes/UserQuotaRoutesTest.java Outdated Show resolved Hide resolved
chibenwa
chibenwa reviewed Mar 11, 2021
View reviewed changes
server/container/guice/cassandra-guice/src/test/java/org/apache/james/ESReporterTest.java
@@ -47,6 +47,7 @@
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Disabled;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's deprecate ES7 metric reporting and only rely on the prometheus endpoints, rather that some custom, unmaintained reporters?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems reasonable to me to rely solely on prometheus

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall i remove it from the project?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can raise the topic on server-dev mailing list, explain that it do not work with ES v6, that the underlying library is unmaintained (gasp) and that we have a pronmetheus alternative.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll give my 👍 there.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://www.mail-archive.com/server-dev@james.apache.org/msg69816.html

@chibenwa
Copy link
Contributor

I'm surprised JMAP integration tests did not break...

@andrevka
Copy link
Author

Where should the elasticsearch update instructions be added?

@chibenwa
Copy link
Contributor

In /upgrade-instructions.md

chibenwa
chibenwa reviewed Mar 12, 2021
View reviewed changes
upgrade-instructions.md Outdated Show resolved Hide resolved
@chibenwa
Copy link
Contributor

Shall we rename packages and maven modules to remove the v7 suffix?

@chibenwa
Copy link
Contributor

Nice job BTW!

@chibenwa
Copy link
Contributor

We need a documentation update as well...

dockerfiles/run/docker-compose-branch-master.yml
dockerfiles/run/docker-compose.yml
README.adoc
docs/modules/servers/pages/distributed/run.adoc
docs/modules/servers/pages/distributed/operate/guide.adoc
docs/modules/servers/pages/distributed/operate/logging.adoc
docs/modules/servers/pages/distributed/run-docker.adoc
docs/modules/servers/pages/distributed/configure/elasticsearch.adoc
src/site/xdoc/server/quick-start-cassandra.xml
src/site/xdoc/server/config-elasticsearch.xml
src/site/markdown/server/install/guice-cassandra.md
src/site/markdown/server/install/guice-cassandra-rabbitmq-swift.md

Arsnael
Arsnael approved these changes Mar 12, 2021
View reviewed changes
Copy link
Contributor

@Arsnael Arsnael left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, huge thanks :)

@chibenwa
Copy link
Contributor

Hello!

Huge thanks for your work on this topic. I did just merge this PR.

I will then start working on the 3.6.0 release from tomorrow.

Cheers,

@Arsnael Arsnael closed this Mar 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@chibenwa chibenwa chibenwa approved these changes

@Arsnael Arsnael Arsnael approved these changes

@LanKhuat LanKhuat LanKhuat approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
5 participants
@andrevka @mbaechler @chibenwa @Arsnael @LanKhuat