JAMES-3266 Offer an option to disable ElasticSearch in Distributed James product #3425

chibenwa · 2020-06-04T04:57:03Z

No description provided.

chibenwa · 2020-06-04T10:28:17Z

test this please

.../guice/cassandra-guice/src/main/java/org/apache/james/CassandraJamesServerConfiguration.java

server/container/guice/cassandra-guice/src/main/java/org/apache/james/SearchModuleChooser.java

...andra-rabbitmq-guice/src/main/java/org/apache/james/CassandraRabbitMQJamesConfiguration.java

chibenwa · 2020-06-05T03:24:01Z


[24c824870006ef81b1843e13306d8a7a30bbd047] [ERROR]   ESReporterTest.timeMetricsShouldBeReportedWhenImapCommandsReceived:129 » ConditionTimeout
[24c824870006ef81b1843e13306d8a7a30bbd047] [ERROR]   ESReporterTest.timeMetricsShouldBeReportedWhenJmapRequestsReceived:151 » ConditionTimeout

ieugen · 2020-06-23T09:05:58Z

We are working on documentation and every feature should be documented.
Could you please add some documentation on how to use this feature?

Some questions and hints, take them or leave them:

The documentation should be about how to configure IMAP, IMAP search in James
Is this feature meant for an administrator or a developer?
What are the steps that I have to take to OPT-IN or OPT-OUT of this feature
You could focus just on this one and we will add the others in time.

chibenwa · 2020-06-23T10:11:13Z

This is meaningful, I'll handle these documentation topics.

chibenwa · 2020-06-24T02:10:02Z

Unrelated

test this please

chibenwa · 2020-06-24T06:36:45Z

Unrelated

test this please

chibenwa · 2020-06-25T01:34:09Z

[76fd1a1] [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.2:test (default-test) on project james-server-mailets-integration-testing: There was a timeout or other error in the fork -> [Help 1]

test this please

mbaechler

I think this PR deserves some more love before integrating it.

server/container/guice/cassandra-guice/src/main/java/org/apache/james/SearchModuleChooser.java

mbaechler · 2020-06-26T07:49:21Z

.../guice/cassandra-guice/src/main/java/org/apache/james/CassandraJamesServerConfiguration.java

+import com.github.fge.lambdas.Throwing;
+
+public class CassandraJamesServerConfiguration implements Configuration {
+    public static class Builder {


it looks like a copy/paste of existing code. Why do we have to do that? I don't care about duplicating methods but there's some logic too that could be delegated to another object, right?

I make a simple refactor here, please take a look
nvduc91@1d3078f

I do have a different proposal #3531

mbaechler · 2020-06-26T07:51:10Z

...container/guice/cassandra-guice/src/main/java/org/apache/james/CassandraJamesServerMain.java

@@ -167,7 +160,7 @@
    );

    public static void main(String[] args) throws Exception {
-        Configuration configuration = Configuration.builder()
+        CassandraJamesServerConfiguration configuration = CassandraJamesServerConfiguration.builder()


How CassandraJamesServerConfiguration is different from Configuration?

I need an explanation here.

Are you asking for...

Configuration configuration = CassandraJamesServerConfiguration.builder() ...

Sorry but I asked a question that is quite simple: we create a class B that is very similar to what we did with class A so I ask what is the difference between A and B.
You ask me if I want A a = B.builder()....build() but I have no idea, how does it relate to me initial question?

It differs from the module choices offered

Ok, so we may choose a naming pattern (and maybe some javadoc) that allow to discover which Configuration type one need. Could it be an inner class of the server we want to instantiate? I think that would make it more discoverable.

mbaechler · 2020-06-26T07:53:01Z

server/container/guice/cassandra-guice/src/main/java/org/apache/james/SearchConfiguration.java

+    }
+
+    static SearchConfiguration from(Configuration configuration) {
+        if (configuration.getBoolean("enabled", true)) {


This configuration looks really bad to me: to have scanning search we have to go to elasticsearch configuration and disable it?

Please propose an alternative.

like that #3425 (comment) ? Did you read my comments before asking ?

i dont understand why i must disabled ES to get scanning?

either you get scanning, either ES. Scanning is not backed at all by ES.

mbaechler · 2020-06-26T07:54:11Z

server/container/guice/cassandra-guice/src/main/java/org/apache/james/SearchModuleChooser.java

+        switch (searchConfiguration.getImplementation()) {
+            case ElasticSearch:
+                return ImmutableList.of(
+                    new ElasticSearchClientModule(),


can't we have a single module for each implementation?

Sure, we can create a meta module

are we agree on that or not?

...ntainer/guice/cassandra-guice/src/test/java/org/apache/james/WithCassandraBlobStoreTest.java

mbaechler · 2020-06-26T07:57:02Z

...er/guice/cassandra-rabbitmq-guice/src/test/java/org/apache/james/WithScanningSearchTest.java

+import org.junit.jupiter.api.extension.ExtendWith;
+
+@ExtendWith(WithScanningSearchExtension.class)
+class WithScanningSearchTest implements JmapJamesServerContract, MailsShouldBeWellReceived, JamesServerContract {


how much time it adds to the testsuite?

Not much. Containers are already started, tables populated. Around a minute.

src/site/xdoc/server/config-elasticsearch.xml

…mes product

src/site/xdoc/server/config-elasticsearch.xml

chibenwa · 2020-07-03T02:17:54Z


[964dc38bcbe1e256c18501705e8044763660c538] [ERROR]   RabbitMQDeletedMessageVaultIntegrationTest>DeletedMessageVaultIntegrationTest.vaultExportShouldExportZipContainsVaultMessagesToShareeWhenImapDeletedMailbox:543 
[964dc38bcbe1e256c18501705e8044763660c538] Expecting zipFile to contains 1 entries but actually contains (0) entries

Co-authored-by: Raphaël Ouazana <rouazana@linagora.com>

…roperties configuration file

chibenwa

👍

chibenwa · 2020-07-09T02:02:39Z

RabbitMQDeletedMessageVaultIntegrationTest>DeletedMessageVaultIntegrationTest.vaultExportShouldExportZipContainsVaultMessagesToShareeWhenImapDeletedMailbox:543 
[f64e4486b7bf41e2acdd510ea4fcdd850ca27bef] Expecting zipFile to contains 1 entries but actually contains (0) entries

test this please

rouazana · 2020-07-09T07:48:55Z

[f64e4486b7bf41e2acdd510ea4fcdd850ca27bef] [ERROR] Failures: 
[f64e4486b7bf41e2acdd510ea4fcdd850ca27bef] [ERROR]   RecomputeUserFastViewProjectionItemsRequestToTaskTest.postShouldFailUponBadUsername:247 1 expectation failed.
[f64e4486b7bf41e2acdd510ea4fcdd850ca27bef] JSON path details doesn't match.
[f64e4486b7bf41e2acdd510ea4fcdd850ca27bef] Expected: is "The username should not contain multiple domain delimiter."
[f64e4486b7bf41e2acdd510ea4fcdd850ca27bef]   Actual: The username should not contain multiple domain delimiter. Value: bad@bad@bad

strange, master is broken or merge issue?

test this please

Arsnael · 2020-07-09T08:02:33Z

[f64e4486b7bf41e2acdd510ea4fcdd850ca27bef] [ERROR] Failures: 
[f64e4486b7bf41e2acdd510ea4fcdd850ca27bef] [ERROR]   RecomputeUserFastViewProjectionItemsRequestToTaskTest.postShouldFailUponBadUsername:247 1 expectation failed.
[f64e4486b7bf41e2acdd510ea4fcdd850ca27bef] JSON path details doesn't match.
[f64e4486b7bf41e2acdd510ea4fcdd850ca27bef] Expected: is "The username should not contain multiple domain delimiter."
[f64e4486b7bf41e2acdd510ea4fcdd850ca27bef]   Actual: The username should not contain multiple domain delimiter. Value: bad@bad@bad

strange, master is broken or merge issue?

Possibly a merge issue I think. Let me check

Arsnael · 2020-07-09T08:10:25Z

=> #3565

chibenwa · 2020-07-10T02:08:02Z

test this please

Arsnael · 2020-07-15T04:49:28Z

Merged

chibenwa added the POC Proof of Concept. Feedback is welcomed but don't expect high code quality... label Jun 4, 2020

chibenwa added this to the Polish 2020-06 milestone Jun 4, 2020

chibenwa self-assigned this Jun 4, 2020

Arsnael reviewed Jun 4, 2020

View reviewed changes

chibenwa force-pushed the choose-es branch from abd4c2b to fee39e0 Compare June 5, 2020 07:46

chibenwa force-pushed the choose-es branch from 32c5b1a to 76fd1a1 Compare June 23, 2020 10:24

rouazana changed the title ~~POC Offer an option to disable ElasticSearch in Distributed James pro…~~ Offer an option to disable ElasticSearch in Distributed James product Jun 23, 2020

rouazana approved these changes Jun 23, 2020

View reviewed changes

chibenwa changed the title ~~Offer an option to disable ElasticSearch in Distributed James product~~ JAMES-3266 Offer an option to disable ElasticSearch in Distributed James product Jun 24, 2020

chibenwa removed the POC Proof of Concept. Feedback is welcomed but don't expect high code quality... label Jun 24, 2020

Arsnael approved these changes Jun 24, 2020

View reviewed changes

mbaechler reviewed Jun 26, 2020

View reviewed changes

chibenwa mentioned this pull request Jun 29, 2020

Distributed James: Disable ElasticSearch #3490

Closed

chibenwa linked an issue Jun 29, 2020 that may be closed by this pull request

Distributed James: Disable ElasticSearch #3490

Closed

chibenwa modified the milestones: Polish 2020-06, Sprint - common - 1 Jun 29, 2020

chibenwa and others added 3 commits July 1, 2020 17:08

JAMES-3266 Offer an option to disable ElasticSearch in Distributed Ja…

caa5c93

…mes product

JAMES-3266 Document ElasticSearch opt-in/opt-out

7d71e5f

fixup! JAMES-3266 Document ElasticSearch opt-in/opt-out

964dc38

chibenwa force-pushed the choose-es branch from 76fd1a1 to 964dc38 Compare July 2, 2020 08:02

rouazana approved these changes Jul 2, 2020

View reviewed changes

src/site/xdoc/server/config-elasticsearch.xml Outdated Show resolved Hide resolved

src/site/xdoc/server/config-elasticsearch.xml Outdated Show resolved Hide resolved

Update src/site/xdoc/server/config-elasticsearch.xml

0522506

Co-authored-by: Raphaël Ouazana <rouazana@linagora.com>

JAMES-3266 Move disable ElasticSearch option into a separate search.p…

f64e448

…roperties configuration file

chibenwa commented Jul 9, 2020

View reviewed changes

Arsnael approved these changes Jul 9, 2020

View reviewed changes

rouazana approved these changes Jul 9, 2020

View reviewed changes

chibenwa added the waiting_merge We are about to merge this! label Jul 15, 2020

Arsnael closed this Jul 15, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

JAMES-3266 Offer an option to disable ElasticSearch in Distributed James product #3425

JAMES-3266 Offer an option to disable ElasticSearch in Distributed James product #3425

chibenwa commented Jun 4, 2020 •

edited by rouazana

chibenwa commented Jun 4, 2020

chibenwa commented Jun 5, 2020

ieugen commented Jun 23, 2020

chibenwa commented Jun 23, 2020

chibenwa commented Jun 24, 2020

chibenwa commented Jun 24, 2020

chibenwa commented Jun 25, 2020

mbaechler left a comment

mbaechler Jun 26, 2020

nvduc91 Jul 1, 2020

chibenwa Jul 1, 2020

mbaechler Jun 26, 2020

chibenwa Jun 26, 2020

mbaechler Jul 1, 2020

chibenwa Jul 1, 2020

mbaechler Jul 2, 2020

mbaechler Jun 26, 2020

chibenwa Jun 26, 2020

mbaechler Jul 1, 2020

nvduc91 Jul 2, 2020

rouazana Jul 2, 2020

mbaechler Jun 26, 2020

chibenwa Jun 26, 2020

nvduc91 Jul 2, 2020

mbaechler Jun 26, 2020

chibenwa Jun 26, 2020

chibenwa commented Jul 3, 2020

chibenwa left a comment

chibenwa commented Jul 9, 2020

rouazana commented Jul 9, 2020

Arsnael commented Jul 9, 2020

Arsnael commented Jul 9, 2020

chibenwa commented Jul 10, 2020

Arsnael commented Jul 15, 2020

JAMES-3266 Offer an option to disable ElasticSearch in Distributed James product #3425

JAMES-3266 Offer an option to disable ElasticSearch in Distributed James product #3425

Conversation

chibenwa commented Jun 4, 2020 • edited by rouazana

chibenwa commented Jun 4, 2020

chibenwa commented Jun 5, 2020

ieugen commented Jun 23, 2020

chibenwa commented Jun 23, 2020

chibenwa commented Jun 24, 2020

chibenwa commented Jun 24, 2020

chibenwa commented Jun 25, 2020

mbaechler left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chibenwa commented Jul 3, 2020

chibenwa left a comment

Choose a reason for hiding this comment

chibenwa commented Jul 9, 2020

rouazana commented Jul 9, 2020

Arsnael commented Jul 9, 2020

Arsnael commented Jul 9, 2020

chibenwa commented Jul 10, 2020

Arsnael commented Jul 15, 2020

chibenwa commented Jun 4, 2020 •

edited by rouazana