Skip to content

Limit the heap size of Cassandra and ES docker containers for tests#469

Merged
Arsnael merged 2 commits into
apache:masterfrom
Arsnael:docker-memory-opts
Jun 3, 2021
Merged

Limit the heap size of Cassandra and ES docker containers for tests#469
Arsnael merged 2 commits into
apache:masterfrom
Arsnael:docker-memory-opts

Conversation

@Arsnael
Copy link
Copy Markdown
Contributor

@Arsnael Arsnael commented Jun 1, 2021

Well as I'm getting tired of having the RAM of my laptop burning to the ground half of the time I run integration tests on James, I decided to take a closer look.

Cassandra by default is deployed with 4Go RAM of heap. ES is deployed with 1Go (well I observed those when running locally)

Now the interesting thing is that some work has been done a while ago to limit the size of Cassandra to 650M for the tests, but the env variable setup was wrong. Thus we were still living with a 4Go memory eater Cassandra.

I fixed that, and while I was at it, I'm taking a shot at ES too (half, from 1Go to 512M). The rational being, if it passes with Cassandra limited like this, might be as well with ES.

I would like to have a good bunch of CI runs though first to be sure it does not bring instability to the build. If it does, I will change those values and go up gradually until we got a stable build.

If any feedback on this or proposition, don't hesitate :)

@Arsnael Arsnael self-assigned this Jun 1, 2021
@Arsnael
Copy link
Copy Markdown
Contributor Author

Arsnael commented Jun 1, 2021

First build: green :)

@Arsnael
Copy link
Copy Markdown
Contributor Author

Arsnael commented Jun 1, 2021

Tsss not sure related

09:54:58,815 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.2:test (default-test) on project james-server-webadmin-data: There was a timeout or other error in the fork -> [Help 1]

@Arsnael
Copy link
Copy Markdown
Contributor Author

Arsnael commented Jun 2, 2021

One green and one red:

16:18:00,883 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.2:test (default-test) on project james-server-webadmin-mailbox: There was a timeout or other error in the fork -> [Help 1]

Not related either...

@Arsnael
Copy link
Copy Markdown
Contributor Author

Arsnael commented Jun 2, 2021

06:29:22,974 [ERROR] Tests run: 6, Failures: 0, Errors: 1, Skipped: 1, Time elapsed: 166.506 s <<< FAILURE! - in org.apache.james.webadmin.integration.rabbitmq.vault.RabbitMQDeletedMessageVaultIntegrationTest
06:29:22,976 [ERROR] vaultEndpointShouldRestoreImapDeletedMailbox{GuiceJamesServer}  Time elapsed: 128.653 s  <<< ERROR!
org.awaitility.core.ConditionTimeoutException: 
Assertion condition defined as a lambda expression in org.apache.james.webadmin.integration.vault.DeletedMessageVaultIntegrationTest 
Expected size: 1 but was: 0 in:
[] within 2 minutes.
Caused by: java.lang.AssertionError: 

Expected size: 1 but was: 0 in:
[]

Hmmm... Not sure what to think of this one. Well I'm gonna raise up a bit to be sure

@Arsnael
Copy link
Copy Markdown
Contributor Author

Arsnael commented Jun 3, 2021

2 greens builds :) Ok I squash the fixups and if I get an other green I merge

Arsnael added 2 commits June 3, 2021 08:53
It was not working, as in testcontainers we know we define an ENV variable by using the `env()` method, so declaring "ENV" in the key is a mistake. It's been a long time this is intended to run as 650M heap memory size for Cassandra, while in fact it's been the default 4Go running meanwhile.
The default is 1Go. I'm betting on half here (512M).
@Arsnael Arsnael force-pushed the docker-memory-opts branch from acf5ac9 to adb7a56 Compare June 3, 2021 01:54
@Arsnael Arsnael merged commit 9f7425e into apache:master Jun 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants