UNOMI-921: Replace elasticsearch-maven-plugin with Docker-based Elasticsearch in integration tests#759
Open
sergehuber wants to merge 2 commits into
Open
UNOMI-921: Replace elasticsearch-maven-plugin with Docker-based Elasticsearch in integration tests#759sergehuber wants to merge 2 commits into
sergehuber wants to merge 2 commits into
Conversation
…icsearch in integration tests Implements the core configuration switch from UNOMI-921: https://issues.apache.org/jira/browse/UNOMI-921 Replace the com.github.alexcojocaru:elasticsearch-maven-plugin (binary download + forked JVM) with io.fabric8:docker-maven-plugin in the elasticsearch profile of itests, mirroring how the opensearch profile already runs OpenSearch in a Docker container. itests/pom.xml (elasticsearch profile) * Add an <elasticsearch.port>9400</elasticsearch.port> property and pass it through the failsafe systemPropertyVariables so tests resolve the HTTP port from a single source (unchanged from the previous 9400). * Replace the elasticsearch-maven-plugin block with a docker-maven-plugin block that runs docker.elastic.co/elasticsearch/elasticsearch:${elasticsearch.test.version}, binds target/snapshots_repository to /tmp/snapshots_repository, and waits on the HTTP port before the integration-test phase. Heap aligned to 8GB (-Xms8g -Xmx8g) to match the OpenSearch configuration and the ES 9 recommendation. Discovery=single-node, replicas=0, xpack.ml and xpack.security disabled. * Container lifecycle matches OpenSearch exactly: pre-integration-test runs stop+remove then start (with showLogs); post-integration-test runs stop only -- container is kept around for inspection. * Add a chmod -R ugo+rwx on snapshots_repository in the antrun unzip step: the ES container runs as UID 1000, so on Linux CI the bind-mounted snapshot repo otherwise hits access_denied during repository verify. pom.xml (root) * Declare <docker-maven-plugin.version>0.48.0</docker-maven-plugin.version> and add the pluginManagement entry so the elasticsearch profile (and any future user of the plugin) inherits a single version. Scope kept minimal for the PR #757 stack split: only the test infrastructure switch lives here. The follow-up UNOMI-921 acceptance items below ship in the platform PR (P) once it lands: * Remove BaseIT.fixDefaultTemplateIfNeeded() and the call in checkSearchEngine() (no longer needed with Docker). * Migrate16xToCurrentVersionIT: replace hardcoded ES_BASE_URL = "http://localhost:9400" with dynamic getSearchPort(). * Drop the comments referring to the elasticsearch-maven-plugin template-override workaround. See docs/PR-757-stack-extraction-tracker.md for the full split plan and how this PR fits in the stack.
df1dc2a to
a2e008c
Compare
| </ports> | ||
| <env> | ||
| <discovery.type>single-node</discovery.type> | ||
| <ES_JAVA_OPTS>-Xms8g -Xmx8g -Dcluster.default.index.settings.number_of_replicas=0</ES_JAVA_OPTS> |
| </ports> | ||
| <env> | ||
| <discovery.type>single-node</discovery.type> | ||
| <ES_JAVA_OPTS>-Xms8g -Xmx8g -Dcluster.default.index.settings.number_of_replicas=0</ES_JAVA_OPTS> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked PR (merge order)
masterUNOMI-itests-es-dockerThis PR is stacked: it is the bottom of a chain. Merge into
masterfirst; downstream branches that build on top will resync onto the newmastertip.For more information (each PR targets the branch below until the bottom merges): https://github.github.com/gh-stack/introduction/overview/
JIRA: https://issues.apache.org/jira/browse/UNOMI-921 — Replace elasticsearch-maven-plugin with Docker-based Elasticsearch Instance in integration tests.
Why this change
The
elasticsearchprofile initests/pom.xmlcurrently usescom.github.alexcojocaru:elasticsearch-maven-pluginto download an Elasticsearch tarball and run it as a forked JVM during the Maven build. This approach:default_templatethat overrides user templates on ES 8/9 — currently worked around byBaseIT.fixDefaultTemplateIfNeeded(),opensearchprofile (sameitests/pom.xml), which already uses Docker, andAligning Elasticsearch on the same Docker-based approach the
opensearchprofile uses removes the template-override workaround, eliminates the binary download, and makes both search-engine profiles symmetric.What changed
itests/pom.xml—elasticsearchprofile<elasticsearch.port>9400</elasticsearch.port>and surface it to Failsafe viasystemPropertyVariables, so tests resolve the HTTP port from a single property (value unchanged:9400).elasticsearch-maven-pluginexecution with anio.fabric8:docker-maven-pluginexecution running:docker.elastic.co/elasticsearch/elasticsearch:${elasticsearch.test.version}9200→ host9400-Xms8g -Xmx8g— aligned with OpenSearch and the ES 9 sizing guidance (was4g)discovery.type=single-node,cluster.default.index.settings.number_of_replicas=0,xpack.ml.enabled=false,xpack.security.enabled=false,cluster.routing.allocation.disk.threshold_enabled=false,path.repo=/tmp/snapshots_repository${project.build.directory}/snapshots_repository→/tmp/snapshots_repositorypre-integration-testrunsstop+remove(idempotent cleanup) thenstartwithshowLogs;post-integration-testrunsstoponly — the container is kept around (not removed) so logs can be inspected after a failed run.chmod -R ugo+rwxontarget/snapshots_repositoryin the antrununzipstep. The official ES image runs as UID 1000, so on Linux CI the bind-mounted snapshot repository otherwise hitsaccess_deniedduringrepository verifyoperations.pom.xml(root)<docker-maven-plugin.version>0.48.0</docker-maven-plugin.version>and add the corresponding<pluginManagement>entry so theelasticsearchprofile (and any future user of the plugin) inherits a single version.Verification
mvn -P integration-tests,elasticsearch -DskipITs=false verifyonitests: container starts on9400, all integration tests run against it, container is stopped (but not removed) after the run.docker ps -a --filter name=unomi-itests-elasticsearchafter the run; logs accessible viadocker logs.Follow-ups (tracked under the same JIRA)
These items are part of UNOMI-921 but are kept out of this PR to keep the diff strictly limited to the build-infrastructure swap. They will land in a follow-up PR:
Remove
BaseIT.fixDefaultTemplateIfNeeded()and its call incheckSearchEngine()(now dead code under Docker).Migrate16xToCurrentVersionIT: replace the hardcodedES_BASE_URL = "http://localhost:9400"with the dynamicgetSearchPort()resolution.Drop the comments referring to the
elasticsearch-maven-plugintemplate-override workaround.I hereby declare this contribution to be licenced under the Apache License Version 2.0, January 2004