Skip to content

UNOMI-921: Replace elasticsearch-maven-plugin with Docker-based Elasticsearch in integration tests#759

Open
sergehuber wants to merge 2 commits into
masterfrom
UNOMI-itests-es-docker
Open

UNOMI-921: Replace elasticsearch-maven-plugin with Docker-based Elasticsearch in integration tests#759
sergehuber wants to merge 2 commits into
masterfrom
UNOMI-itests-es-docker

Conversation

@sergehuber
Copy link
Copy Markdown
Contributor

@sergehuber sergehuber commented May 18, 2026

Stacked PR (merge order)

Role Branch
Base (merge into) master
Head (this PR) UNOMI-itests-es-docker

This PR is stacked: it is the bottom of a chain. Merge into master first; downstream branches that build on top will resync onto the new master tip.

For more information (each PR targets the branch below until the bottom merges): https://github.github.com/gh-stack/introduction/overview/


JIRA: https://issues.apache.org/jira/browse/UNOMI-921Replace elasticsearch-maven-plugin with Docker-based Elasticsearch Instance in integration tests.

Why this change

The elasticsearch profile in itests/pom.xml currently uses com.github.alexcojocaru:elasticsearch-maven-plugin to download an Elasticsearch tarball and run it as a forked JVM during the Maven build. This approach:

  • requires downloading Elasticsearch binaries during the build,
  • creates a default_template that overrides user templates on ES 8/9 — currently worked around by BaseIT.fixDefaultTemplateIfNeeded(),
  • is inconsistent with the opensearch profile (same itests/pom.xml), which already uses Docker, and
  • has a more complex lifecycle to manage.

Aligning Elasticsearch on the same Docker-based approach the opensearch profile uses removes the template-override workaround, eliminates the binary download, and makes both search-engine profiles symmetric.

What changed

itests/pom.xmlelasticsearch profile

  • Add <elasticsearch.port>9400</elasticsearch.port> and surface it to Failsafe via systemPropertyVariables, so tests resolve the HTTP port from a single property (value unchanged: 9400).
  • Replace the elasticsearch-maven-plugin execution with an io.fabric8:docker-maven-plugin execution running:
    • Image: docker.elastic.co/elasticsearch/elasticsearch:${elasticsearch.test.version}
    • Port mapping: container 9200 → host 9400
    • Heap: -Xms8g -Xmx8g — aligned with OpenSearch and the ES 9 sizing guidance (was 4g)
    • Settings: discovery.type=single-node, cluster.default.index.settings.number_of_replicas=0, xpack.ml.enabled=false, xpack.security.enabled=false, cluster.routing.allocation.disk.threshold_enabled=false, path.repo=/tmp/snapshots_repository
    • Volume bind: ${project.build.directory}/snapshots_repository/tmp/snapshots_repository
    • HTTP wait probe before the integration-test phase
  • Container lifecycle matches the OpenSearch profile exactly: pre-integration-test runs stop + remove (idempotent cleanup) then start with showLogs; post-integration-test runs stop only — the container is kept around (not removed) so logs can be inspected after a failed run.
  • Add chmod -R ugo+rwx on target/snapshots_repository in the antrun unzip step. The official ES image runs as UID 1000, so on Linux CI the bind-mounted snapshot repository otherwise hits access_denied during repository verify operations.

pom.xml (root)

  • Declare <docker-maven-plugin.version>0.48.0</docker-maven-plugin.version> and add the corresponding <pluginManagement> entry so the elasticsearch profile (and any future user of the plugin) inherits a single version.

Verification

  • mvn -P integration-tests,elasticsearch -DskipITs=false verify on itests: container starts on 9400, all integration tests run against it, container is stopped (but not removed) after the run.
  • Container is visible via docker ps -a --filter name=unomi-itests-elasticsearch after the run; logs accessible via docker logs.

Follow-ups (tracked under the same JIRA)

These items are part of UNOMI-921 but are kept out of this PR to keep the diff strictly limited to the build-infrastructure swap. They will land in a follow-up PR:

  • Remove BaseIT.fixDefaultTemplateIfNeeded() and its call in checkSearchEngine() (now dead code under Docker).

  • Migrate16xToCurrentVersionIT: replace the hardcoded ES_BASE_URL = "http://localhost:9400" with the dynamic getSearchPort() resolution.

  • Drop the comments referring to the elasticsearch-maven-plugin template-override workaround.

  • I hereby declare this contribution to be licenced under the Apache License Version 2.0, January 2004

…icsearch in integration tests

Implements the core configuration switch from UNOMI-921:
https://issues.apache.org/jira/browse/UNOMI-921

Replace the com.github.alexcojocaru:elasticsearch-maven-plugin (binary
download + forked JVM) with io.fabric8:docker-maven-plugin in the
elasticsearch profile of itests, mirroring how the opensearch profile
already runs OpenSearch in a Docker container.

itests/pom.xml (elasticsearch profile)
* Add an <elasticsearch.port>9400</elasticsearch.port> property and pass
  it through the failsafe systemPropertyVariables so tests resolve the
  HTTP port from a single source (unchanged from the previous 9400).
* Replace the elasticsearch-maven-plugin block with a docker-maven-plugin
  block that runs docker.elastic.co/elasticsearch/elasticsearch:${elasticsearch.test.version},
  binds target/snapshots_repository to /tmp/snapshots_repository, and
  waits on the HTTP port before the integration-test phase. Heap aligned
  to 8GB (-Xms8g -Xmx8g) to match the OpenSearch configuration and the
  ES 9 recommendation. Discovery=single-node, replicas=0, xpack.ml and
  xpack.security disabled.
* Container lifecycle matches OpenSearch exactly: pre-integration-test
  runs stop+remove then start (with showLogs); post-integration-test
  runs stop only -- container is kept around for inspection.
* Add a chmod -R ugo+rwx on snapshots_repository in the antrun unzip step:
  the ES container runs as UID 1000, so on Linux CI the bind-mounted
  snapshot repo otherwise hits access_denied during repository verify.

pom.xml (root)
* Declare <docker-maven-plugin.version>0.48.0</docker-maven-plugin.version>
  and add the pluginManagement entry so the elasticsearch profile (and
  any future user of the plugin) inherits a single version.

Scope kept minimal for the PR #757 stack split: only the test
infrastructure switch lives here. The follow-up UNOMI-921 acceptance
items below ship in the platform PR (P) once it lands:

* Remove BaseIT.fixDefaultTemplateIfNeeded() and the call in
  checkSearchEngine() (no longer needed with Docker).
* Migrate16xToCurrentVersionIT: replace hardcoded
  ES_BASE_URL = "http://localhost:9400" with dynamic getSearchPort().
* Drop the comments referring to the elasticsearch-maven-plugin
  template-override workaround.

See docs/PR-757-stack-extraction-tracker.md for the full split plan and
how this PR fits in the stack.
@sergehuber sergehuber force-pushed the UNOMI-itests-es-docker branch from df1dc2a to a2e008c Compare May 18, 2026 13:47
@sergehuber sergehuber changed the title UNOMI-880 (split A0): migrate Elasticsearch integration tests to Docker UNOMI-921: Replace elasticsearch-maven-plugin with Docker-based Elasticsearch in integration tests May 18, 2026
@sergehuber sergehuber requested a review from Copilot May 18, 2026 14:29
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

Comment thread itests/pom.xml
</ports>
<env>
<discovery.type>single-node</discovery.type>
<ES_JAVA_OPTS>-Xms8g -Xmx8g -Dcluster.default.index.settings.number_of_replicas=0</ES_JAVA_OPTS>
Comment thread itests/pom.xml
</ports>
<env>
<discovery.type>single-node</discovery.type>
<ES_JAVA_OPTS>-Xms8g -Xmx8g -Dcluster.default.index.settings.number_of_replicas=0</ES_JAVA_OPTS>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants