UNOMI-139/880/878/884: Multi-tenant platform core#757
Closed
sergehuber wants to merge 14 commits into
Closed
Conversation
… unified cache, scheduler, migrations, V2 compat Why one PR These changes share the same execution model (tenant context, unified cache, scheduler, and persistence) and are wired through the same services, blueprints, and integration tests. Splitting them into separate commits would produce artificial intermediate states that neither compile nor pass ITs, and would force heavy merge or rebase work across the same files repeatedly. They are therefore delivered together as one coherent “3.1 platform slice” for review, even though the work maps to several JIRA tickets. This change set backports and wires the Unomi 3.1 multi-tenant platform work into the integration branch: tenant-aware execution, centralized caching, the new scheduler service, V3 migration assets, and optional V2 REST compatibility. Tracing and full type-validation services from unomi-3-dev are not included; call sites use ParserHelper / defensive handling where those stacks were removed for a stacked follow-up branch. UNOMI-139 Multi-tenancy - Introduce tenant model and APIs (Tenant, TenantService, API keys, security/audit interfaces, quotas, lifecycle hooks) and propagate tenantId on Item and related types. - Add ExecutionContextManager and thread-local tenant context so services, persistence, and REST layers resolve the active tenant consistently. - Adjust persistence (Elasticsearch/OpenSearch) and services to scope reads/writes and inheritance (e.g. system tenant vs tenant overrides). UNOMI-880 Unified multi-tenant caching - Add MultiTypeCacheService / CacheableTypeConfig and AbstractMultiTypeCachingService for shared cache lifecycle, bundle predefined items, periodic refresh, and statistics. - Migrate DefinitionsServiceImpl and SegmentServiceImpl (and related paths) onto the unified cache; remove duplicated ad-hoc cache listeners where superseded. - Add Karaf shell cache commands (unomi:cache) and align tests with empty stats / CSV output edge cases. UNOMI-878 Cluster-aware task scheduling - Replace legacy scheduler usage with the new SchedulerService API (scheduled tasks, task executors, persistence-backed vs in-memory tasks). - Add shell commands for task inspection and management; integrate periodic work (e.g. cache refresh, rules refresh, purges) with the new scheduler where backported from unomi-3-dev. - Update dependents (e.g. Groovy actions service) to schedule work through the new API instead of removed executor accessors. UNOMI-884 Migration to V3 - Add migration Groovy scripts and request bodies for 3.1.0 (tenant document IDs, system item id fixes, tenant initialization, legacy queryBuilder updates, profile visit fixes as applicable). - Extend migration configuration and tests (e.g. MigrationUtilsTest) to cover new steps aligned with multi-tenancy and index updates. UNOMI-904 V2 API compatibility mode - Add REST authentication configuration (Metatype) for toggling V2 compatibility and choosing the default tenant when the mode is on. - Implement AuthenticationFilter handling so legacy V2-style access patterns can be enabled during migration without forcing every client to API keys immediately. Reliability and tests - All integration tests (AllITs) pass in a green run; they remain flaky locally—roughly one in four full-suite runs still fails with unrelated timing or environment symptoms (shared Elasticsearch, indexing lag, suite order). A follow-up should harden CI or add targeted waits where needed; this commit does not claim zero flake rate. - Align condition/action resolution with HEAD + selective unomi-3-dev fixes (PropertyHelper for past-event numeric parameters, null conditionType in evaluation, optional action type resolution in the dispatcher, segment recalculation paths). - Stabilize integration tests: wait for segment visibility after GraphQL create, relax cache stats assertions when global stats are empty, adjust invalid-segment test when numeric strings are coerced without validation service. Docs: https://issues.apache.org/jira/browse/UNOMI-139 https://issues.apache.org/jira/browse/UNOMI-880 https://issues.apache.org/jira/browse/UNOMI-878 https://issues.apache.org/jira/browse/UNOMI-884 https://issues.apache.org/jira/browse/UNOMI-904
… on CI - PastEventConditionOSQueryBuilder: accept java.util.Date or String for fromDate/toDate (parity with PastEventConditionESQueryBuilder) to fix EventServiceIT past-event tests under OpenSearch. - itests: chmod -R ugo+rwx snapshots_repository after unzip on Unix so Elasticsearch in Docker (UID 1000) can verify FS snapshot repos on Linux runners (e.g. GitHub Actions).
Increase retries and delay for unomi:crud list after create/read so rule and goal list assertions tolerate persistence/cache lag on slow runners.
Run mikepenz/action-junit-report after every integration matrix leg (if: always), with update_check and per-matrix check_name, so a green re-run replaces the stale red JUnit check. Use fail_on_failure=false and require_tests=false; continue-on-error so missing XML cannot fail the job.
…' into UNOMI-139-UNOMI-880-multitenancy
…80-multitenancy # Conflicts: # api/src/main/java/org/apache/unomi/api/actions/ActionType.java # api/src/main/java/org/apache/unomi/api/conditions/ConditionType.java
…80-multitenancy # Conflicts: # .vscode/settings.json
…80-multitenancy # Conflicts: # tools/shell-dev-commands/pom.xml
…80-multitenancy # Conflicts: # extensions/router/router-core/src/main/java/org/apache/unomi/router/core/context/RouterCamelContext.java
sergehuber
added a commit
that referenced
this pull request
May 18, 2026
Replace the com.github.alexcojocaru:elasticsearch-maven-plugin (binary download + forked JVM) with io.fabric8:docker-maven-plugin so the elasticsearch profile of itests runs ES in a Docker container, mirroring how the opensearch profile already runs OpenSearch. itests/pom.xml (elasticsearch profile) * Add an <elasticsearch.port>9400</elasticsearch.port> property and pass it through the failsafe systemPropertyVariables so tests resolve the HTTP port from a single source. * Replace the elasticsearch-maven-plugin block with a docker-maven-plugin block that starts/stops a docker.elastic.co/elasticsearch/elasticsearch container, binds target/snapshots_repository to /tmp/snapshots_repository, and waits on the HTTP port before the integration-test phase. * Add a chmod -R ugo+rwx on snapshots_repository in the antrun unzip step: the ES container runs as UID 1000, so on Linux CI the bind-mounted snapshot repo otherwise hits access_denied during repository verify. pom.xml (root) * Declare <docker-maven-plugin.version>0.48.0</docker-maven-plugin.version> and add the pluginManagement entry so the elasticsearch profile (and any future user of the plugin) inherits a single version. This is phase A0 of the PR #757 stack split (see docs/PR-757-stack-extraction-tracker.md). It is intentionally small and self-contained: no test/code changes, only the test infrastructure switch.
1 task
…80-multitenancy # Conflicts: # pom.xml
sergehuber
added a commit
that referenced
this pull request
May 18, 2026
…icsearch in integration tests Implements the core configuration switch from UNOMI-921: https://issues.apache.org/jira/browse/UNOMI-921 Replace the com.github.alexcojocaru:elasticsearch-maven-plugin (binary download + forked JVM) with io.fabric8:docker-maven-plugin in the elasticsearch profile of itests, mirroring how the opensearch profile already runs OpenSearch in a Docker container. itests/pom.xml (elasticsearch profile) * Add an <elasticsearch.port>9400</elasticsearch.port> property and pass it through the failsafe systemPropertyVariables so tests resolve the HTTP port from a single source (unchanged from the previous 9400). * Replace the elasticsearch-maven-plugin block with a docker-maven-plugin block that runs docker.elastic.co/elasticsearch/elasticsearch:${elasticsearch.test.version}, binds target/snapshots_repository to /tmp/snapshots_repository, and waits on the HTTP port before the integration-test phase. Heap aligned to 8GB (-Xms8g -Xmx8g) to match the OpenSearch configuration and the ES 9 recommendation. Discovery=single-node, replicas=0, xpack.ml and xpack.security disabled. * Container lifecycle matches OpenSearch exactly: pre-integration-test runs stop+remove then start (with showLogs); post-integration-test runs stop only -- container is kept around for inspection. * Add a chmod -R ugo+rwx on snapshots_repository in the antrun unzip step: the ES container runs as UID 1000, so on Linux CI the bind-mounted snapshot repo otherwise hits access_denied during repository verify. pom.xml (root) * Declare <docker-maven-plugin.version>0.48.0</docker-maven-plugin.version> and add the pluginManagement entry so the elasticsearch profile (and any future user of the plugin) inherits a single version. Scope kept minimal for the PR #757 stack split: only the test infrastructure switch lives here. The follow-up UNOMI-921 acceptance items below ship in the platform PR (P) once it lands: * Remove BaseIT.fixDefaultTemplateIfNeeded() and the call in checkSearchEngine() (no longer needed with Docker). * Migrate16xToCurrentVersionIT: replace hardcoded ES_BASE_URL = "http://localhost:9400" with dynamic getSearchPort(). * Drop the comments referring to the elasticsearch-maven-plugin template-override workaround. See docs/PR-757-stack-extraction-tracker.md for the full split plan and how this PR fits in the stack.
sergehuber
changed the title
1 task
Contributor
Author
|
Superseded by #760 — platform-only slice on |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked PR (merge order)
UNOMI-888-import-export-javadocUNOMI-139-UNOMI-880-multitenancyThis PR is stacked: it should merge into
UNOMI-888-import-export-javadocfirst; after that PR is onmaster(or the next stack parent), this slice reduces to the multi-tenant / cache / scheduler / migration / V2-compat delta relative to that line.Merge-base with
origin/master(when last checked):c85fb8856— same tip asorigin/masterat fork time for the underlying stack; the branch contains prior stacked commits (UNOMI-920 → UNOMI-879 → UNOMI-888) before this work.Why a single PR
These changes share the same execution model (tenant context, unified cache, scheduler, and persistence) and are wired through the same services, blueprints, and integration tests. Splitting them into separate commits would produce artificial intermediate states that neither compile nor pass ITs, and would force heavy merge or rebase work across the same files repeatedly. They are therefore delivered together as one coherent “3.1 platform slice” for review, even though the work maps to several JIRA tickets.
This change set backports and wires the Unomi 3.1 multi-tenant platform work into the integration branch: tenant-aware execution, centralized caching, the new scheduler service, V3 migration assets, and optional V2 REST compatibility. Tracing and full type-validation services from unomi-3-dev are not included; call sites use ParserHelper / defensive handling where those stacks were removed for a stacked follow-up branch.
UNOMI-139 Multi-tenancy
UNOMI-880 Unified multi-tenant caching
UNOMI-878 Cluster-aware task scheduling
UNOMI-884 Migration to V3
UNOMI-904 V2 API compatibility mode
Reliability and tests
Docs: https://issues.apache.org/jira/browse/UNOMI-139
https://issues.apache.org/jira/browse/UNOMI-880
https://issues.apache.org/jira/browse/UNOMI-878
https://issues.apache.org/jira/browse/UNOMI-884
https://issues.apache.org/jira/browse/UNOMI-904