feat(api-search-index-tasks): platform agnostic index tasks - #5487
Merged
Conversation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…s dependency The package needs webiny.config.js for yarn build -p to find it, and @webiny/background-tasks for the TaskController augmentation (response, state, logger, runtime). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Register all search index tasks and runners in a single feature. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…rFactory Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…iter Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove unused @webiny/error and @webiny/utils from api-search-index-tasks - Remove unused @webiny/aws-sdk and @webiny/error from api-search-index-tasks-ddb-os - Re-register old IndexManagerFactory/settings impls in ElasticsearchTasksFeature (dataSynchronization depends on old DI key "ElasticsearchTasks/IndexManagerFactory") - Format all files Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
IndexSettingsManager abstraction, DisableIndexing, EnableIndexing, and their error classes are engine-agnostic logic — they operate on the IndexSettingsManager abstraction, not on OS directly. Only the IndexSettingsManager impl (which calls OS putSettings/getSettings) stays in the bridge package. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Bridge consumers only need SearchIndexTasksDdbOsFeature. All abstractions come from the core package. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Group files into storage/, indexManager/, settings/ directories instead of flat root layout. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
No need for separate feature.ts when index.ts only exports the feature. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Separate OpenSearch-specific impls (IndexSettingsManager, IndexManager, IndexManagerFactory) into @webiny/api-search-index-tasks-os. The ddb-os bridge now composes the OS feature + DDB scanner/writer. Package hierarchy: api-search-index-tasks (platform-agnostic tasks + abstractions) api-search-index-tasks-os (OpenSearch impls) api-search-index-tasks-ddb-os (DDB impls, composes OS feature) When pg-os arrives, it will compose the same OS feature with PG scanner/writer — no duplication. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Package fully replaced by: @webiny/api-search-index-tasks (platform-agnostic tasks) @webiny/api-search-index-tasks-os (OpenSearch impls) @webiny/api-search-index-tasks-ddb-os (DDB impls) dataSynchronization task dropped — can be rebuilt if needed. Migrated consumers: - api-headless-cms-ddb-es: OpenSearchTenantIndexFactory → TenantIndexFactory - api-background-tasks-os: ElasticsearchTasksFeature → SearchIndexTasks + DdbOs features Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Port indexManager.test.ts from deleted api-elasticsearch-tasks. Tests OsIndexManager construction, disable indexing, and enable indexing with mocked OpenSearch client. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ReindexRunner now creates missing indexes using TenantIndexFactory configs instead of skipping them. ReindexTask builds an index configs map from TenantIndexFactory before calling the runner. 9 unit tests covering: empty scan, multiple indexes, create missing index, skip when no config, matching filter, published + latest records, timeout continuation, overwrite via put, skip empty entity. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ReindexTask no longer knows about TenantIndexFactory, TenantContext, or ListTenantsUseCase. ReindexRunner resolves index configs internally so it works standalone without the task definition. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Use Abstraction.Interface/Settings/SettingsMap instead of direct interface imports. Remove unused IListIndicesResponse. Add vitest devDep to api-search-index-tasks-os. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
SQL storage adds isLatest/isPublished columns for query efficiency but these are internal — not part of the CMS entry contract. Strip them from the returned storageEntry so all backends return consistent shape. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Pure wiring package with no logic — consumers should depend on api-search-index-tasks and api-search-index-tasks-ddb-os directly. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Pure pass-through — only re-exported BackgroundTasksFeature from @webiny/background-tasks. Consumers should depend on background-tasks directly. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Pure re-export of HcmsTasksFeature from @webiny/api-headless-cms-tasks. Consumers already import from the source package directly. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
brunozoric
marked this pull request as ready for review
July 25, 2026 20:45
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Our current api-elasticsearch-tasks are strictly connected to DynamoDB + OpenSearch.
Split api-elasticsearch-tasks into base api-search-index - can be OS or ES, unfortunately they have some differences in APIs.
Implementations for the OpenSearch now live in api-search-index-os.
Implementations for the DynamoDB + OpenSearch now live in api-search-index-ddb-os. This package wires the OS impl and adds its own stuff for DynamoDB.
Reasoning
We will soon have Postgres + OpenSearch tasks implementation. It will probably be a single pg-os package, but I wanted to split as much of the abstractions as possible so it is later easier to add Elasticsearch support if someone requires it.
TODO
Basically, all current strict OpenSearch packages should follow this same principle: have index engine abstractions, as much as possible, then dedicated -os or -es packages implement their own specific stuff if there is any. If not, single package can be reused in OS / ES deployments.