-
Notifications
You must be signed in to change notification settings - Fork 3.3k
feat(search): unified entity index #14966
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
✅ Meticulous spotted 0 visual differences across 993 screens tested: view results. Meticulous evaluated ~8 hours of user flows against your PR. Expected differences? Click here. Last updated for commit d937d9e. This comment will update as new commits are pushed. |
Bundle ReportBundle size has no change ✅ |
# Conflicts: # datahub-upgrade/src/main/java/com/linkedin/datahub/upgrade/loadindices/LoadIndices.java # datahub-upgrade/src/main/java/com/linkedin/datahub/upgrade/loadindices/LoadIndicesArgs.java # datahub-upgrade/src/main/java/com/linkedin/datahub/upgrade/loadindices/LoadIndicesIndexManager.java # datahub-upgrade/src/main/java/com/linkedin/datahub/upgrade/loadindices/LoadIndicesStep.java # datahub-upgrade/src/main/java/com/linkedin/datahub/upgrade/loadindices/config/LoadIndicesConfig.java # datahub-upgrade/src/test/java/com/linkedin/datahub/upgrade/UpgradeCliApplicationTestConfiguration.java # datahub-upgrade/src/test/java/com/linkedin/datahub/upgrade/loadindices/LoadIndicesIndexManagerTest.java # datahub-upgrade/src/test/java/com/linkedin/datahub/upgrade/loadindices/LoadIndicesStepTest.java # datahub-upgrade/src/test/java/com/linkedin/datahub/upgrade/loadindices/config/LoadIndicesConfigTest.java # docs/how/load-indices.md # metadata-io/src/main/java/com/linkedin/metadata/search/elasticsearch/indexbuilder/ESIndexBuilder.java # metadata-io/src/main/java/com/linkedin/metadata/search/elasticsearch/update/BulkListener.java # metadata-io/src/main/java/com/linkedin/metadata/search/elasticsearch/update/ESBulkProcessor.java # metadata-io/src/main/java/com/linkedin/metadata/search/elasticsearch/update/ESWriteDAO.java # metadata-io/src/main/java/com/linkedin/metadata/service/UpdateIndicesService.java # metadata-io/src/testFixtures/java/io/datahubproject/test/search/BulkProcessorTestUtils.java # metadata-service/factories/src/main/java/com/linkedin/gms/factory/common/RestHighLevelClientFactory.java
Supports: v2/v3 index management v3 settings/mappings Dual v2/v3 writes Not including: query
bbc4123 to
af40867
Compare
fix misssing break
...a/com/linkedin/metadata/search/elasticsearch/index/entity/v3/MultiEntityMappingsBuilder.java
Show resolved
Hide resolved
...rc/main/java/com/linkedin/gms/factory/entity/update/indices/UpdateIndicesServiceFactory.java
Show resolved
Hide resolved
metadata-io/src/main/java/com/linkedin/metadata/search/elasticsearch/index/MappingsBuilder.java
Outdated
Show resolved
Hide resolved
...a/com/linkedin/metadata/search/elasticsearch/index/entity/v3/MultiEntityMappingsBuilder.java
Show resolved
Hide resolved
...a/com/linkedin/metadata/search/elasticsearch/index/entity/v3/MultiEntityMappingsBuilder.java
Show resolved
Hide resolved
...a/com/linkedin/metadata/search/elasticsearch/index/entity/v3/MultiEntityMappingsBuilder.java
Show resolved
Hide resolved
...a/com/linkedin/metadata/search/elasticsearch/index/entity/v3/MultiEntityMappingsBuilder.java
Outdated
Show resolved
Hide resolved
...a/com/linkedin/metadata/search/elasticsearch/index/entity/v3/MultiEntityMappingsBuilder.java
Outdated
Show resolved
Hide resolved
...m/linkedin/metadata/search/elasticsearch/index/entity/v3/MultiEntityMappingsBuilderTest.java
Show resolved
Hide resolved
...est/java/com/linkedin/metadata/search/elasticsearch/index/DelegatingMappingsBuilderTest.java
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approving.
I flagged a couple of tests for over-testing. There must be others but I haven't looked.
I think #15073 has proved its usefulness (the guidelines are good but I haven't verified that claude code does the right thing when generating new unit tests).
2851564 to
ef3490e
Compare
Introduce a new unified entity index which consolidates entity indices (100+) into a few indices. It will eventually also replace the system metadata index. This is disabled by default and only contains code for the write path at this time.
Major Features
Key Components Added
MultiEntityMappingsBuilder- Unified mapping generationUpdateIndicesV2Strategy&UpdateIndicesV3Strategy- Index update strategiesSearchableAnnotationValidator- Validation for search annotationsDelegatingMappingsBuilder&DelegatingSettingsBuilder- Delegation patternsConfiguration Changes