Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Implement dynamic schema update using index mapping difference #18622

Merged
merged 9 commits into from
May 23, 2024
Merged

feat: Implement dynamic schema update using index mapping difference #18622

merged 9 commits into from
May 23, 2024

Conversation

nathansandi
Copy link
Contributor

@nathansandi nathansandi commented May 17, 2024 •
edited

Description

  • Retrieve the Index/Template Mappings over RetryElasticSearchclient
  • Placeholders for the OpenSearch implementation
  • Update the Schema Startup to consider the Update Schema setup
  • Updated the (Abstract) Index descriptors to return getSchemaClasspathFilename() and getAllVersionsIndexNameRegexPattern()
  • Update ElastciSearch Schema Manager to Return the Expected Index Fields from the mapping, considering the differences
  • Adjust Schema Manager for create Index and Template making the public as well in order to use the methods on test
  • Extract ElasticSearchReadJson Utill to a different method (Refactor)

Its the second part of the implementation:
Part 1: #18618

Related issues

closes #18486

@github-actions github-actions bot added the component/tasklist Related to the Tasklist component/team label May 17, 2024
@nathansandi nathansandi changed the title feature: Implement dynamic schema update using index mapping difference feat: Implement dynamic schema update using index mapping difference May 17, 2024
@nathansandi nathansandi mentioned this pull request May 17, 2024
Merged
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented May 17, 2024 •
edited

Tasklist Test Results

0 tests  ±0   0 ✅ ±0   0s ⏱️ ±0s
0 suites ±0   0 💤 ±0 
0 files   ±0   0 ❌ ±0 

Results for commit f1670f6. ± Comparison against base commit 34917b3.

♻️ This comment has been updated with latest results.

@nathansandi nathansandi mentioned this pull request May 21, 2024
Merged
@nathansandi nathansandi marked this pull request as ready for review May 21, 2024 08:36
houssain-barouni
houssain-barouni reviewed May 22, 2024
View reviewed changes
Copy link
Collaborator

@houssain-barouni houssain-barouni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few comments :)

tasklist/common/src/main/java/io/camunda/tasklist/property/TasklistElasticsearchProperties.java Outdated Show resolved Hide resolved
tasklist/common/src/main/java/io/camunda/tasklist/property/TasklistElasticsearchProperties.java Outdated Show resolved Hide resolved
tasklist/els-schema/pom.xml Show resolved Hide resolved
tasklist/els-schema/src/main/java/io/camunda/tasklist/es/RetryElasticsearchClient.java Outdated Show resolved Hide resolved
tasklist/els-schema/src/main/java/io/camunda/tasklist/schema/SchemaStartup.java Outdated Show resolved Hide resolved
.../els-schema/src/main/java/io/camunda/tasklist/schema/manager/ElasticsearchSchemaManager.java Outdated Show resolved Hide resolved
@nathansandi
Merge branch 'tasklist/implement-indexdiffmappingg' into Implement-dy…
c352a1b 
…namic-schema-update-usind-index-mapping-difference
@nathansandi
Copy link
Contributor Author

Few comments :)

Thanks @houssain-barouni I updated the branch considering all your comments.

houssain-barouni
houssain-barouni approved these changes May 23, 2024
View reviewed changes
Copy link
Collaborator

@houssain-barouni houssain-barouni left a comment •
edited by nathansandi

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is still

<version>33.2.0-jre</version>

to be removed from the pom.xml, but you can do it in the feature branch

@nathansandi
Copy link
Contributor Author

Yes, I removed and gonna merge those changes on the feature final branch 💯

@nathansandi
test: Dynamic schema update Integration Tests (Elastic Search) (#18623)
c4f7a9e 
## Description

Integration test classes for Dynamic Schema Updates.

Implementation Tickets:
Part 1: #18618 
Part 2: #18622

## Related issues

closes #18486
@nathansandi nathansandi merged commit e23bfc4 into tasklist/implement-indexdiffmappingg May 23, 2024
12 of 15 checks passed
@nathansandi nathansandi deleted the Implement-dynamic-schema-update-usind-index-mapping-difference branch May 23, 2024 13:47
github-merge-queue bot pushed a commit that referenced this pull request Jun 4, 2024
@nathansandi
feat: dynamic schema update feature PR (#18618)
8abcf37 
## Description

Initial PR changes:
- Add the Index Schema Mapping Difference
- This PR Implements the Index Difference part for Elastic search
- It is a divided PR for the implementation of Elastic Search Update

The second PR will make this one functional
A third PR will be open for the tests

Complementary pull requests merged: (Elastic Search implementation) //
Already Approved
1) #18622
2) #18623

OpenSearch implementation: (To be merged/reviewed)
1) #18684
2) #18769
3) #18812

Refactor:
1) #18896

## Related issues

closes #18486
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@houssain-barouni houssain-barouni houssain-barouni approved these changes

Assignees
No one assigned
Labels
component/tasklist Related to the Tasklist component/team
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@nathansandi @houssain-barouni