Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(elasticsearch): make indexNameMapping in IndexConventionImpl threadsafe #7565

Merged
merged 4 commits into from Mar 17, 2023

Conversation

iprentic
Copy link
Contributor

There was a ConcurrentModificationException seen in this class:

2023-03-05 03:46:06,210 [Thread-134] ERROR c.l.d.g.e.DataHubDataFetcherExceptionHandler:21 - Failed to execute DataFetcher
java.util.concurrent.CompletionException: java.lang.RuntimeException: Failed to list Views
at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:314)
at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:319)
at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1702)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.RuntimeException: Failed to list Views
at com.linkedin.datahub.graphql.resolvers.view.ListMyViewsResolver.lambda$get$0(ListMyViewsResolver.java:91)
at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
... 1 common frames omitted
Caused by: java.util.ConcurrentModificationException: null
at java.base/java.util.HashMap.computeIfAbsent(HashMap.java:1135)
at com.linkedin.metadata.utils.elasticsearch.IndexConventionImpl.getIndexName(IndexConventionImpl.java:57)
at com.linkedin.metadata.utils.elasticsearch.IndexConventionImpl.getEntityIndexName(IndexConventionImpl.java:63)
at com.linkedin.metadata.utils.elasticsearch.IndexConventionImpl.getIndexName(IndexConventionImpl.java:51)
at com.linkedin.metadata.search.elasticsearch.query.ESSearchDAO.search(ESSearchDAO.java:151)
at com.linkedin.metadata.search.elasticsearch.ElasticSearchService.fullTextSearch(ElasticSearchService.java:112)
at com.linkedin.metadata.client.JavaEntityClient.search(JavaEntityClient.java:311)
at com.linkedin.datahub.graphql.resolvers.view.ListMyViewsResolver.lambda$get$0(ListMyViewsResolver.java:71)

Make the data structure accessed to be a ConcurrentHashMap to handle multiple threads accessing at once.

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
  • For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

@RyanHolstien RyanHolstien changed the title make indexNameMapping in IndexConventionImpl threadsafe fix(elasticsearch): make indexNameMapping in IndexConventionImpl threadsafe Mar 13, 2023
Copy link
Collaborator

@david-leifker david-leifker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sure green before merge. Thanks!

@iprentic iprentic merged commit 44a8536 into datahub-project:master Mar 17, 2023
28 of 31 checks passed
yoonhyejin pushed a commit to yoonhyejin/datahub-project-forked that referenced this pull request Mar 17, 2023
…adsafe (datahub-project#7565)

Co-authored-by: Indy Prentice <indy@Indys-MacBook-Pro.local>
iprentic added a commit to iprentic/datahub that referenced this pull request Mar 20, 2023
…adsafe (datahub-project#7565)

Co-authored-by: Indy Prentice <indy@Indys-MacBook-Pro.local>
shirshanka pushed a commit to shirshanka/datahub that referenced this pull request Mar 22, 2023
…adsafe (datahub-project#7565)

Co-authored-by: Indy Prentice <indy@Indys-MacBook-Pro.local>
shirshanka pushed a commit to shirshanka/datahub that referenced this pull request Mar 22, 2023
…adsafe (datahub-project#7565)

Co-authored-by: Indy Prentice <indy@Indys-MacBook-Pro.local>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants