Skip to content

Comments

[WIP] Fixing synchronization in TransactionManager#4332

Closed
nsivabalan wants to merge 1 commit intoapache:masterfrom
nsivabalan:transManagerSyncFix
Closed

[WIP] Fixing synchronization in TransactionManager#4332
nsivabalan wants to merge 1 commit intoapache:masterfrom
nsivabalan:transManagerSyncFix

Conversation

@nsivabalan
Copy link
Contributor

What is the purpose of the pull request

There could be a potential deadlock with the way methods in Transaction Manager is synchronized. Prior to this patch, all begin and end transaction methods were synchronized. So, consider the following scenario.
Deltastreaamer with async table service. Lets call deltastreamer as writer1 and async table service as writer2 (both will be sharing the same write client).
writer1 acquires the lock and proceeds on to do some work.
writer2 tries to begin transactions now. since lock is not available, it will be put to wait, but is within synchronized block.
writer1 finishes the work and tries to end the transaction, but since writer2 is waiting within synchronized block, writer1 can't end the transaction.
This patch fixes the issue by de-coupling the begin and end transaction methods for synchronization.

Brief change log

(for example:)

  • Modify AnnotationLocation checkstyle rule in checkstyle.xml

Verify this pull request

(Please pick either of the following options)

This pull request is a trivial rework / code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end.
  • Added HoodieClientWriteTest to verify the change.
  • Manually verified the change by running a job locally.

Committer checklist

  • Has a corresponding JIRA in PR title & commit

  • Commit message is descriptive of the change

  • CI is green

  • Necessary doc changes done or have another open PR

  • For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

@nsivabalan nsivabalan added the status:in-progress Work in progress label Dec 15, 2021
@hudi-bot
Copy link
Collaborator

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

@nsivabalan nsivabalan closed this Dec 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status:in-progress Work in progress

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants