Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ALLUXIO-3199]add try lock block method to BlockLockManager #7296

Open
wants to merge 9 commits into
base: master
from

Conversation

@Mitisky
Copy link
Contributor

commented May 16, 2018

@AmplabJenkins

This comment has been minimized.

Copy link

commented May 16, 2018

Merged build finished. Test FAILed.

@AmplabJenkins

This comment has been minimized.

Copy link

commented May 16, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/19637/

Build result: FAILURE

[...truncated 2427 lines...][JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/fs/pom.xml to org.alluxio/alluxio-core-client-fs/1.8.0-SNAPSHOT/alluxio-core-client-fs-1.8.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/fs/target/alluxio-core-client-fs-1.8.0-SNAPSHOT.jar to org.alluxio/alluxio-core-client-fs/1.8.0-SNAPSHOT/alluxio-core-client-fs-1.8.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/fs/target/alluxio-core-client-fs-1.8.0-SNAPSHOT-sources.jar to org.alluxio/alluxio-core-client-fs/1.8.0-SNAPSHOT/alluxio-core-client-fs-1.8.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/fs/target/alluxio-core-client-fs-1.8.0-SNAPSHOT-tests.jar to org.alluxio/alluxio-core-client-fs/1.8.0-SNAPSHOT/alluxio-core-client-fs-1.8.0-SNAPSHOT-tests.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/pom.xml to org.alluxio/alluxio-core-common/1.8.0-SNAPSHOT/alluxio-core-common-1.8.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.8.0-SNAPSHOT.jar to org.alluxio/alluxio-core-common/1.8.0-SNAPSHOT/alluxio-core-common-1.8.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.8.0-SNAPSHOT-sources.jar to org.alluxio/alluxio-core-common/1.8.0-SNAPSHOT/alluxio-core-common-1.8.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.8.0-SNAPSHOT-tests.jar to org.alluxio/alluxio-core-common/1.8.0-SNAPSHOT/alluxio-core-common-1.8.0-SNAPSHOT-tests.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/server/worker/pom.xml to org.alluxio/alluxio-core-server-worker/1.8.0-SNAPSHOT/alluxio-core-server-worker-1.8.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/logserver/pom.xml to org.alluxio/alluxio-logserver/1.8.0-SNAPSHOT/alluxio-logserver-1.8.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/pom.xml to org.alluxio/alluxio-core-client/1.8.0-SNAPSHOT/alluxio-core-client-1.8.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/examples/pom.xml to org.alluxio/alluxio-examples/1.8.0-SNAPSHOT/alluxio-examples-1.8.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/examples/target/alluxio-examples-1.8.0-SNAPSHOT.jar to org.alluxio/alluxio-examples/1.8.0-SNAPSHOT/alluxio-examples-1.8.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/examples/target/alluxio-examples-1.8.0-SNAPSHOT-sources.jar to org.alluxio/alluxio-examples/1.8.0-SNAPSHOT/alluxio-examples-1.8.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/pom.xml to org.alluxio/alluxio-core/1.8.0-SNAPSHOT/alluxio-core-1.8.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/server/proxy/pom.xml to org.alluxio/alluxio-core-server-proxy/1.8.0-SNAPSHOT/alluxio-core-server-proxy-1.8.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/server/proxy/target/alluxio-core-server-proxy-1.8.0-SNAPSHOT.jar to org.alluxio/alluxio-core-server-proxy/1.8.0-SNAPSHOT/alluxio-core-server-proxy-1.8.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/server/proxy/target/alluxio-core-server-proxy-1.8.0-SNAPSHOT-sources.jar to org.alluxio/alluxio-core-server-proxy/1.8.0-SNAPSHOT/alluxio-core-server-proxy-1.8.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/shell/pom.xml to org.alluxio/alluxio-shell/1.8.0-SNAPSHOT/alluxio-shell-1.8.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/shell/target/alluxio-shell-1.8.0-SNAPSHOT.jar to org.alluxio/alluxio-shell/1.8.0-SNAPSHOT/alluxio-shell-1.8.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/shell/target/alluxio-shell-1.8.0-SNAPSHOT-sources.jar to org.alluxio/alluxio-shell/1.8.0-SNAPSHOT/alluxio-shell-1.8.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/tests/pom.xml to org.alluxio/alluxio-tests/1.8.0-SNAPSHOT/alluxio-tests-1.8.0-SNAPSHOT.pomchannel stoppedArchiving artifacts
Test FAILed.

@AmplabJenkins

This comment has been minimized.

Copy link

commented May 16, 2018

Merged build finished. Test PASSed.

@AmplabJenkins

This comment has been minimized.

Copy link

commented May 16, 2018

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/19639/
Test PASSed.

@alluxio-bot

This comment has been minimized.

Copy link
Collaborator

commented May 16, 2018

Automated checks report:

  • Valid pull request title: PASS
  • Contains link to JIRA ticket: PASS
  • Commits associated with Github account: PASS

All checks passed!

@Mitisky

This comment has been minimized.

Copy link
Contributor Author

commented May 22, 2018

@apc999 PTAL.thanks

@AmplabJenkins

This comment has been minimized.

Copy link

commented May 22, 2018

Merged build finished. Test FAILed.

@AmplabJenkins

This comment has been minimized.

Copy link

commented May 22, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/19723/

Failed Tests: 1

org.alluxio:alluxio-tests: 1


Test FAILed.

@Mitisky

This comment has been minimized.

Copy link
Contributor Author

commented May 23, 2018

jenkins, test this please

@AmplabJenkins

This comment has been minimized.

Copy link

commented May 23, 2018

Merged build finished. Test FAILed.

@AmplabJenkins

This comment has been minimized.

Copy link

commented May 23, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/19744/

Failed Tests: 1

org.alluxio:alluxio-tests: 1


Test FAILed.

@Mitisky

This comment has been minimized.

Copy link
Contributor Author

commented May 29, 2018

jenkins, test this please

@AmplabJenkins

This comment has been minimized.

Copy link

commented May 29, 2018

Merged build finished. Test FAILed.

@AmplabJenkins

This comment has been minimized.

Copy link

commented May 29, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/19877/

Build result: FAILURE

[...truncated 2461 lines...][JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/fs/target/alluxio-core-client-fs-1.8.0-SNAPSHOT.jar to org.alluxio/alluxio-core-client-fs/1.8.0-SNAPSHOT/alluxio-core-client-fs-1.8.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/fs/target/alluxio-core-client-fs-1.8.0-SNAPSHOT-sources.jar to org.alluxio/alluxio-core-client-fs/1.8.0-SNAPSHOT/alluxio-core-client-fs-1.8.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/fs/target/alluxio-core-client-fs-1.8.0-SNAPSHOT-tests.jar to org.alluxio/alluxio-core-client-fs/1.8.0-SNAPSHOT/alluxio-core-client-fs-1.8.0-SNAPSHOT-tests.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/pom.xml to org.alluxio/alluxio-core-common/1.8.0-SNAPSHOT/alluxio-core-common-1.8.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.8.0-SNAPSHOT.jar to org.alluxio/alluxio-core-common/1.8.0-SNAPSHOT/alluxio-core-common-1.8.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.8.0-SNAPSHOT-sources.jar to org.alluxio/alluxio-core-common/1.8.0-SNAPSHOT/alluxio-core-common-1.8.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.8.0-SNAPSHOT-tests.jar to org.alluxio/alluxio-core-common/1.8.0-SNAPSHOT/alluxio-core-common-1.8.0-SNAPSHOT-tests.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/integration/checker/pom.xml to org.alluxio/alluxio-checker/1.8.0-SNAPSHOT/alluxio-checker-1.8.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/server/worker/pom.xml to org.alluxio/alluxio-core-server-worker/1.8.0-SNAPSHOT/alluxio-core-server-worker-1.8.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/logserver/pom.xml to org.alluxio/alluxio-logserver/1.8.0-SNAPSHOT/alluxio-logserver-1.8.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/pom.xml to org.alluxio/alluxio-core-client/1.8.0-SNAPSHOT/alluxio-core-client-1.8.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/examples/pom.xml to org.alluxio/alluxio-examples/1.8.0-SNAPSHOT/alluxio-examples-1.8.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/examples/target/alluxio-examples-1.8.0-SNAPSHOT.jar to org.alluxio/alluxio-examples/1.8.0-SNAPSHOT/alluxio-examples-1.8.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/examples/target/alluxio-examples-1.8.0-SNAPSHOT-sources.jar to org.alluxio/alluxio-examples/1.8.0-SNAPSHOT/alluxio-examples-1.8.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/pom.xml to org.alluxio/alluxio-core/1.8.0-SNAPSHOT/alluxio-core-1.8.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/server/proxy/pom.xml to org.alluxio/alluxio-core-server-proxy/1.8.0-SNAPSHOT/alluxio-core-server-proxy-1.8.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/server/proxy/target/alluxio-core-server-proxy-1.8.0-SNAPSHOT.jar to org.alluxio/alluxio-core-server-proxy/1.8.0-SNAPSHOT/alluxio-core-server-proxy-1.8.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/server/proxy/target/alluxio-core-server-proxy-1.8.0-SNAPSHOT-sources.jar to org.alluxio/alluxio-core-server-proxy/1.8.0-SNAPSHOT/alluxio-core-server-proxy-1.8.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/shell/pom.xml to org.alluxio/alluxio-shell/1.8.0-SNAPSHOT/alluxio-shell-1.8.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/shell/target/alluxio-shell-1.8.0-SNAPSHOT.jar to org.alluxio/alluxio-shell/1.8.0-SNAPSHOT/alluxio-shell-1.8.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/shell/target/alluxio-shell-1.8.0-SNAPSHOT-sources.jar to org.alluxio/alluxio-shell/1.8.0-SNAPSHOT/alluxio-shell-1.8.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/tests/pom.xml to org.alluxio/alluxio-tests/1.8.0-SNAPSHOT/alluxio-tests-1.8.0-SNAPSHOT.pomchannel stoppedArchiving artifacts
Test FAILed.

@Mitisky

This comment has been minimized.

Copy link
Contributor Author

commented May 29, 2018

jenkins, test this please

@AmplabJenkins

This comment has been minimized.

Copy link

commented May 29, 2018

Merged build finished. Test PASSed.

@AmplabJenkins

This comment has been minimized.

Copy link

commented May 29, 2018

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/19878/
Test PASSed.

@Mitisky

This comment has been minimized.

Copy link
Contributor Author

commented May 30, 2018

@apc999 PTAL.Thanks

@yupeng9 yupeng9 requested a review from apc999 Jun 2, 2018
BlockStoreLocation oldLocation, BlockStoreLocation newLocation, boolean wait)
throws BlockDoesNotExistException, BlockAlreadyExistsException,
InvalidWorkerStateException, IOException {
long lockId = wait ? mLockManager.lockBlock(sessionId, blockId, BlockLockType.WRITE) :

This comment has been minimized.

Copy link
@yupeng9

yupeng9 Jun 11, 2018

Collaborator

I feel the wait and retry logic should be handled by the client but not the server. @apc999 what do you think?

@alluxio-bot

This comment has been minimized.

Copy link
Collaborator

commented Jul 4, 2018

Automated checks report:

  • Valid pull request title: PASS
  • Contains link to JIRA ticket: PASS
  • Commits associated with Github account: PASS

All checks passed!

1 similar comment
@alluxio-bot

This comment has been minimized.

Copy link
Collaborator

commented Jul 6, 2018

Automated checks report:

  • Valid pull request title: PASS
  • Contains link to JIRA ticket: PASS
  • Commits associated with Github account: PASS

All checks passed!

@Reidddddd

This comment has been minimized.

Copy link
Contributor

commented Jul 13, 2018

Thanks for ping me here @Mitisky
But i doubt it is a good solution, not only moveBlock but also other write behavior can get blocked.
And the root cause why is described in https://alluxio.atlassian.net/browse/ALLUXIO-3249
FYI @apc999

@Mitisky

This comment has been minimized.

Copy link
Contributor Author

commented Jul 16, 2018

Thanks for your suggestion @Reidddddd
Indeed your description is the root reason. My solution is to avoid competition between write operations.Do you have a good idea about this now?

@Mitisky

This comment has been minimized.

Copy link
Contributor Author

commented Jul 19, 2018

@apc999 I encountered another situation the worker got stuck, which form a dead lock.Almost all thread is waiting to promote a block which run out of the thread pool.It will block my task forever and the locked block will never be released.I add a waiting time parameter in the promote method too.It can solve my problem,but I think this part code logic should be reviewed.

@AmplabJenkins

This comment has been minimized.

Copy link

commented Jul 30, 2018

Can one of the admins verify this patch?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.