Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove the contributor locks #415

Open
ibaryshnikov opened this issue Sep 30, 2021 · 0 comments
Open

Remove the contributor locks #415

ibaryshnikov opened this issue Sep 30, 2021 · 0 comments
Labels
1 high priority enhancement New feature or request

Comments

@ibaryshnikov
Copy link

Contributor's logic heavily relies on acquiring the locks. The lock itself signals that the contributor is working on a specific task at the moment. And the task is a combination of chunk id and contribution id. We have a lot of logic related to handling contributor locks. I've made an experiment moving the verifier lock logic to LRU cache on the application side. It allowed to remove the verifier lock logic from phase1-coordinator and improved overall stability of the service. We can expect the same outcome for the contributors: improved stability and better user experience.

Key parts of the change:

  • remove the contributor locks in phase1-coordinator
  • use LRU cache for assigned tasks in coordinator web service
  • revisit contributor drop logic
  • use assigned_tasks and completed_tasks in ParticipantInfo to choose what's the next task to work on. Remove pending_tasks, disposing_tasks and disposed_tasks
@ibaryshnikov ibaryshnikov added enhancement New feature or request 1 high priority labels Sep 30, 2021
@ibaryshnikov ibaryshnikov added this to the Phase II Setup milestone Sep 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1 high priority enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant