Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reduce timeout when acquring lock #4481

Merged
merged 1 commit into from
Sep 1, 2020
Merged

Conversation

skshetry
Copy link
Member

Previously, it'd sleep for 5s. Now, it should retry 6 times
before timing out.

Also reduced the timeout to 3s.

Thank you for the contribution - we'll try to review it as soon as possible. πŸ™

Previously, it'd sleep for 5s. Now, it should retry 6 times
before timing out.

Also reduced the timeout to 3s.
@skshetry skshetry added the enhancement Enhances DVC label Aug 27, 2020
@skshetry skshetry requested a review from efiop August 27, 2020 05:32
@skshetry skshetry self-assigned this Aug 27, 2020
@skshetry skshetry added this to In progress in DVC 25 August - 8 September 2020 via automation Aug 27, 2020
@skshetry skshetry moved this from In progress to Review in progress in DVC 25 August - 8 September 2020 Aug 27, 2020
Comment on lines +59 to +61
retries = 6
delay = DEFAULT_TIMEOUT / retries
lock_retry = retry(retries, LockError, timeout=delay)(self._do_lock)
Copy link
Member

@efiop efiop Aug 27, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

delay is clearly incorrect terminology πŸ™‚ Let's just

Suggested change
retries = 6
delay = DEFAULT_TIMEOUT / retries
lock_retry = retry(retries, LockError, timeout=delay)(self._do_lock)
lock_retry = retry(RETRIES, LockError, timeout=TIMEOUT)(self._do_lock)

with, e.g.

TIMEOUT = 0.1
RETRIES = 50

and then just patch for tests as needed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you think so?
From funcy's documentation:

There will be a delay in timeout seconds between tries

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@skshetry Precisely because it is called timeout and not delay. Dividing stuff is not obvious, better to keep explicit TIMEOUT and RETRIES (and not hardcode it).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like timeout has two different meanings? I was only familiar with one.

Copy link
Member

@efiop efiop Aug 27, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@skshetry They are pretty much the same, but you are thinking about the total lock timeout, while I'm talking about this particular piece of code, where timeout is about the time between tries. In the future there might be a total-timeout CLI option, but right now I'm just concerned about this piece of code.

Copy link
Member Author

@skshetry skshetry Sep 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@efiop, HardlinkLock uses DEFAULT_TIMEOUT as well. I don't want to have two different timeouts, so I'd have to do something like following:

Suggested change
retries = 6
delay = DEFAULT_TIMEOUT / retries
lock_retry = retry(retries, LockError, timeout=delay)(self._do_lock)
delay = DEFAULT_TIMEOUT / RETRIES
lock_retry = retry(RETRIES, LockError, timeout=delay)(self._do_lock)

which is no different from this PR. Also, I want to mock as less as possible, and just mocking total timeout is enough for the tests.

@efiop efiop merged commit 8a27032 into iterative:master Sep 1, 2020
DVC 25 August - 8 September 2020 automation moved this from Review in progress to Done Sep 1, 2020
@skshetry skshetry deleted the lock-retry branch September 1, 2020 16:57
@skshetry skshetry changed the title lock: retry when trying to acquire lock reduce timeout when acquring lock Sep 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhances DVC
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

None yet

2 participants