Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

https://github.com/lukas-krecan/ShedLock/issues/366 added provider fo… #371

Merged
merged 3 commits into from Jan 10, 2021

Conversation

grofoli
Copy link
Contributor

@grofoli grofoli commented Jan 3, 2021

Implemented a provider for etcd like requested by @goudai in #366.

Note that etcds leases only support TTLs in seconds, therefore the integration tests needed some adjustments for this provider.

@lukas-krecan
Copy link
Owner

Thanks a lot, I will take a look at it next week.

Copy link
Owner

@lukas-krecan lukas-krecan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again. As I am not etcd expert, I have few questions

  1. Can you please document it. Especially please explain how the leases are used.
  2. If I understand the docs correctly, lease TTL is the minumum time, the server can decide to keep the lease longer. Can it really happen?
  3. In theory we could implement it without leases, just by using Transactions. The advantage of leases is the keep-alive mechanism, but the code without them might be easier. Did you consider it?

throw new LockException("Can not remove node", e);
}
} else {
Long leaseId = etcdTemplate.createLease(keepLockFor);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about the old lease?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be timed out and then removed, eventually.

} else {
Long leaseId = etcdTemplate.createLease(keepLockFor);

etcdTemplate.updateLease(key, buildValue(), leaseId);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the method updateLease update lease or the value? Do we really want to update the value?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the lease for the key is updated, similar to the CLI command etcdctl put foo2 bar2 --lease 694d76c7c330b909

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is one missing answer, I owe you: whether or not we want to update the lease value.

Well, this should implement the lockAtLeast functionality. I checked the redis-jedis provider before starting this implementation, and it has a similar implementation AFAIK.

As always, I am open to any suggestions/improvements, though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I renamed the method to putWithLeaseId, because this is actually what the method does :-)

@grofoli
Copy link
Contributor Author

grofoli commented Jan 3, 2021

@lukas-krecan I am very open to simplify the code. What would be the idea to solve the timeout?

@lukas-krecan
Copy link
Owner

@lukas-krecan I am very open to simplify the code. What would be the idea to solve the timeout?

I think it's better with leases. The alternative would be to do compare and swap either based on versions or value containing the lock validity. But let's keep the leases.

@lukas-krecan
Copy link
Owner

lukas-krecan commented Jan 4, 2021

I dug deeper to jetcd and if I understand it correctly, for keep-alive to work one has to call leaseClient.keepAlive. Is that correct?

@grofoli
Copy link
Contributor Author

grofoli commented Jan 5, 2021

I dug deeper to jetcd and if I understand it correctly, for keep-alive to work one has to call leaseClient.keepAlive. Is that correct?

I called keepAlive in an earlier local version of mine, but after some tests with etcdctl I found out that it should work without calling it. As far as I remember, the integration test testing the timeout even failed, which might be connected to fact that keepAlive keeps the lease alive forever.
At least this is how I understood it.

@grofoli
Copy link
Contributor Author

grofoli commented Jan 5, 2021

  1. Can you please document it. Especially please explain how the leases are used.

Yes, sure, I will.

@lukas-krecan
Copy link
Owner

Thanks, I will look at it over the weekend. If you agree, I will do some minor cleanup. I feel it's time consuming to polish the code through PR comments, so I prefer to do minor changes myself. I plan to change the module name to something like shedlock-provider-etcd-jetcd as I am quite nervous from the Beta nature of the jetcd API an deeper name structure would allow us to create etcd-jetcd1 in the future if needed. And maybe I will try to release the first lease, after we create the second one in the unlock method (if it's possible). It bothers me to just leave it there.

@grofoli
Copy link
Contributor Author

grofoli commented Jan 6, 2021

Thanks, I will look at it over the weekend. If you agree, I will do some minor cleanup. I feel it's time consuming to polish the code through PR comments, so I prefer to do minor changes myself. I plan to change the module name to something like shedlock-provider-etcd-jetcd as I am quite nervous from the Beta nature of the jetcd API an deeper name structure would allow us to create etcd-jetcd1 in the future if needed. And maybe I will try to release the first lease, after we create the second one in the unlock method (if it's possible). It bothers me to just leave it there.

Yes, sounds good. If you created the branch issue-366-etcd-support in your repository as well, I could switch the PR's target branch from master to issue-366-etcd-support. WDYT? Would this be a good option for you?

@lukas-krecan
Copy link
Owner

I just planned to add commits on top of yours. The way I usually do it is that I create a new PR from your branch. So feel free to do whatever changes you want to this PR and over the weekend I will build on top of it.

@lukas-krecan lukas-krecan merged commit 6b52104 into lukas-krecan:master Jan 10, 2021
@lukas-krecan
Copy link
Owner

Released as 4.20.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants