Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add handling for "etcdserver: revision of auth store is old" #1103

Merged
merged 1 commit into from
Nov 16, 2022

Conversation

darvay
Copy link

@darvay darvay commented Nov 7, 2022

Error experienced by customers when Role changes are made to the ETCD Cluster.

We are seeing a use case when ETCD Role grants are updated we are getting an error "etcdserver: revision of auth store is old" on all JETCD clients right after the grant is applied. The only fix now is for the client to restart and reconnect.

ETCD Cluster config:

ETCD_AUTH_TOKEN: jwt,pub-key=/auth-token/authToken.pub,priv-key=/auth-token/authToken.key,sign-method=RS256,ttl=8h

@darvay darvay force-pushed the auth_store_issue branch 2 times, most recently from d76e26d to d92c88c Compare November 7, 2022 21:58
@lburgazzoli
Copy link
Collaborator

is there any way to add a test ?

@darvay
Copy link
Author

darvay commented Nov 7, 2022

Thanks for the review @lburgazzoli , I wasn't sure how to trigger this error condition via unit tests, I see WatchUnitTest.java does a check for that error message but didn't see a test for Errors.INVALID_AUTH_TOKEN_ERROR_MESSAGE . We have been relying on an integration test running Etcd 3.5.2 with multiple etcdctl role add/updates to try to trigger the error condition.

@lburgazzoli
Copy link
Collaborator

Yeah we don have test for all the cases but if there is a way to test it, I would really appreciated given my very limited time to support the project

@darvay
Copy link
Author

darvay commented Nov 9, 2022

@lburgazzoli let me know if this test will work

…perienced by customers when Role changes are made to the ETCD Cluster.

We are seeing a use case when ETCD Role grants are updated we are getting an error "etcdserver: revision of auth store is old" on all JETCD clients right after the grant is applied. The only fix now is for the client to restart and reconnect.

ETCD Cluster config:

ETCD_AUTH_TOKEN: jwt,pub-key=/auth-token/authToken.pub,priv-key=/auth-token/authToken.key,sign-method=RS256,ttl=8h

Signed-off-by: darvay <darvay@apple.com>
@darvay
Copy link
Author

darvay commented Nov 15, 2022

@lburgazzoli hope all is well, did you have a chance to review the added test? Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants