Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backport of keyring: support prepublishing keys into release/1.8.x #23651

Conversation

hc-github-team-nomad-core
Copy link
Contributor

Backport

This PR is auto-generated from #23577 to be assessed for backporting due to the inclusion of the label backport/1.8.x.

The below text is copied from the body of the original PR.


When a root key is rotated, the servers immediately start signing Workload Identities with the new active key. But workloads may be using those WI tokens to sign into external services, which may not have had time to fetch the new public key and which might try to fetch new keys as needed.

Add support for prepublishing keys. Prepublished keys will be visible in the JWKS endpoint but will not be used for signing or encryption until their PublishTime. Update the periodic key rotation to prepublish keys at half the root_key_rotation_threshold window, and promote prepublished keys to active after the PublishTime.

This changeset also fixes three bugs in periodic root key rotation and garbage collection, none of which can be safely fixed without implementing prepublishing:

  • Periodic root key rotation would never happen because the default root_key_rotation_threshold of 720h exceeds the 72h maximum window of the FSM time table. We now compare the CreateTime against the wall clock time instead of the time table. (We expect to remove the time table in future work, ref GC limits > 3 days are in effect infinite b/c of FSM timetable limit #16359)
  • Root key garbage collection could GC keys that were used to sign identities. We now wait until root_key_rotation_threshold + root_key_gc_threshold before GC'ing a key.
  • When rekeying a root key, the core job did not mark the key as inactive after the rekey was complete.

Ref: https://hashicorp.atlassian.net/browse/NET-10398
Ref: https://hashicorp.atlassian.net/browse/NET-10280
Fixes: #19669
Fixes: #23528
Fixes: #19368


Notes for reviewers:

  • This is strictly speaking an enhancement but because it's the only way to safely fix the 3 bugs listed (i.e. without orphaning Workload Identities), I'm planning on backporting it to supported Nomad Enterprise versions. Alternately, we could remove periodic key rotation entirely from those older versions and that would "fix" the bug there.
  • I'll need to manually backport the docs changes to 1.7.x and 1.6.x CE branches.

Overview of commits

@hc-github-team-nomad-core hc-github-team-nomad-core force-pushed the backport/keyring-rotation-and-prepublish/seriously-national-panda branch from e1010b6 to e3ab362 Compare July 19, 2024 17:30
@tgross tgross merged commit 7ac0c38 into release/1.8.x Jul 19, 2024
21 checks passed
@tgross tgross deleted the backport/keyring-rotation-and-prepublish/seriously-national-panda branch July 19, 2024 17:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants