Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for generating Delta CRLs #16773

Merged
merged 6 commits into from
Aug 29, 2022
Merged

Support for generating Delta CRLs #16773

merged 6 commits into from
Aug 29, 2022

Conversation

cipherboy
Copy link
Contributor

@cipherboy cipherboy commented Aug 17, 2022

While switching to periodic rebuilds of CRLs alleviates the constant
rebuild pressure on Vault during times of high revocation, the CRL
proper becomes stale. One response to this is to switch to OCSP, but not
every system has support for this. Additionally, OCSP usually requires
connectivity and isn't used to augment a pre-distributed CRL (and is
instead used independently).

By generating delta CRLs containing only new revocations, an existing
CRL can be supplemented with newer revocations without requiring Vault
to rebuild all complete CRLs. Admins can periodically fetch the delta
CRL and add it to the existing CRL and applications should be able to
support using serials from both.

Because delta CRLs are emptied when the next complete CRL is rebuilt, it
is important that applications fetch the delta CRL and correlate it to
their complete CRL; if their complete CRL is older than the delta CRL's
extension number, applications MUST fetch the newer complete CRL to
ensure they have a correct combination.

This modifies the revocation process and adds several new configuration
options, controlling whether Delta CRLs are enabled and when we'll
rebuild it.


@cipherboy cipherboy added this to the 1.12.0-rc1 milestone Aug 17, 2022
@cipherboy cipherboy force-pushed the cipherboy-identify-issuer-on-revocation branch 2 times, most recently from c99f0c2 to 93da855 Compare August 23, 2022 20:36
@cipherboy cipherboy force-pushed the cipherboy-delta-crls branch 2 times, most recently from 032235f to 76bad87 Compare August 25, 2022 16:23
@cipherboy cipherboy changed the base branch from cipherboy-identify-issuer-on-revocation to main August 25, 2022 16:23
@cipherboy cipherboy force-pushed the cipherboy-delta-crls branch 2 times, most recently from 05f2f2f to b814748 Compare August 25, 2022 17:08
Copy link
Contributor

@kitography kitography left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Holding off on an approval until tests (that I could walk through to try to make sense of the locks-on-build-where), but I think you are on the right track.

builtin/logical/pki/crl_util.go Outdated Show resolved Hide resolved
builtin/logical/pki/crl_util.go Outdated Show resolved Hide resolved
builtin/logical/pki/crl_util.go Show resolved Hide resolved
}

// Last is setup during newCRLBuilder(...), so we don't need to deal with
// a zero condition.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is super smart

Copy link
Contributor

@stevendpclark stevendpclark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A great start for this feature! I think I need another pass on Monday, but added some thoughts for this initial review

Comment on lines +296 to +321
cfg, err := cb.getConfigWithUpdate(sc)
if err != nil {
return err
}

if !cfg.EnableDelta {
// We explicitly do not update the last check time here, as we
// want to persist the last rebuild window if it hasn't been set.
return nil
}

deltaRebuildDuration, err := time.ParseDuration(cfg.DeltaRebuildInterval)
if err != nil {
return err
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this happen prior to us grabbing the _builder.Lock()?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to fix the config instead? Or do you want to delay the lock until after this conditional, so if fetching the config in the first place fails, we bail?

That said, we shouldn't bail here as we verify this condition during setting of the config :P

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either option is acceptable to me, if we are fixing then the lock prior makes sense. If we are just going to bail might as well do it prior to grabbing the lock.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we wait to grab the lock until we commit to doing the actual rebuild then...? Hmmm

builtin/logical/pki/crl_util.go Outdated Show resolved Hide resolved
builtin/logical/pki/crl_util.go Outdated Show resolved Hide resolved
if _, err := time.ParseDuration(deltaRebuildInterval); err != nil {
return logical.ErrorResponse(fmt.Sprintf("given delta_rebuild_interval could not be decoded: %s", err)), nil
}
config.DeltaRebuildInterval = deltaRebuildInterval
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Validate that we haven't been passed in a value less than periodFunc interval?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sadly we don't have a response right now for this handler, so we can't add warnings... Do we want to return the set structure instead?

We (abuse) this code path in the test cases right now, like we do with AutoRebuild grace period, maybe we should later refactor the config to return the set config and add the warnings?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good.

builtin/logical/pki/crl_util.go Outdated Show resolved Hide resolved
builtin/logical/pki/backend.go Show resolved Hide resolved
builtin/logical/pki/crl_util.go Show resolved Hide resolved
While switching to periodic rebuilds of CRLs alleviates the constant
rebuild pressure on Vault during times of high revocation, the CRL
proper becomes stale. One response to this is to switch to OCSP, but not
every system has support for this. Additionally, OCSP usually requires
connectivity and isn't used to augment a pre-distributed CRL (and is
instead used independently).

By generating delta CRLs containing only new revocations, an existing
CRL can be supplemented with newer revocations without requiring Vault
to rebuild all complete CRLs. Admins can periodically fetch the delta
CRL and add it to the existing CRL and applications should be able to
support using serials from both.

Because delta CRLs are emptied when the next complete CRL is rebuilt, it
is important that applications fetch the delta CRL and correlate it to
their complete CRL; if their complete CRL is older than the delta CRL's
extension number, applications MUST fetch the newer complete CRL to
ensure they have a correct combination.

This modifies the revocation process and adds several new configuration
options, controlling whether Delta CRLs are enabled and when we'll
rebuild it.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Thanks Steve!

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Copy link
Contributor

@stevendpclark stevendpclark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@cipherboy
Copy link
Contributor Author

@stevendpclark I've updated the PR so it only invokes the periodic func on active nodes.

We need to ensure we read the updated config (in case of OCSP request
handling on standby nodes), but otherwise want to avoid CRL/DeltaCRL
re-building.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
@cipherboy cipherboy merged commit 041d837 into main Aug 29, 2022
@cipherboy cipherboy deleted the cipherboy-delta-crls branch December 1, 2022 14:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants