-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
release-22.2: cli: add --redact
flag to debug zip
, redact tables
#88266
Merged
abarganier
merged 2 commits into
cockroachdb:release-22.2
from
abarganier:backport22.2-86180
Sep 20, 2022
Merged
release-22.2: cli: add --redact
flag to debug zip
, redact tables
#88266
abarganier
merged 2 commits into
cockroachdb:release-22.2
from
abarganier:backport22.2-86180
Sep 20, 2022
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
As we step up our efforts to create PCI compliant configurations of CockroachDB, we need to create a PCI compliant form of `debug zip` bundles. This commit updates the CLI flags supported by the `debug zip` command accordingly. First, it deprecates the narrowly scoped `--redact-logs` flag in favor of the broader scoped `--redact` flag. This flag takes over as a superset of the `--redact-logs` flag, applying not only to logs but to all other areas of debug zip as well. One exception that's been noted by our compliance team is that range key data is acceptable to keep unredacted in `debug zip` bundles, as it's been deemed necessary to support the product. This is noted in the `--redact` flag's documentation. Release note (cli change): `debug zip`'s `--redact-logs` flag has been deprecated in favor of the `--redact` flag, which applies to a broader scope than just logs (but also includes logs). The new `--redact` flag will trigger the redaction of all sensitive data in debug zip bundles, except for range keys, which have been deemed necessary to keep unredacted as they are essential to support CockroachDB. The `--redact-logs` flag will still remain, but users of debug zip will be warned of its deprecation if they use it, and it will be interpreted as `--redact` instead. Release justification: low risk, high value change necessary for upcoming compliance mandates
As part of our initiative to make CockroachDB PCI compliant, source of observability data need to support a redacted form to meet our goals. This commit updates the `debug zip` handler to omit sensitive columns from its dumps of `crdb_internal`/`system` table contents. This is done via a `DebugZipTableRegistry` where each table must be registered with a list of non-sensitive columns, or a custom query that's appropriate in cases where we want to redact. In the future we hope to move this responsibility into the `crdb_internal`/`system` table handlers themselves, perhaps via a session setting to indicate whether or not the query should be redacted. However, for the short term, this solution gets us to a baseline level of compliance quickly while we develop more robust solutions. Release note (security update): The following types of data are now considered "safe" for reporting from within debug.zip: - Range start/end keys, which can include data from any indexed SQL column. - Key spans, which can include data from any indexed SQL column. - Usernames and role names. - SQL object names (including DB, schema, table, sequence, view, type, and UDF names) Release justification: high value observability changes necessary to meet our upcoming compliance mandates.
Thanks for opening a backport. Please check the backport criteria before merging:
If some of the basic criteria cannot be satisfied, ensure that the exceptional criteria are satisfied within.
Add a brief release justification to the body of your PR to justify this backport. Some other things to consider:
|
dhartunian
approved these changes
Sep 20, 2022
TFTR! |
This was referenced Sep 20, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As we step up our efforts to create PCI compliant configurations of
CockroachDB, we need to create a PCI compliant form of
debug zip
bundles.
This patch updates the CLI flags supported by the
debug zip
commandaccordingly. First, it deprecates the narrowly scoped
--redact-logs
flag in favor of the broader scoped
--redact
flag. Thisflag takes over as a superset of the
--redact-logs
flag, applyingnot only to logs but to all other areas of debug zip as well.
One exception that's been noted by our compliance team is that range
key data is acceptable to keep unredacted in
debug zip
bundles, asit's been deemed necessary to support the product. This is noted
in the
--redact
flag's documentation.This commit also updates the debug zip logic to dump
system
and
crdb_internal
tables via use of a DebugZipTableRegistry,where non-sensitive columns are explicitly defined for each
table, and all others are excluded from the the debug zip bundle
if the
--redact
flag is passed.In the future we hope to move this table-wise redaction
functionality further server side into the query handlers themselves,
perhaps using a session setting. However, for the short term,
this solution gets us to a baseline level of compliance quickly
while we develop more robust solutions.
Release note (cli change):
debug zip
's--redact-logs
flag hasbeen deprecated in favor of the
--redact
flag, which applies toa broader scope than just logs (but also includes logs). The new
--redact
flag will trigger the redaction of all sensitive datain debug zip bundles, except for range keys, which have been
deemed necessary to keep unredacted as they are essential to support
CockroachDB. The
--redact-logs
flag will still remain, butusers of debug zip will be warned of its deprecation if they
use it, and it will be interpreted as
--redact
instead.Release justification: low risk, high value change necessary
for upcoming compliance mandatesBackport 2/2 commits from #86180.
/cc @cockroachdb/release
Release justification: low impact, high value change to meet our compliance baseline goals in CockroachDB v22.2
Addresses #86593