Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

release-22.2: cli: add --redact flag to debug zip, redact tables #88266

Merged
merged 2 commits into from
Sep 20, 2022

Conversation

abarganier
Copy link
Contributor

@abarganier abarganier commented Sep 20, 2022

As we step up our efforts to create PCI compliant configurations of
CockroachDB, we need to create a PCI compliant form of debug zip
bundles.

This patch updates the CLI flags supported by the debug zip command
accordingly. First, it deprecates the narrowly scoped --redact-logs
flag in favor of the broader scoped --redact flag. This
flag takes over as a superset of the --redact-logs flag, applying
not only to logs but to all other areas of debug zip as well.

One exception that's been noted by our compliance team is that range
key data is acceptable to keep unredacted in debug zip bundles, as
it's been deemed necessary to support the product. This is noted
in the --redact flag's documentation.

This commit also updates the debug zip logic to dump system
and crdb_internal tables via use of a DebugZipTableRegistry,
where non-sensitive columns are explicitly defined for each
table, and all others are excluded from the the debug zip bundle
if the --redact flag is passed.

In the future we hope to move this table-wise redaction
functionality further server side into the query handlers themselves,
perhaps using a session setting. However, for the short term,
this solution gets us to a baseline level of compliance quickly
while we develop more robust solutions.

Release note (cli change): debug zip's --redact-logs flag has
been deprecated in favor of the --redact flag, which applies to
a broader scope than just logs (but also includes logs). The new
--redact flag will trigger the redaction of all sensitive data
in debug zip bundles, except for range keys, which have been
deemed necessary to keep unredacted as they are essential to support
CockroachDB. The --redact-logs flag will still remain, but
users of debug zip will be warned of its deprecation if they
use it, and it will be interpreted as --redact instead.

Release justification: low risk, high value change necessary
for upcoming compliance mandatesBackport 2/2 commits from #86180.

/cc @cockroachdb/release

Release justification: low impact, high value change to meet our compliance baseline goals in CockroachDB v22.2

Addresses #86593

As we step up our efforts to create PCI compliant configurations of
CockroachDB, we need to create a PCI compliant form of `debug zip`
bundles.

This commit updates the CLI flags supported by the `debug zip` command
accordingly. First, it deprecates the narrowly scoped `--redact-logs`
flag in favor of the broader scoped `--redact` flag. This
flag takes over as a superset of the `--redact-logs` flag, applying
not only to logs but to all other areas of debug zip as well.

One exception that's been noted by our compliance team is that range
key data is acceptable to keep unredacted in `debug zip` bundles, as
it's been deemed necessary to support the product. This is noted
in the `--redact` flag's documentation.

Release note (cli change): `debug zip`'s `--redact-logs` flag has
been deprecated in favor of the `--redact` flag, which applies to
a broader scope than just logs (but also includes logs). The new
`--redact` flag will trigger the redaction of all sensitive data
in debug zip bundles, except for range keys, which have been
deemed necessary to keep unredacted as they are essential to support
CockroachDB. The `--redact-logs` flag will still remain, but
users of debug zip will be warned of its deprecation if they
use it, and it will be interpreted as `--redact` instead.

Release justification: low risk, high value change necessary
for upcoming compliance mandates
As part of our initiative to make CockroachDB PCI compliant,
source of observability data need to support a redacted form
to meet our goals.

This commit updates the `debug zip` handler to omit sensitive
columns from its dumps of `crdb_internal`/`system` table contents.
This is done via a `DebugZipTableRegistry` where each table must be
registered with a list of non-sensitive columns, or a custom
query that's appropriate in cases where we want to redact.

In the future we hope to move this responsibility into the
`crdb_internal`/`system` table handlers themselves, perhaps via
a session setting to indicate whether or not the query should
be redacted. However, for the short term, this solution gets
us to a baseline level of compliance quickly while we develop
more robust solutions.

Release note (security update): The following types of data
are now considered "safe" for reporting from within debug.zip:
- Range start/end keys, which can include data from any indexed
  SQL column.
- Key spans, which can include data from any indexed SQL column.
- Usernames and role names.
- SQL object names (including DB, schema, table, sequence, view,
  type, and UDF names)

Release justification: high value observability changes necessary
to meet our upcoming compliance mandates.
@abarganier abarganier requested a review from a team September 20, 2022 17:48
@abarganier abarganier requested review from a team as code owners September 20, 2022 17:48
@blathers-crl
Copy link

blathers-crl bot commented Sep 20, 2022

Thanks for opening a backport.

Please check the backport criteria before merging:

  • Patches should only be created for serious issues or test-only changes.
  • Patches should not break backwards-compatibility.
  • Patches should change as little code as possible.
  • Patches should not change on-disk formats or node communication protocols.
  • Patches should not add new functionality.
  • Patches must not add, edit, or otherwise modify cluster versions; or add version gates.
If some of the basic criteria cannot be satisfied, ensure that the exceptional criteria are satisfied within.
  • There is a high priority need for the functionality that cannot wait until the next release and is difficult to address in another way.
  • The new functionality is additive-only and only runs for clusters which have specifically “opted in” to it (e.g. by a cluster setting).
  • New code is protected by a conditional check that is trivial to verify and ensures that it only runs for opt-in clusters.
  • The PM and TL on the team that owns the changed code have signed off that the change obeys the above rules.

Add a brief release justification to the body of your PR to justify this backport.

Some other things to consider:

  • What did we do to ensure that a user that doesn’t know & care about this backport, has no idea that it happened?
  • Will this work in a cluster of mixed patch versions? Did we test that?
  • If a user upgrades a patch version, uses this feature, and then downgrades, what happens?

@cockroach-teamcity
Copy link
Member

This change is Reviewable

@abarganier abarganier removed request for a team September 20, 2022 18:45
@abarganier
Copy link
Contributor Author

TFTR!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants