Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Redact the API secret for Confluent Cloud sinks from show jobs output #113503

Closed
kathancox opened this issue Oct 31, 2023 · 2 comments · Fixed by #115535
Closed

Redact the API secret for Confluent Cloud sinks from show jobs output #113503

kathancox opened this issue Oct 31, 2023 · 2 comments · Fixed by #115535
Assignees
@wenyihu6
Labels
A-cdc Change Data Capture C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. T-cdc
Milestone
23.2

Comments

@kathancox
Copy link
Contributor

kathancox commented Oct 31, 2023 •
edited by cockroach-jira-scripts

Describe the problem

When you run SHOW CHANGEFEED JOB {job id}; on a changefeed to a confluent-cloud sink, the API secret is not redacted in the SHOW output. This is also the case for regular SHOW JOBS.

To Reproduce

  1. Create a Confluent Cloud kafka sink, and create API keys.
  2. Create a changefeed to the sink passing the API key and secret.
  3. Run SHOW CHANGEFEED JOB on that changefeed job.

Expected behavior
The api_secret parameter value is redacted for a SHOW command.

CockroachDB v23.2.0-alpha.4

Jira issue: CRDB-33028
@kathancox kathancox added C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. A-cdc Change Data Capture labels Oct 31, 2023
@blathers-crl blathers-crl bot added the T-cdc label Oct 31, 2023
@blathers-crl blathers[crl]
Copy link

blathers-crl bot commented Oct 31, 2023

cc @cockroachdb/cdc

@miretskiy miretskiy added this to the 23.2 milestone Nov 8, 2023
@wenyihu6
Copy link
Contributor

wenyihu6 commented Nov 30, 2023 •
edited

It looks like the api_key and api_secret do not show up if we create an external connection before creating the changefeed. But it does show up if we create the changefeed using sinkURI directly.

CREATE EXTERNAL CONNECTION confluent1 AS 'confluent-cloud://nope?api_key=fee&api_secret=bar'
CREATE CHANGEFEED FOR foo INTO 'external://confluent1/
SHOW CHANGEFEED JOBS

does not show secrets

[[921728271371862017 CREATE CHANGEFEED FOR TABLE foo INTO 'external://confluent1/' root running NULL 2023-11-30 15:52:25.316143 +0000 UTC NULL NULL 2023-11-30 15:52:25.323003 +0000 UTC NULL  external://confluent1/ {d.public.foo} NULL json]]

But it does show up if we create changefeed using the sinkURI directly.

CREATE CHANGEFEED FOR foo INTO 'confluent-cloud://nope?api_key=fee&api_secret=bar'

does not redact secrets

[[921728158211244033 CREATE CHANGEFEED FOR TABLE foo INTO 'confluent-cloud://nope?api_key=fee&api_secret=bar' root running NULL 2023-11-30 15:51:50.782255 +0000 UTC NULL NULL 2023-11-30 15:51:50.782255 +0000 UTC NULL  confluent-cloud://nope?api_key=fee&api_secret=bar {d.public.foo} NULL json]]

Is this the intended behaviour or should we redact api_secret?

SHOW CREATE EXTERNAL CONNECTION does not redact the secrets but it seems to be the intended behaviour since it is admin only. I think SHOW CHANGEFEED JOB also has a few privileges required in order to be used.

@wenyihu6 wenyihu6 mentioned this issue Dec 4, 2023
Merged
craig bot pushed a commit that referenced this issue Dec 4, 2023
@wenyihu6 @rickystewart
Merge #115535 #115554 #115557
ee279d9 
115535: changefeedccl: redact user-sensitive info from SHOW JOBS output r=jayshrivastava a=wenyihu6

Previously, `SHOW CHANGEFEED JOB` revealed sensitive user data like `api_secret`
for confluent cloud sinks. This patch now redacts `api_secret`, `sasl_password`,
`client_cert`, and `ca_cert` in the job description and sinkURI output column.

Fixes: #113503

Release note (enterprise change): SHOW CHANGEFEED JOB,  SHOW CHANGEFEED JOBS,
and SHOW JOBS no longer expose user sensitive infromation(`api_secret`,
`sasl_password`, `client_cert`, and `ca_cert`) in the job description and
sinkURI output column would reveal sensitive user information (api_secret,
sasl_password, client_cert, ca_cert).


115554: schemafeed: bump size of test r=rail a=rickystewart

This timed out in CI.

https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Ci_TestsGcpLinuxX8664BigVm_CclUnitTests/12973040?hideProblemsFromDependencies=false&hideTestsFromDependencies=false&expandBuildChangesSection=true&expandBuildDeploymentsSection=true&expandBuildProblemsSection=true&expandBuildTestsSection=true

Epic: none
Release note: None

115557: stress: re-set GITHUB_API_TOKEN r=rail a=rickystewart

This change was made accidentally in #114681.

Epic: CRDB-8308
Release note: None

Co-authored-by: Wenyi Hu <wenyi@cockroachlabs.com>
Co-authored-by: Ricky Stewart <ricky@cockroachlabs.com>
@craig craig bot closed this as completed in d3c9ada Dec 4, 2023
blathers-crl bot pushed a commit that referenced this issue Dec 4, 2023
@wenyihu6
changefeedccl: redact user-sensitive info from SHOW JOBS output
da52700 
Previously, `SHOW CHANGEFEED JOB` revealed sensitive user data like `api_secret`
for confluent cloud sinks. This patch now redacts `api_secret`, `sasl_password`,
`client_cert`, and `ca_cert` in the job description and sinkURI output column.

Fixes: #113503

Release note (enterprise change): SHOW CHANGEFEED JOB,  SHOW CHANGEFEED JOBS,
and SHOW JOBS no longer expose user sensitive infromation(`api_secret`,
`sasl_password`, `client_cert`, and `ca_cert`) in the job description and
sinkURI output column would reveal sensitive user information (api_secret,
sasl_password, client_cert, ca_cert).
@blathers-crl blathers-crl bot mentioned this issue Dec 4, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@wenyihu6 wenyihu6

Labels
A-cdc Change Data Capture C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. T-cdc
Projects
None yet
Milestone
23.2
Development

Successfully merging a pull request may close this issue.

changefeedccl: redact user-sensitive info from SHOW JOBS output wenyihu6/cockroach
4 participants
@jayshrivastava @kathancox @miretskiy @wenyihu6