Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SensitiveDataMasker to exceptions messages #42940

Merged
merged 2 commits into from Nov 21, 2022

Conversation

filimonov
Copy link
Contributor

Changelog category (leave one):

  • Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Allow to remove sensitive information from the exception messages also. Resolves #41418

Information about CI checks: https://clickhouse.com/docs/en/development/continuous-integration/

@robot-ch-test-poll robot-ch-test-poll added the pr-improvement Pull request with some product improvements label Nov 3, 2022
@filimonov filimonov added the can be tested Allows running workflows for external contributors label Nov 3, 2022
@vitlibar vitlibar self-assigned this Nov 4, 2022
Copy link
Contributor

@Enmk Enmk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You also would have to invoke wipeSensitiveData on DB::Exception::addMessage

, remote(remote_)
{
handle_error_code(msg, code, remote, getStackFramePointers());
handle_error_code(msg_masked.msg, code, remote, getStackFramePointers());
Copy link
Member

@vitlibar vitlibar Nov 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've implemented a different solution recently, which is based on a AST tree, not on regular expressions, so my solution is going to work better for more various cases like s3(url, key, secret_key) or CREATE USER IDENTIFIER WITH password. But it seems I forgot to cover this case in my solution, so I'll still have to improve it. I'd prefer to take your test but use that my solution with replacings in AST tree. Ok?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if it can really be done 100% safe on AST level...

Imagine you can't parse the query correctly, but it have sensitive data, or that sensitive data does not come from the query at all. So i think that those 'sensitivedatamaster' is still needed (and also it's ok to have some predefined stuff on AST level)

And test - sure you can reuse it.

Copy link
Member

@vitlibar vitlibar Nov 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't suggest to remove the regular expression approach. In my opinion it's better to use both: first hardcoded wiping based on AST and then customizable wiping based on regexp.

@filimonov filimonov marked this pull request as ready for review November 14, 2022 18:13
@vitlibar vitlibar merged commit 6c2b53f into ClickHouse:master Nov 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
can be tested Allows running workflows for external contributors pr-improvement Pull request with some product improvements
Projects
None yet
Development

Successfully merging this pull request may close these issues.

query_masking_rules not work for exception messages at query_log
4 participants