Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhancement(redact): Add redactor option #640

Merged
merged 3 commits into from
Feb 6, 2024

Conversation

tmccombs
Copy link
Contributor

@tmccombs tmccombs commented Jan 12, 2024

This option allows you to specify either a custom string, or a hash function to use as the redactor, in addition to the current behavior of the fixed string "[Redacted]".

I'm opening this as a draft to make sure this is a good approach.

If this is the API that is desired, I can add some addition examples and tests.

closes: #633

  • add examples and tests

}
}
}

/// Compute the digest of some bytes as hex string
fn hex_digest<T: digest::Digest>(value: &[u8]) -> String {
hex::encode(T::digest(value))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would it be better to use base64 encoding? or have another option for the text encoding so the caller can choose between hex and base64?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a potential user of this, what would you find most useful? My instinct is that base64 would be more common. Maybe we could use the same approach as we use for the filters argument to this function where additional options could be specified, if needed, for a given redactor like:

redact("my id is 123456", filters: [r'\d+'], redactor: {name: "sha256", encoding: "base64"})

But where you could also just specify:

redact("my id is 123456", filters: [r'\d+'], redactor: "sha256")

For the default behavior (say base64 encoding). What would you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My instinct is that base64 would be more common

that is my instinct as well.

@tmccombs tmccombs marked this pull request as ready for review January 26, 2024 18:29
Copy link
Collaborator

@pront pront left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this looks good!

  • Can you also update the vector docs (see this example) please?
  • Examples look great. Can you add similar unit tests?

src/stdlib/redact.rs Show resolved Hide resolved
@tmccombs
Copy link
Contributor Author

vector documentation PR: vectordotdev/vector#19749

@pront
Copy link
Collaborator

pront commented Jan 30, 2024

vector documentation PR: vectordotdev/vector#19749

Awesome, thank you.

Copy link
Collaborator

@pront pront left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added one suggestion. Otherwise, this looks good. Thank you!

@pront pront enabled auto-merge January 30, 2024 22:55
auto-merge was automatically disabled January 31, 2024 00:27

Head branch was pushed to by a user without write access

@pront pront enabled auto-merge February 1, 2024 14:58
auto-merge was automatically disabled February 1, 2024 21:43

Head branch was pushed to by a user without write access

@pront
Copy link
Collaborator

pront commented Feb 6, 2024

@tmccombs this looks good to me. Do you think it is ready to merge?

Copy link
Member

@jszwedko jszwedko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to see this addition!

@jszwedko
Copy link
Member

jszwedko commented Feb 6, 2024

@tmccombs it looks like there are some merge conflicts now too when you get a second.

This option allows you to specify either a custom string, or a hash function to use as the redactor,
in addition to the current behavior of the fixed string "[Redacted]".

Fixes vectordotdev#633
To specify how to encode hash values.
@tmccombs
Copy link
Contributor Author

tmccombs commented Feb 6, 2024

yeah, I think this is ready to merge.

@pront pront enabled auto-merge February 6, 2024 16:51
@pront pront added this pull request to the merge queue Feb 6, 2024
Merged via the queue into vectordotdev:main with commit 47c06a5 Feb 6, 2024
11 checks passed
@tmccombs tmccombs deleted the more-redactors branch February 6, 2024 18:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

redact a pattern, but replace with a hash instead of a string
3 participants