Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(pii): Adopt new selectors #818

Merged
merged 10 commits into from
Nov 2, 2020
Merged

feat(pii): Adopt new selectors #818

merged 10 commits into from
Nov 2, 2020

Conversation

flub
Copy link
Contributor

@flub flub commented Oct 22, 2020

This adds new selectors for the minidump for the heap and stack.

@untitaker as the new tests show this sadly means any generic rule for $binary will apply to the stack. Which is what we were trying to avoid. So I'm not sure this proposed change will work. I guess the question is whether we still want to this for just the heap or not.

This adds new selectors for the minidump for the heap and stack.
@flub flub requested review from a team and untitaker October 22, 2020 16:00
Copy link
Member

@untitaker untitaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

preliminary review... Pii::Maybe should prevent $binary from applying, not sure why it wouldn't work... but the tests seem broken

let slice = data
.get_mut(range)
.ok_or(ScrubMinidumpError::InvalidAddress)?;
let attrs = Cow::Owned(FieldAttrs::new().pii(Pii::Maybe));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't have to reinstantiate this I believe? Shouldn't you be able to pass down a Cow::Borrowed for the individual visits?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't pass this to multiple calls and it also is not clonable. And since in the future we'll want some to be set to Pii::Yes de-duplicating this seems more cognitive overhead than it saves. That was the logic anyway, I don't feel strongly so could change this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FieldAttrs is Copy

relay-general/src/pii/minidumps.rs Outdated Show resolved Hide resolved
relay-general/src/pii/minidumps.rs Outdated Show resolved Hide resolved
relay-general/src/pii/minidumps.rs Outdated Show resolved Hide resolved
@flub
Copy link
Contributor Author

flub commented Oct 22, 2020

Good points on the tests, I fixed them but it doesn't actually change the outcome. In particular the test_stack_scrubbing_valuetype_not_fully_qualified still fails, but similar for the wildcards.

@flub flub marked this pull request as draft October 22, 2020 16:23
@flub
Copy link
Contributor Author

flub commented Oct 23, 2020

On reflection the original asserts were fine due to the @anything:mask rules, either they were scrubbed or they were not. And arguably these asserts are easier to read? Depends on how readable you find ! (one of the cases where I miss python's not).

@untitaker
Copy link
Member

untitaker commented Oct 23, 2020

@flub the remaining one test works as designed right now, * is legitimate in fully qualified selectors due to arrays (eventually we will land the refactor that really only allows * in array-index positions if pii = maybe)

@flub
Copy link
Contributor Author

flub commented Oct 30, 2020

@untitaker I added some tests in selectors.rs which show a bunch of new problems. I spent some time trying to come up with something constructive rather than just find problems but... by now I'm wondering if this is too difficult and perhaps just giving up and shipping what we have is better.

serde_json::json!(
{
"applications": {
"$minidump.$stack_memory": ["@anything:mask"],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This selector doesn't have to work, customers currently use $stack_memory

Floris Bruynooghe added 3 commits November 2, 2020 17:16
With some justifications.  We'll just have to document these things as
known issues.
@flub flub marked this pull request as ready for review November 2, 2020 16:31
@flub flub requested a review from untitaker November 2, 2020 16:32
@flub
Copy link
Contributor Author

flub commented Nov 2, 2020

Frankly, I think this entirely defeats the point now since any selector with $binary will never match anything marked as Pii::Maybe we might as well not have bothered and stuck with the existing $stack_memory. I can't actually document any meaningful use of $binary currently so this seems rather pointless. The proposal to consider value types a set might be more useful in the future.

So I'm actually -1 on merging this.

The changes in attachments.rs should arguably still be merged though. They seem like they should apply in any case.

@flub
Copy link
Contributor Author

flub commented Nov 2, 2020

To address my own comment, once we change some fields to Pii::True this will make some sense again, so merging it anyway.

@flub flub merged commit e01273a into master Nov 2, 2020
@flub flub deleted the feat/minidump-selectors branch November 2, 2020 17:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants