Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added mask of rule action secret parameters #4788

Conversation

jeansfelix
Copy link
Contributor

Closes #4784

@CLAassistant
Copy link

CLAassistant commented Sep 12, 2019

CLA assistant check
All committers have signed the CLA.

@pull-request-size pull-request-size bot added the size/M PR that changes 30-99 lines. Good size to review. label Sep 12, 2019
@Kami
Copy link
Member

Kami commented Sep 16, 2019

Thanks for the contribution.

The change looks good to me, but we would need some tests for it before we can merge it (including model level tests and API level tests).

We will likely also need to update mandatory include fields for rule API endpoints to ensure secrets masking works correctly when ?include_attributes and ?exclude_attributes API query param filters are used.

@Nicodemos305 Nicodemos305 force-pushed the fix/mask-secrets-rule-parameters branch 2 times, most recently from f00fb0c to 7cfb390 Compare October 10, 2019 17:33
…rs for test_get_all_parameters_mask_with_include_parameters and test_get_one_parameters_mask_with_exclude_parameters for test_get_all_parameters_mask_with_exclude_parameters
@jeansfelix jeansfelix force-pushed the fix/mask-secrets-rule-parameters branch from 7cfb390 to 42eb393 Compare October 10, 2019 18:25
@Nicodemos305
Copy link
Contributor

Dear @Kami , the tests on SO Ubuntu and Centos failed but I couldn't see the error log, would you help me?

@blag
Copy link
Contributor

blag commented Oct 18, 2019

Merging updates from master into this branch fixed CI.

@blag blag added this to the 3.2.0 milestone Oct 18, 2019
@arm4b arm4b requested a review from Kami October 18, 2019 18:17
@arm4b
Copy link
Member

arm4b commented Oct 18, 2019

Thanks @blag for the PR rescue! 👍

@Nicodemos305 Please add a Changelog for this fix to make sure this PR is 💯 ready for final review and merge.

@nicholasamorim
Copy link
Contributor

nicholasamorim commented Oct 18, 2019

@armab I'm not the PR author :D I've reported a similar issue, though, #4802. I guess this PR closes that too.

@arm4b
Copy link
Member

arm4b commented Oct 18, 2019

@nicholasamorim Fixed, sorry. Confused by Github autocomplete for the first 3 characters in @

@Nicodemos305
Copy link
Contributor

@armab Done!
👍

@arm4b arm4b requested a review from m4dcoder October 18, 2019 22:02
result = copy.deepcopy(value)
if('action' not in result):
return result
action_db = self._get_referenced_models(rule=result)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this change looks good to me, my only concern is performance impact of this change.

This change means we now need to perform 1 additional query for each Rule DB object.

This is not an issue for "get one" operation (only one additional query), but when retrieving multiple rules aka "get all" API operations, this would result in N additional queries where N is number of rules on the page.

return action_db

def _get_entity(self, model_persistence, ref, query_args):
q = Q(**query_args(ref))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like we really only need parameters field from the action model, so we should update this query to only retrieve that field by utilizing only_fields argument / query.only(**fields) function.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On a related note - is there a specific reason why you used Q object directly?

To not break the abstraction, we should probably utilize Action.query() method instead.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a heads up. I will look into making the changes mentioned above^.

This way we can hopefully get this across the finish line and merge it soon.

@Kami Kami self-assigned this Oct 29, 2019
@Kami
Copy link
Member

Kami commented Oct 29, 2019

I had a look at it and this needed a bit more work, so I tried to finish it.

I made the following changes:

  1. Used existing DB abstraction to retrieve the corresponding DB model and only retrieved the fields we need - bf421a3.

  2. Made sure those two API endpoints also support ?show_secrets=true query parameters for RBAC admins - 70f960b, 0343096. All the API endpoints which support secrets masking also need to support that query parameter.

  3. Added some test cases which verify secrets masking works correctly - 3400a76.

Unless I missed something, this is now ready to be merged.


My original comment with regards to performance impact / overhead still stands, but that's probably fine for now.

In the future, we can implement some kind of caching / similar to avoid this action retrieval overhead on get / list all API operation.

@Kami Kami closed this in #4807 Oct 30, 2019
Kami added a commit that referenced this pull request Oct 30, 2019
…ters

/v1/rules API endpoint action parameters secrets masking (additional changes on top of #4788)
@Kami
Copy link
Member

Kami commented Oct 30, 2019

Merged into master via #4807.

Thanks again for the contribution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
service: api size/M PR that changes 30-99 lines. Good size to review.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Secret action field show as plaint text in rule view
7 participants