fix(custom): execute custom checks

[sejimhp]

Context

Custom rules were not available and I would like to to fix it.

It is caused by copying the custom checks folder after loading the checks.
so, added when files are copied

Description

copying the custom check in the parse_checks_from_folder function here
But check loading is called in load_checks_to_execute before that

There was no process to assign a custom check for that transition, so the custom check was not executed!

Added support for updating pre-loaded sets at the time of copying custom check folders.

Check

Here is an example of gcp.
Prepare the following directory structure rules.

custom_rule
└── cloudsql_test_custom_rule
    ├── __init__.py
    ├── cloudsql_test_custom_rule.metadata.json
    └── cloudsql_test_custom_rule.py

Execute the following command

without custome rule scan

prowler gcp --project-ids {project_id}

with custome rule scan

prowler gcp --project-ids {project_id} -x custom_rule

Confirmed that the number of scans performed and the number of compliant evaluations changes

License

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[pedrooot]

Hey! @sejimhp This looks great! Could you fix the tests in order to ensure that all it's correct? Thank you 💯

[sejimhp]

@pedrooot Thank you for review!
I fixed the test.

[sejimhp]

I had to add an additional push because it failed in the format.

[codecov]

Codecov Report

Attention: Patch coverage is 66.66667% with 2 lines in your changes missing coverage. Please review.

Project coverage is 86.67%. Comparing base (0a41ec4) to head (2d0ca66).
Report is 1 commits behind head on master.

Files	Patch %	Lines
prowler/__main__.py	0.00%	2 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #4202   +/-   ##
=======================================
  Coverage   86.66%   86.67%           
=======================================
  Files         818      818           
  Lines       25674    25675    +1     
=======================================
+ Hits        22251    22253    +2     
+ Misses       3423     3422    -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[sejimhp]

The coverage drop seems to be caused by main.py, but I couldn't find a test for it...

[sergargar]

Good catch @sejimhp, thank you so much for the fix!

[kagahd]

No, this is wrong and a regression! Including checks from the checks_folder does not necessarily mean that we want to execute all of them! You may want to execute only checks that are defined by parameter -c or --checks. See prowlers documentation. Your code prevents this behavior. It worked still in the previous prowler version 4.2.3 but it's broken now in the actual v4.2.4.

[jfagoagas]

This workaround fixed that #4256

[kagahd]

This workaround fixed that #4256

That's great!
PS: I clicked on "submit review" only the next day of my comment. That's why my comment only appeared now, after having fixed the issue already.

[jfagoagas]

No problem, thanks!