fix(config/html): handle encoding issues and improve error handling in config and HTML file loading functions

[lshw54]

Context

This pull request addresses issues related to file encoding and error handling in configuration file loading functions (load_and_validate_config_file and load_and_validate_fixer_config_file) and the HTML output generation function (fill_html_overview_statistics). The changes ensure that files are read and written with the correct encoding, preventing UnicodeDecodeError.

Description

1. Encoding Handling:

• Updated load_and_validate_config_file, load_and_validate_fixer_config_file, and fill_html_overview_statistics to open files with encoding='utf-8' to ensure they are read and written correctly as UTF-8 encoded text.

• This change prevents UnicodeDecodeError caused by incorrect or unsupported file encodings.

2. Specific Exception Handling:

• Added specific exception handling for FileNotFoundError, yaml.YAMLError, and UnicodeDecodeError in the configuration file loading functions.
• Added specific exception handling for FileNotFoundError and UnicodeDecodeError in the HTML output generation function.
• This provides clearer error messages and makes debugging easier.

3. Default Values:

• Used default values in stats.get() in fill_html_overview_statistics to handle cases where keys might be missing.

License

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[pedrooot]

I think the sys.exit(1) should happen inside each exception right?

[pedrooot]

Same here

[pedrooot]

Thanks for the improvements! @lshw54 Could you please review my comments? 🚀

[lshw54]

Thanks for the improvements! @lshw54 Could you please review my comments? 🚀

Thanks!!I have made the changes to move sys.exit(1) inside each specific exception block in load_and_validate_config_file, load_and_validate_fixer_config_file, and fill_html_overview_statistics.

[pedrooot]

Could you have a look to the tests? It seems that flake8 it's failing. Using pre-commit should be enough https://docs.prowler.com/projects/prowler-open-source/en/latest/developer-guide/introduction/

[lshw54]

Could you have a look to the tests? It seems that flake8 it's failing. Using pre-commit should be enough https://docs.prowler.com/projects/prowler-open-source/en/latest/developer-guide/introduction/

All pre-commit run result is passed

[codecov]

Codecov Report

Attention: Patch coverage is 32.14286% with 19 lines in your changes missing coverage. Please review.

Project coverage is 86.61%. Comparing base (28b9e26) to head (3e70fd5).
Report is 6 commits behind head on master.

Files	Patch %	Lines
prowler/config/config.py	47.36%	10 Missing ⚠️
prowler/lib/outputs/html/html.py	0.00%	9 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4203      +/-   ##
==========================================
- Coverage   86.66%   86.61%   -0.06%     
==========================================
  Files         818      818              
  Lines       25674    25691      +17     
==========================================
+ Hits        22251    22252       +1     
- Misses       3423     3439      +16     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[lshw54]

Hi @pedrooot Thanks for your comment, it seems that I added some new comments and optimized a small portion of the code which resulted in a drop in coverage.

[pedrooot]

True, don't worry about that. Btw we are reviewing it now 😄

[sergargar]

Hi @lshw54, thanks for this fix!
Could you please change the log level to error and not exiting in those new errors? I don't think it could be a reason to exit Prowler execution since the scan can still continue.

[lshw54]

Could you please change the log level to error and not exiting in those new errors? I don't think it could be a reason to exit Prowler execution since the scan can still continue.

I have made the requested changes. The log level has been changed to error for the specific exceptions, and the program will no longer exit. This allows the scan to continue even if these errors occur. Errors are now logged at the error level, and the function allows the scan to proceed without exiting the Prowler.

[sergargar]

Could you please change the log level to error and not exiting in those new errors? I don't think it could be a reason to exit Prowler execution since the scan can still continue.

I have made the requested changes. The log level has been changed to error for the specific exceptions, and the program will no longer exit. This allows the scan to continue even if these errors occur. Errors are now logged at the error level, and the function allows the scan to proceed without exiting the Prowler.

Thanks for the changes @lshw54 ! I also change the error messages to follow our pattern.