ci: Enhanced code checking

[hpohekar]

closes #1673

Integrated vulnerability check in pre-commit hooks which will run locally.

There is already work in progress for the deprecated command message, we will use it once it is ready.

Case 1 - check passed.

$ git commit -m "ci: exit handling"
black................................................(no files to check)Skipped
isort................................................(no files to check)Skipped
flake8...............................................(no files to check)Skipped
codespell................................................................Passed
check yaml...............................................................Passed
pylint...............................................(no files to check)Skipped
Add License Headers..................................(no files to check)Skipped
Check Vulnerabilities Dependencies.......................................Passed
Check Vulnerabilities....................................................Passed
- hook id: check-vulnerabilities
- duration: 20.14s

+===========================================================================================================================================================================================+


DEPRECATED: this command (`check`) has been DEPRECATED, and will be unsupported beyond 01 June 2024.


We highly encourage switching to the new `scan` command which is easier to use, more powerful, and can be set up to mimic the deprecated command if required.


+===========================================================================================================================================================================================+




+===========================================================================================================================================================================================+


DEPRECATED: this command (`check`) has been DEPRECATED, and will be unsupported beyond 01 June 2024.


We highly encourage switching to the new `scan` command which is easier to use, more powerful, and can be set up to mimic the deprecated command if required.


+===========================================================================================================================================================================================+


[main]  INFO    profile include tests: None
[main]  INFO    profile exclude tests: B112,B403,B404,B110,B607,B314,B604,B311,B603,B307,B310,B405,B105,B602
[main]  INFO    cli include tests: None
[main]  INFO    cli exclude tests: None
Safety check performed.
Working... ---------------------------------------- 100% 0:00:10
[json]  INFO    JSON output written to file: info_bandit.json
Bandit check performed.
Advisory files generated successfully.
Dry run... not creating advisories and issues.
Information will be presented on screen.


*******************************************
Total 'safety' advisories detected: 0
Total 'safety' advisories reported: 0
Total 'bandit' advisories detected: 0
Total 'bandit' advisories reported: 0
*******************************************
Total advisories detected: 0
Total advisories reported: 0
*******************************************

[ci/enhanced_code_checking 7cd7c6ca3b] ci: exit handling
 1 file changed, 1 insertion(+), 1 deletion(-)

Case 2 - check failed.

$ pre-commit run --all-files
black....................................................................Passed
isort....................................................................Passed
flake8...................................................................Passed
codespell................................................................Passed
check yaml...............................................................Passed
pylint...................................................................Passed
Add License Headers......................................................Passed
Check Vulnerabilities Dependencies.......................................Passed
Check Vulnerabilities....................................................Failed
- hook id: check-vulnerabilities
- duration: 19.95s
- exit code: 1

+===========================================================================================================================================================================================+


DEPRECATED: this command (`check`) has been DEPRECATED, and will be unsupported beyond 01 June 2024.


We highly encourage switching to the new `scan` command which is easier to use, more powerful, and can be set up to mimic the deprecated command if required.


+===========================================================================================================================================================================================+




+===========================================================================================================================================================================================+


DEPRECATED: this command (`check`) has been DEPRECATED, and will be unsupported beyond 01 June 2024.


We highly encourage switching to the new `scan` command which is easier to use, more powerful, and can be set up to mimic the deprecated command if required.


+===========================================================================================================================================================================================+


[main]  INFO    profile include tests: None
[main]  INFO    profile exclude tests: B310,B604,B405,B602,B404,B110,B307,B112,B311,B403,B603,B314,B105
[main]  INFO    cli include tests: None
[main]  INFO    cli exclude tests: None
Safety check performed.
Working... ---------------------------------------- 100% 0:00:10
[json]  INFO    JSON output written to file: info_bandit.json
Bandit check performed.
Advisory files generated successfully.
Dry run... not creating advisories and issues.
Information will be presented on screen.

===========================================

Bandit [B607:start_process_with_partial_path] on ./src\ansys\fluent\core\fluent_connection.py - Hash: 03762c17822813383a8d90c4ece2ae0f

Starting a process with a partial executable path

#### Code

On file ./src\ansys\fluent\core\fluent_connection.py:

512     def _close_slurm(self):
513         subprocess.run(["scancel", f"{self._slurm_job_id}"])
514

#### CWE - 78
.
.
.
.

#### CWE - 78

For more information see https://cwe.mitre.org/data/definitions/78.html

#### More information

Visit https://bandit.readthedocs.io/en/1.8.2/plugins/b607_start_process_with_partial_path.html to find out more information.


*******************************************
Total 'safety' advisories detected: 0
Total 'safety' advisories reported: 0
Total 'bandit' advisories detected: 7
Total 'bandit' advisories reported: 7
*******************************************
Total advisories detected: 7
Total advisories reported: 7
*******************************************

[RobPasMue]

I would encourage your team to give it a local pass to the tool before enabling it on CI/CD. I'm sure Bandit will detect many issues and it will populate your advisories with a lot of draft ones... See https://actions.docs.ansys.com/version/stable/vulnerability-actions/index.html#check-vulnerabilities-action on how to run the tool locally

[hpohekar]

I would encourage your team to give it a local pass to the tool before enabling it on CI/CD. I'm sure Bandit will detect many issues and it will populate your advisories with a lot of draft ones... See https://actions.docs.ansys.com/version/stable/vulnerability-actions/index.html#check-vulnerabilities-action on how to run the tool locally

@RobPasMue Sure. Thanks a lot.

[RobPasMue]

Blocking PR until local resolutions have been performed by @hpohekar to avoid creation of unnecessary advisories

[seanpearsonuk]

DEPENDENCY_CHECK_TOKEN = your GitHub PAT.

Any need to be concerned about this?

[hpohekar]

DEPENDENCY_CHECK_TOKEN = your GitHub PAT.

Any need to be concerned about this?

@seanpearsonuk No. I have already discussed this with @RobPasMue.

[seanpearsonuk]

DEPENDENCY_CHECK_TOKEN = your GitHub PAT.
Any need to be concerned about this?

@seanpearsonuk No. I have already discussed this with @RobPasMue.

OK

[mkundu1]

DEPENDENCY_CHECK_TOKEN = your GitHub PAT.
Any need to be concerned about this?

@seanpearsonuk No. I have already discussed this with @RobPasMue.

OK

@hpohekar In check_vulnerabilities.py, the github integration seems to be used only for creatings issues or security advisories in the github repo. Maybe we can skip those for local run and remove or avoid the github integration?

A separate concern is to maintain a copy of a file from another repo in our repo, but we can address it later (may require some change in the original file in the other repo).

[hpohekar]

DEPENDENCY_CHECK_TOKEN = your GitHub PAT.
Any need to be concerned about this?

@seanpearsonuk No. I have already discussed this with @RobPasMue.

OK

@hpohekar In check_vulnerabilities.py, the github integration seems to be used only for creatings issues or security advisories in the github repo. Maybe we can skip those for local run and remove or avoid the github integration?

A separate concern is to maintain a copy of a file from another repo in our repo, but we can address it later (may require some change in the original file in the other repo).

@mkundu1 Done. Thank you.

[RobPasMue]

@mkundu1 @seanpearsonuk @hpohekar - as I posted here #3731 (comment), it is important that these checks are not only performed on pre-commit hooks but also on GitHub actions since they will be the ones reporting the vulnerabilities (if any) to our internal dashboard and on GitHub (privately, of course - users won't see them). Please address this request whenever you have the chance

[hpohekar]

@mkundu1 @seanpearsonuk @hpohekar - as I posted here #3731 (comment), it is important that these checks are not only performed on pre-commit hooks but also on GitHub actions since they will be the ones reporting the vulnerabilities (if any) to our internal dashboard and on GitHub (privately, of course - users won't see them). Please address this request whenever you have the chance

@RobPasMue @seanpearsonuk @mkundu1

We have removed this from local pre-commit hook in this PR. It was giving warnings for common Python packages like 'pillow' 'zipp' 'virtualenv' 'urllib3' etc.