Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: extend static analysis and compute confidence scores for deploy commands #673

Merged
merged 14 commits into from
Apr 2, 2024

Conversation

behnazh-w
Copy link
Member

@behnazh-w behnazh-w commented Mar 20, 2024

This PR extends and makes changes to the static analysis of CI configurations in Macaron with the high-level goal of finding build and deploy commands more accurately. To achieve that, some of the abstractions had to be replaced to allow writing customized analyses, such as detecting build language setup in a GitHub Actions workflow or detecting reachable secrets. The list of extensions and changes are as follows:

  • Replaces BashCommands which consisted of build tool commands collected after analyzing GitHub Actions and passed to checks with BuildToolCommand. With this PR BuildToolCommand are created by each build tool instead, and can be detected by calling a build tool API as part of the check. This delegation of analysis to each build tool allows to customize them per build tool. Moreover, now each build tool has direct access to the parsed AST of GitHub Actions and bash scripts, which opens opportunities to add new analyses more easily.
  • The build related checks are refactored and simplified to use BuildToolCommand.
  • The callgraph analysis, which needs to be implemented for each CI service, is extended with new node types for GitHub Actions. The callgraph plays the role of Intermediate Representation and is available to all checks.
  • The mcn_build_script_1 check does not depend on any checks and always runs by default based on a customer request.
  • The mcn_build_as_code_1 check now reports deploy commands with confidence scores. The confidence scores are computed based on additional facts collected from GitHub Actions, such as the CI event, reachable secrets, and name of the workflows.
  • New analysis is added to resolve the value of expression variables, which is used for other analysis, such as reachable secrets and build language detection.
  • New abstractions are added to model third-party GitHub Actions. This feature is used to collect data about build Language setup.
  • Fixes this parsing issue: Error when parsing the bash script in reactor/reactor-core's GitHub Action workflow #107
  • Changes the check result log level to debug mode.

@oracle-contributor-agreement oracle-contributor-agreement bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Mar 20, 2024
@behnazh-w behnazh-w marked this pull request as ready for review March 20, 2024 01:22
@behnazh-w behnazh-w requested a review from tromai as a code owner March 20, 2024 01:22
src/macaron/config/defaults.ini Show resolved Hide resolved
src/macaron/config/defaults.ini Outdated Show resolved Hide resolved
src/macaron/config/defaults.ini Outdated Show resolved Hide resolved
src/macaron/parsers/bashparser.py Show resolved Hide resolved
src/macaron/errors.py Show resolved Hide resolved
behnazh-w added a commit that referenced this pull request Mar 22, 2024
This PR temporarily excludes mcn_provenance_available_1 check for micronaut-core integration test because provenances have failed to publish due to an issue in generator_generic_slsa3.yml@v1.9.0.

It also excludes mcn_infer_artifact_pipeline_1, which is due to a non-deterministic behavior in deploy command detection, which will be fixed in PR #673.

Signed-off-by: behnazh-w <behnaz.hassanshahi@oracle.com>
Signed-off-by: behnazh-w <behnaz.hassanshahi@oracle.com>
Copy link
Member

@tromai tromai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I have put down all of my questions for now. Thanks for the changes!

@behnazh-w
Copy link
Member Author

behnazh-w commented Mar 26, 2024

This commit 071e99a adds node_id as an optional attribute of all callgraph nodes, makes the build tool detection deterministic by always sorting it based on the object string representation, and improves the store_inferred_provenance function.

Signed-off-by: behnazh-w <behnaz.hassanshahi@oracle.com>
Signed-off-by: behnazh-w <behnaz.hassanshahi@oracle.com>
Signed-off-by: behnazh-w <behnaz.hassanshahi@oracle.com>
@tromai
Copy link
Member

tromai commented Mar 27, 2024

This commit adds node_id as an optional attribute of all callgraph nodes, makes the build tool detection deterministic by always sorting it based on the object string representation, and improves the store_inferred_provenance function.

@behnazh-w I think the commit is not available anymore (I got 404 when I clicked on the link).

Signed-off-by: behnazh-w <behnaz.hassanshahi@oracle.com>
@behnazh-w
Copy link
Member Author

This commit adds node_id as an optional attribute of all callgraph nodes, makes the build tool detection deterministic by always sorting it based on the object string representation, and improves the store_inferred_provenance function.

@behnazh-w I think the commit is not available anymore (I got 404 when I clicked on the link).

Please check again.

@tromai
Copy link
Member

tromai commented Mar 27, 2024

I can access it again. Thanks @behnazh-w

Signed-off-by: behnazh-w <behnaz.hassanshahi@oracle.com>
Signed-off-by: behnazh-w <behnaz.hassanshahi@oracle.com>
Signed-off-by: behnazh-w <behnaz.hassanshahi@oracle.com>
Signed-off-by: behnazh-w <behnaz.hassanshahi@oracle.com>
Signed-off-by: behnazh-w <behnaz.hassanshahi@oracle.com>
Signed-off-by: behnazh-w <behnaz.hassanshahi@oracle.com>
Signed-off-by: behnazh-w <behnaz.hassanshahi@oracle.com>
Signed-off-by: behnazh-w <behnaz.hassanshahi@oracle.com>
Copy link
Member

@tromai tromai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for the changes.

@behnazh-w behnazh-w merged commit a3842b4 into staging Apr 2, 2024
20 checks passed
@behnazh-w behnazh-w deleted the improve-build-checks branch May 15, 2024 00:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
OCA Verified All contributors have signed the Oracle Contributor Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants