Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CodeQL Filtering and GitHub Workflow Support [Rebase & FF] #692

Merged
merged 3 commits into from
Sep 7, 2023

Conversation

makubacki
Copy link
Member

@makubacki makubacki commented Sep 5, 2023

Description

CodeQL was previously enabled in the repo to the point that it could
be run locally with the --codeql flag. It was not enabled in CI
because the pre-existing CodeQL GitHub workflow did not support platform
builds.

This change hooks PlatformBuild.py into the newer stuart_codeql helper
functionality, adds proper filtering support, and a platform workflow that
allows CodeQL to run in this repo on every PR.

Running CodeQL at a "platform" level is advantageous because it can catch
similar CodeQL violations found when building physical platform code.

Note: codeql-platform.yml is directly checked into the repo here as it has
been tested and it is more clearly explained attached to this PR. In the
future, it will be synced from mu_devops.
There is some similarity with the pre-existing CodeQL CI workflow but those
are relatively simple tasks not expected to change much and may be converged
in the future but that is not a goal right now.

Note: CodeQL is only enabled for QemuQ35Pkg as the CodeQL extractor fails
on Linux for edk2 style builds and QemuSbsaPkg does not build on Windows/
Visual Studio at this time.


pip: bump edk2-pytool-extensions from 0.24.0 to 0.24.1

Includes the edk2toolext.codeql functions needed in upcoming
changes.


QemuQ35Pkg/PlatformBuild.py: Add CodeQL filtering support

Makes a number of adjustments in PlatformBuild.py as outlined below.
The main improvement is adding support to recursively gather CodeQL
filter files within the repo.

  1. Remove unused imports at the top of the file.
  2. Use the CodeQL functions newly added to edk2-pytool-extensions.
  3. Replace local functionality with common implementation in the
    codeql module.
  4. Remove trailing whitespace throughout the file.

Add CodeQL platform GitHub workflow

Adds a new GitHub workflow that allows CodeQL to run against platform
builds. Previously, only a "CI" CodeQL workflow existed that did not
support platform builders.

This file is being added directly to the repo as it is paired with
other changes that it has been tested alongside. In the future, it
will automatically be synced from mu_devops.

Nothing about the file is specific to mu_tiano_platforms or any
particular platform. It works by discovering all buildable platforms
in a repo before any dependencies are cloned and then verifying
the build files in the platform package directory support platform
build. If they do, it is checked if they support CodeQL. Only
platforms that meet all of these conditions are actually built via
a dynamic platform package matrix.

This allows the workflow to scale across platform repos and
automatically pick up new platforms as they onboard support for
CodeQL.


  • Impacts functionality?
    • Functionality - Does the change ultimately impact how firmware functions?
    • Examples: Add a new library, publish a new PPI, update an algorithm, ...
  • Impacts security?
    • Security - Does the change have a direct security impact on an application,
      flow, or firmware?
    • Examples: Crypto algorithm change, buffer overflow fix, parameter
      validation improvement, ...
  • Breaking change?
    • Breaking change - Will anyone consuming this change experience a break
      in build or boot behavior?
    • Examples: Add a new library class, move a module to a different repo, call
      a function in a new library class in a pre-existing module, ...
  • Includes tests?
    • Tests - Does the change include any explicit test code?
    • Examples: Unit tests, integration tests, robot tests, ...
  • Includes documentation?
    • Documentation - Does the change contain explicit documentation additions
      outside direct code modifications (and comments)?
    • Examples: Update readme file, add feature readme file, link to documentation
      on an a separate Web page, ...

How This Was Tested

  1. Ran the QemuQ35Pkg CodeQL build locally
  2. Ran the CodeQL - Platform GitHub workflow
    • Verified successful detection and build of QemuQ35Pkg

Integration Instructions

Moving forward, it is recommended to run CodeQL locally when making C source
code changes in QemuQ35Pkg. Also, CodeQL success will become a required
status check in mu_tiano_platforms CI for QemuQ35Pkg. See the following
CodeQL plugin documentation for more info.

@makubacki makubacki added type:enhancement New feature or pull request type:feature-request A new feature proposal labels Sep 5, 2023
@makubacki makubacki self-assigned this Sep 5, 2023
@github-actions github-actions bot added language:python Pull requests that update Python code impact:non-functional Does not have a functional impact labels Sep 5, 2023
@makubacki makubacki force-pushed the add_codeql_workflow_CI_support branch 4 times, most recently from 42208ae to 669d785 Compare September 6, 2023 19:55
@makubacki makubacki changed the title Add CodeQL Filtering and GitHub Workflow Support Add CodeQL Filtering and GitHub Workflow Support [Rebase & FF] Sep 6, 2023
@makubacki makubacki force-pushed the add_codeql_workflow_CI_support branch 2 times, most recently from 791c904 to 5387168 Compare September 6, 2023 23:08
Includes the `edk2toolext.codeql` functions needed in upcoming
changes.

Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Makes a number of adjustments in PlatformBuild.py as outlined below.
The main improvement is adding support to recursively gather CodeQL
filter files within the repo.

1. Remove unused imports at the top of the file.
2. Replace local functionality with common implementation in the
   `codeql_helpers` module.
3. Remove trailing whitespace throughout the file.

Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Adds a new GitHub workflow that allows CodeQL to run against platform
builds. Previously, only a "CI" CodeQL workflow existed that did not
support platform builders.

This file is being added directly to the repo as it is paired with
other changes that it has been tested alongside. In the future, it
will automatically be synced from mu_devops.

Nothing about the file is specific to mu_tiano_platforms or any
particular platform. It works by discovering all buildable platforms
in a repo before any dependencies are cloned and then verifying
the build files in the platform package directory support platform
build. If they do, it is checked if they support CodeQL. Only
platforms that meet all of these conditions are actually built via
a dynamic platform package matrix.

This allows the workflow to scale across platform repos and
automatically pick up new platforms as they onboard support for
CodeQL.

Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
@makubacki makubacki force-pushed the add_codeql_workflow_CI_support branch from 5387168 to e7fd774 Compare September 7, 2023 15:14
@github-advanced-security
Copy link

This pull request sets up GitHub code scanning for this repository. Once the scans have completed and the checks have passed, the analysis results for this pull request branch will appear on this overview. Once you merge this pull request, the 'Security' tab will show more code scanning analysis results (for example, for the default branch). Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results. For more information about GitHub code scanning, check out the documentation.

@makubacki makubacki enabled auto-merge (rebase) September 7, 2023 16:04
@makubacki makubacki merged commit 1731ed3 into microsoft:main Sep 7, 2023
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
impact:non-functional Does not have a functional impact language:python Pull requests that update Python code type:enhancement New feature or pull request type:feature-request A new feature proposal
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants