diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index b202fdf35..7ced8bf6d 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,6 +1,6 @@ # Contributing to Macaron -Oracle welcomes contributions to this repository from anyone. +We welcome contributions to this repository from anyone. If you want to submit a pull request to fix a bug or enhance an existing feature, please first open an issue and link to that issue when you submit your pull request. @@ -25,8 +25,8 @@ or `-s`, e.g. git commit --signoff ``` -Only pull requests from committers that can be verified as having signed the OCA -can be accepted. +Finally, make sure to sign your commits using a GPG key. See the instructions [here](https://docs.github.com/en/authentication/managing-commit-signature-verification/generating-a-new-gpg-key) for more information. A green `verified` label will appear next to your commit on GitHub if it is successfully signed. + ### Pull request process @@ -58,19 +58,136 @@ can be accepted. - Each new feature or change PR should provide meaningful CI tests. - All the tests and security scanning checks in CI should pass and achieve the expected coverage. +- To avoid unnecessary failures in GitHub Actions, make sure `make check` and `make test` work locally. +See below for instructions to set up the development environment. ### Merging PRs -- Before a PR is merged, all commits in the PR should be rebased into meaningful commits. +- Before a PR is merged, all commits in the PR should be meaningful. - Its commit message should be the same as the PR title if there only one commit. -- PRs should be merged using the fast-forward only strategy. - - This ensures a linear commit history for the project. - - If the fast-forward merge does not work due to new commits on `main` (in this case the two branches diverge and the PR branch cannot be fast-forwarded), the PR branch should be rebased onto `main` first. +- PRs should be merged using the `Squash and merge` strategy. In most cases a single commit with +a detailed commit message body is preferred. Make sure to keep the `Signed-off-by` line in the body. ## Branching model * The `main` branch is only used for releases and the `staging` branch is used for development. We only merge to `main` when we want to create a new release for Macaron. +## Setting up the development environment + +### Prerequisites + +- Python 3.11 +- Go 1.18 +- JDK 17 + +### Prepare the environment + +To contribute to Macaron, clone the project and create a [virtual environment](https://docs.python.org/3/tutorial/venv.html) by using the [Makefile](https://www.gnu.org/software/make/manual/make.html#toc-An-Introduction-to-Makefiles): + +```bash +make venv # Create a new virtual environment in .venv folder using Python 3.11. +``` + +or for a specific version of Python: + +```bash +PYTHON=python3.11 make venv # Same virtual environment for a different Python version. +``` + +Activate the virtual environment: + +```bash +. .venv/bin/activate +``` + +Finally, set up Macaron with all of its extras and initialize the local git hooks: + +```bash +make setup +``` + +**Note**: Running the above command will prompt you for sudo access to install [Soufflé Datalog engine](https://github.com/souffle-lang/souffle). You can install Soufflé on your system before running `make setup` to avoid getting prompted. + +With that in place, you’re ready to build and contribute to Macaron! + +### Updating dependent packages + +It’s likely that during development you’ll add or update dependent packages in the `pyproject.toml` or `go.mod` files, which requires an update to the environment: + +```bash +make upgrade +``` + +### Running Macaron as a Python package + +```bash +usage: macaron [-h] +``` + +### Obtaining the GitHub personal access token + +To obtain a GitHub access token, please see the official instructions [here](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token). + +Ideally, the GitHub token must have **read** permissions for the repositories that you want to analyze: + +- Every [fine-grained personal-access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token#creating-a-fine-grained-personal-access-token) should have read permission to public GitHub repositories. However, if you are analyzing a private repository, please select it in the ``Repository Access section``. +- For [classic personal-access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token#creating-a-personal-access-token-classic), the ``repo.public_repo`` scope must be selected. Please select the whole ``repo`` scope if you are running the analysis against private repositories. + +After generating a GitHub personal-access token, please store its value in an environment variable called ``GITHUB_TOKEN``. This environment variable will be read by Macaron for its **analyze** command. + +### Running checks and tests locally + +#### Git hooks + +Using the pre-commit tool and its `.pre-commit-config.yaml` configuration, a number of [pre-commit hooks](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks#_committing_workflow_hooks) ensure that your code is formatted correctly. + +You can also run these hooks manually, which comes in very handy during daily development tasks. For example + +```bash +make check-code +``` + +runs all the code checks (i.e. `bandit`, `flake8`, `pylint`, `mypy`, `actionlint`), whereas + +```bash +make check +``` + +runs _all_ installed git hooks over your code. For more control over the code checks, the Makefile also implements the `check-bandit`, `check-flake8`, `check-lint`, `check-mypy`, and `check-actionlint` goals. + + +#### Testing + +This repository is set up to test either standalone or as a pre-push git hook. Tests are stored in the `tests/` folder, and you can run them manually like so: +```bash +make test +``` +which runs all unit tests in both your local environment. Test code and branch coverage is already tracked using [coverage](https://github.com/nedbat/coveragepy) and the [pytest-cov](https://github.com/pytest-dev/pytest-cov) plugin for pytest, and it measures how much code in the `src/package/` folder is covered by tests. + +You can also run integration tests locally: +```bash +make integration-test +``` + +Note that integration tests can take a long time to complete. Also the repositories that we clone for these tests will be stored under `output/` directory. If you do not remove/move this directory and run the pre-commit tool you might get errors. + + +#### Generating documentation + +As mentioned above, all package code should make use of [Python docstrings](https://www.python.org/dev/peps/pep-0257/) in [reStructured text format](https://www.python.org/dev/peps/pep-0287/). Using these docstrings and the documentation template in the `docs/source/` folder, you can then generate proper documentation in different formats using the [Sphinx](https://github.com/sphinx-doc/sphinx/) tool: + +```bash +make docs +``` + +This example generates documentation in HTML, which can then be found here: + +```bash +open docs/_build/html/index.html +``` + +For more information see the instructions [here](docs/README.md). + ## Code of conduct Follow the [Golden Rule](https://en.wikipedia.org/wiki/Golden_Rule). If you'd like more specific guidelines, see the [Contributor Covenant Code of Conduct][COC]. diff --git a/README.md b/README.md index 2dc76276a..1ba1c4bfa 100644 --- a/README.md +++ b/README.md @@ -11,236 +11,28 @@ Macaron uses [SLSA requirements specifications v0.1](https://slsa.dev/spec/v0.1/ ## Table of Contents * [Getting started](#getting-started) -* [Running Macaron](#running-macaron) * [How to Contribute](#how-to-contribute) +* [Defining new checks](#defining-new-checks) * [Security issue reports](#security-issue-reports) * [License](#license) - ## Getting started -**Prerequisites** -- Python 3.11 -- Go 1.18 -- JDK 17 - -**Prepare the environment** - -Clone the project and install Macaron. - -```bash -make venv -. .venv/bin/activate -make setup -``` - -**Note**: Running the above command will prompt you for sudo access to install [Soufflé Datalog engine](https://github.com/souffle-lang/souffle). You can install Soufflé on your system before running `make setup` to avoid getting prompted. - -## Running Macaron - -```bash -usage: macaron [-h] [-v] [-o OUTPUT_DIR] {analyze,dump-defaults,verify-policy} ... -``` - -The main parameters for Macaron are: -- `[-v]`: Run Macaron with more debug outputs. -- `[-o OUTPUT_DIR]`: The directory to store the results. The default value will be the Macaron repo path. - -Apart from the main parameters listed above, you should choose the command to run, which requires other parameters. At the moment, Macaron has three commands. -- `analyze`: analyze the SLSA level of a single repository. -- `dump-defaults`: dump the default values in the output directory. -- `verify-policy`: verify a Datalog policy. - - -### Obtaining the GitHub personal access token -Create your own Github access token (please refer to the instructions [here](https://docs.github.com/en/github/authenticating-to-github/keeping-your-account-and-data-secure/creating-a-personal-access-token)). When creating this token, make sure to assign **at least** `repo` permissions. - -The GitHub token should be stored in an **environment variable** called `GITHUB_TOKEN`. Macaron will read the value of this Github token from the environment variable **only** if we use the `analyze` command (see instructions below). - -### Analyzing SLSA levels of a repository -*This section describes the `analyze` command of Macaron.* - -Run this command to determine the SLSA level of a repository: - -```bash -export GITHUB_TOKEN="" - -python -m macaron [-v] analyze -rp [-b ] [-d ] -c [-sbom ] -``` - -**Note**: for the rest of this section, we assume that the Github access token has been set as GITHUB_TOKEN environment variable. - -The main input parameters of the `analyze` command: -- `-rp ` specifies the path to the repository to analyze. This path can be both a local path (e.g. `/path/to/repo`) or a remote path (e.g. `git@github.com:organization/repo_name.git` or `https://github.com/organization/repo_name.git`). -- `[-b ]`: The name of the branch to analyze. If not specified, Macaron will checkout the default branch (if a remote path is supplied) or use the current branch that HEAD is on (if a local path is supplied). -- `[-d ]`: The hash of the commit to checkout in the current branch. This hash must be in full form. If not specified, Macaron will checkout the latest commit of the current branch. -- `-c `: The path to the configuration yaml file. This option cannot be used together with the `-rp` option. -- `[-sbom ]`: The path to the CyclondeDX SBOM of the target repo. - -Example I: Analyzing the GitHub repository [apache/maven](https://github.com/apache/maven) on **the latest commit of the default branch** without using a config file: - -```bash -python -m macaron -o output analyze -rp https://github.com/apache/maven.git -``` - -Example II: Doing the same thing as Example I, but using a config file: - -```bash -python -m macaron -o output analyze -c -``` - -```yaml -target: - id: "apache/maven" - branch: "" - digest: "" - path: "https://github.com/apache/maven.git" -``` - -The results of the examples above will be stored in ``output/reports/github_com/apache/maven/`` in HTML and JSON formats. - -**Notes:** -- Macaron automatically detects and analyzes direct dependencies for Java Maven projects. This process might take a while during the first run but will be faster during the subsequent runs. To skip analyzing the dependencies you can pass ``--skip-deps`` option. -- If you supply a remote path, the repository is cloned to `git_repos/` before the analysis starts. If the repository has already been cloned to `git_repos/`, Macaron will not clone the repository and proceed to analyze the local repo instead. -- The `branch` and `digest` in the config file or `-b` and `-d` in the CLI are all optional and can be omitted. -- If an SBOM is provided via `--sbom-path` option, Macaron will not detect the dependencies automatically. Note that `--skip-deps` option disables dependency analysis even if an SBOM file is provided. - -### Verifying a policy - -Macaron, currently, provides a PoC policy engine that checks a verified SLSA provenance against compliance requirements expressed as a policy. The result is reported in the JSON and HTML reports as a check called `mcn_provenance_expectation_1`. - -```bash -python -m macaron analyze -rp https://github.com/apache/maven.git -pe -``` - -The policy is a YAML file that contains expected values of predicates in SLSA provenance v0.2. Here is an example policy file: - -```yaml -metadata: - id: MACARON_1 - description: "Micronaut policy - SLSA provenance v0.2." - -definition: - _type: https://in-toto.io/Statement/v0.1 - predicateType: https://slsa.dev/provenance/v0.2 - - predicate: - builder: - id: https://github.com/slsa-framework/slsa-github-generator/.github/workflows/generator_generic_slsa3.yml@refs/tags/v1.2.1 - buildType: https://github.com/slsa-framework/slsa-github-generator/generic@v1 - - invocation: - configSource: - uri: git+https://github.com/micronaut-projects/micronaut-security@refs/tags/v3.8.3 - entryPoint: .github/workflows/release.yml -``` - -You can also run the policy verifier directly like below: - -```bash -python -m macaron verify-policy -d -f [-s] -``` - -**Note.** The policy engine is under active development and will support more complex policies soon. Stay tuned. +To learn how to download and run Macaron, see our documentation [here](https://oracle-samples.github.io/macaron/). ## How to Contribute -We welcome contributions! See our [general contribution guidelines](./CONTRIBUTING.md). - -To contribute to Macaron, first create a [virtual environment](https://docs.python.org/3/tutorial/venv.html) by either using the [Makefile](https://www.gnu.org/software/make/manual/make.html#toc-An-Introduction-to-Makefiles): - -```bash -make venv # Create a new virtual environment in .venv folder using Python 3.11. -``` - -or for a specific version of Python: - -```bash -PYTHON=python3.11 make venv # Same virtual environment for a different Python version. -``` - -Activate the virtual environment: - -```bash -. .venv/bin/activate -``` +We welcome contributions! See our [contribution guidelines](./CONTRIBUTING.md). -Finally, set up Macaron with all of its extras and initialize the local git hooks: - -```bash -make setup -``` - -With that in place, you’re ready to build and contribute to Macaron! - -### Defining checks +## Defining new checks After cloning a repository, Macaron parses the CI configuration files and bash scripts that are triggered by the CI, creates call graphs and other intermediate representations as abstractions. Using such abstractions, Macaron implements concrete checks to gather facts and metadata based on a security specification. To learn how to define your own checks, see the steps in the [checks documentation](/src/macaron/slsa_analyzer/checks/README.md). -### Updating dependent packages - -It’s likely that during development you’ll add or update dependent packages in the `pyproject.toml` file, which requires an update to the virtual environment: - -```bash -make upgrade -``` - -### Git hooks - -Using the pre-commit tool and its `.pre-commit-config.yaml` configuration, the following git hooks are active in this repository: - -- When committing code, a number of [pre-commit hooks](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks#_committing_workflow_hooks) ensure that your code is formatted according to [PEP 8](https://www.python.org/dev/peps/pep-0008/) using the [`black`](https://github.com/psf/black) tool, and they’ll invoke [`flake8`](https://github.com/PyCQA/flake8) (and various plugins), [`pylint`](https://github.com/PyCQA/pylint) and [`mypy`](https://github.com/python/mypy) to check for lint and correct types. There are more checks, but those two are the important ones. You can adjust the settings for these tools in the `pyproject.toml` or `.flake8` configuration files. -- The [commit message hook](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks#_committing_workflow_hooks) enforces [conventional commit messages](https://www.conventionalcommits.org/) and that, in turn, enables a _semantic release_ of this package on the Github side: upon merging changes into the `main` branch, the [release action](https://github.com/github.com/oracle-samples/blob/main/.github/workflows/release.yaml) uses the [Commitizen tool](https://commitizen-tools.github.io/commitizen/) to produce a [changelog](https://en.wikipedia.org/wiki/Changelog) and it computes the next version of this package and publishes a release — all based on the commit messages of a release. -- Using a [pre-push hook](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks#_other_client_hooks) this package is also set up to run [`pytest`](https://github.com/pytest-dev/pytest); in addition, the [`coverage`](https://github.com/nedbat/coveragepy) plugin makes sure that _all_ of your package’s code is covered by tests and [Hypothesis](https://hypothesis.works/) is already installed to help with generating test payloads. - -You can also run these hooks manually, which comes in very handy during daily development tasks. For example - -```bash -make check-code -``` - -runs all the code checks (i.e. `bandit`, `flake8`, `pylint` and `mypy`), whereas - -```bash -make check -``` - -runs _all_ installed git hooks over your code. For more control over the code checks, the Makefile also implements the `check-bandit`, `check-flake8`, `check-lint`, `check-mypy`, and `check-go` goals. - -### Testing - -As mentioned above, this repository is set up to use [pytest](https://pytest.org/) either standalone or as a pre-push git hook. Tests are stored in the `tests/` folder, and you can run them manually like so: -```bash -make test -``` - -which runs all tests in both your local Python virtual environment. For more options, see the [pytest command-line flags](https://docs.pytest.org/en/6.2.x/reference.html#command-line-flags). Also note that pytest includes [doctest](https://docs.python.org/3/library/doctest.html), which means that module and function [docstrings](https://www.python.org/dev/peps/pep-0257/#what-is-a-docstring) may contain test code that executes as part of the unit tests. - -Test code coverage is already tracked using [coverage](https://github.com/nedbat/coveragepy) and the [pytest-cov](https://github.com/pytest-dev/pytest-cov) plugin for pytest, and it measures how much code in the `src/macaron/` folder is covered by tests. - -Hypothesis is a package that implements [property based testing](https://en.wikipedia.org/wiki/QuickCheck) and that provides payload generation for your tests based on strategy descriptions ([more](https://hypothesis.works/#what-is-hypothesis)). Using its [pytest plugin](https://hypothesis.readthedocs.io/en/latest/details.html#the-hypothesis-pytest-plugin) Hypothesis is ready to be used for this package. - -To run integration tests run: - -```bash -make integration-test -``` - -### Generating documentation - -As mentioned above, all package code should make use of [Python docstrings](https://www.python.org/dev/peps/pep-0257/) in [reStructured text format](https://www.python.org/dev/peps/pep-0287/). Using these docstrings and the documentation template in the `docs/source/` folder, you can then generate proper documentation in different formats using the [Sphinx](https://github.com/sphinx-doc/sphinx/) tool: - -```bash -make docs -``` - -This example generates documentation in HTML, which can then be found here: +## Generating SLSA provenances for Macaron itself -```bash -open docs/_build/html/index.html -``` +We have integrated [SLSA provenance generation](https://github.com/slsa-framework/slsa-github-generator) for our Docker image and release artifacts. However, due to a strict policy regarding the use of third-party GitHub Actions, we cannot generate the provenances in this repository yet until [this issue](https://github.com/slsa-framework/slsa-github-generator/issues/2204) is resolved. ## Security issue reports diff --git a/docs/README.md b/docs/README.md index 155659d8f..00b05388d 100644 --- a/docs/README.md +++ b/docs/README.md @@ -4,7 +4,7 @@ This directory contains the source for the documentation of Macaron hosted [here ## Build the documentation To build the documentation, please follow these steps: -1. Setup the dev environment for Macaron using the instructions [here](../README.md#getting-started). +1. Setup the dev environment for Macaron using the instructions [here](../CONTRIBUTING.md). 2. Build the documentation by running this command from the root directory of Macaron: ``` make docs