Skip to content

feat: add python as a supported build tool #67

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Apr 4, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

Macaron is a supply chain security analysis tool from [Oracle Labs](https://labs.oracle.com/pls/apex/r/labs/labs/intro), which focuses on the build integrity of an artifact and the artifact dependencies. It is based on the [Supply chain Levels for Software Artifacts (SLSA)](https://slsa.dev/) specification, which aims at preventing some of the software supply chain attacks as the systems get more complex, especially with respect to the use of open-source third-party code in applications. Attacks include stealing credentials, injecting malicious code etc., and it is critical to have security assurance on the third-party code to guarantee that the integrity of the code has not been compromised.

Macaron uses [SLSA requirements specifications v0.1](https://slsa.dev/spec/v0.1/requirements) to define concrete rules for protecting software integrity that can be checked for compliance requirements automatically. Macaron provides a customizable checker platform that makes it easy to define checks that depend on each other. This is particularly useful for implementing checks for SLSA levels. In addition, Macaron also checks a user-specified policy for the repository to detect unexpected behavior in the build process. Macaron is a work-in-progress project and currently supports Maven and Gradle Java build systems only. We plan to support build systems for other languages, such as Python in future.
Macaron uses [SLSA requirements specifications v0.1](https://slsa.dev/spec/v0.1/requirements) to define concrete rules for protecting software integrity that can be checked for compliance requirements automatically. Macaron provides a customizable checker platform that makes it easy to define checks that depend on each other. This is particularly useful for implementing checks for SLSA levels. In addition, Macaron also checks a user-specified policy for the repository to detect unexpected behavior in the build process. Macaron is a work-in-progress project and currently supports Maven and Gradle Java build systems. Support has also been added for Python projects that use Pip or Poetry as their package managers, minus dependency analysis. We plan to support build systems for other languages in future.

## Table of Contents

Expand Down
53 changes: 53 additions & 0 deletions src/macaron/config/defaults.ini
Original file line number Diff line number Diff line change
Expand Up @@ -211,6 +211,59 @@ jenkins =
gradle-git-publish
gitPublishPush

# This is the spec for trusted Pip packaging tools.
[builder.pip]
entry_conf =
build_configs =
setup.py
setup.cfg
pyproject.toml
packager =
pip
pip3
flit
conda
publisher =
twine
flit
conda
# These are the Python interpreters that may be used to load modules.
interpreter =
python
python3
interpreter_flag =
-m
build_arg =
install
build
setup.py
deploy_arg =
publish
upload
[builder.pip.ci.deploy]
github_actions = pypa/gh-action-pypi-publish

# This is the spec for trusted Poetry packaging tools.
[builder.poetry]
entry_conf =
build_configs = pyproject.toml
package_lock = poetry.lock
builder =
poetry
poetry-core
# These are the Python interpreters that may be used to load modules.
interpreter =
python
python3
interpreter_flag =
-m
build_arg =
build
deploy_arg =
publish
[builder.poetry.ci.deploy]
github_actions = pypa/gh-action-pypi-publish

# This is the spec for GitHub Actions CI.
[ci.github_actions]
entry_conf =
Expand Down
6 changes: 4 additions & 2 deletions src/macaron/slsa_analyzer/build_tool/__init__.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
# Copyright (c) 2022 - 2022, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2022 - 2023, Oracle and/or its affiliates. All rights reserved.
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/.

"""The build_tool package contains the supported build tools for Macaron."""

from .base_build_tool import BaseBuildTool
from .gradle import Gradle
from .maven import Maven
from .pip import Pip
from .poetry import Poetry

# The list of supported build tools. The order of the list determine the order
# in which each build tool is checked against the target repository.
BUILD_TOOLS: list[BaseBuildTool] = [Gradle(), Maven()]
BUILD_TOOLS: list[BaseBuildTool] = [Gradle(), Maven(), Poetry(), Pip()]
5 changes: 5 additions & 0 deletions src/macaron/slsa_analyzer/build_tool/base_build_tool.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,12 @@ def __init__(self, name: str) -> None:
self.name = name
self.entry_conf: list[str] = []
self.build_configs: list[str] = []
self.package_lock: list[str] = []
self.builder: list[str] = []
self.packager: list[str] = []
self.publisher: list[str] = []
self.interpreter: list[str] = []
self.interpreter_flag: list[str] = []
self.build_arg: list[str] = []
self.deploy_arg: list[str] = []
self.ci_build_kws: dict[str, list[str]] = {
Expand Down
89 changes: 89 additions & 0 deletions src/macaron/slsa_analyzer/build_tool/pip.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# Copyright (c) 2023 - 2023, Oracle and/or its affiliates. All rights reserved.
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/.

"""This module contains the Pip class which inherits BaseBuildTool.

This module is used to work with repositories that use pip for dependency management.
"""

import logging

from macaron.config.defaults import defaults
from macaron.dependency_analyzer import DependencyAnalyzer, NoneDependencyAnalyzer
from macaron.slsa_analyzer.build_tool.base_build_tool import BaseBuildTool, file_exists

logger: logging.Logger = logging.getLogger(__name__)


class Pip(BaseBuildTool):
"""This class contains the information of the pip build tool."""

def __init__(self) -> None:
"""Initialize instance."""
super().__init__(name="pip")

def load_defaults(self) -> None:
"""Load the default values from defaults.ini."""
if "builder.pip" in defaults:
for item in defaults["builder.pip"]:
if hasattr(self, item):
setattr(self, item, defaults.get_list("builder.pip", item))

if "builder.pip.ci.deploy" in defaults:
for item in defaults["builder.pip.ci.deploy"]:
if item in self.ci_deploy_kws:
self.ci_deploy_kws[item] = defaults.get_list("builder.pip.ci.deploy", item)

def is_detected(self, repo_path: str) -> bool:
"""Return True if this build tool is used in the target repo.

Parameters
----------
repo_path : str
The path to the target repo.

Returns
-------
bool
True if this build tool is detected, else False.
"""
for file in self.build_configs:
if file_exists(repo_path, file):
return True
return False

def prepare_config_files(self, wrapper_path: str, build_dir: str) -> bool:
"""Prepare the necessary wrapper files for running the build.

This method returns False on errors. Pip doesn't require any preparation, therefore this method always
returns True.

Parameters
----------
wrapper_path : str
The path where all necessary wrapper files are located.
build_dir : str
The path of the build dir. This is where all files are copied to.

Returns
-------
bool
True if succeed else False.
"""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please add a comment that pip does not require any preparation?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 9c167f2.

return True

def get_dep_analyzer(self, repo_path: str) -> DependencyAnalyzer:
"""Create a DependencyAnalyzer for the build tool.

Parameters
----------
repo_path: str
The path to the target repo.

Returns
-------
DependencyAnalyzer
The DependencyAnalyzer object.
"""
# TODO: Implement this method.
return NoneDependencyAnalyzer()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please add a TODO to implement it later?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 9c167f2.

125 changes: 125 additions & 0 deletions src/macaron/slsa_analyzer/build_tool/poetry.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
# Copyright (c) 2023 - 2023, Oracle and/or its affiliates. All rights reserved.
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/.

"""This module contains the Poetry class which inherits BaseBuildTool.

This module is used to work with repositories that use Poetry for dependency management.
"""

import glob
import logging
import os
import tomllib
from pathlib import Path

from macaron.config.defaults import defaults
from macaron.dependency_analyzer import DependencyAnalyzer, NoneDependencyAnalyzer
from macaron.slsa_analyzer.build_tool.base_build_tool import BaseBuildTool, file_exists

logger: logging.Logger = logging.getLogger(__name__)


class Poetry(BaseBuildTool):
"""This class contains the information of the poetry build tool."""

def __init__(self) -> None:
"""Initialize instance."""
super().__init__(name="poetry")

def load_defaults(self) -> None:
"""Load the default values from defaults.ini."""
if "builder.poetry" in defaults:
for item in defaults["builder.poetry"]:
if hasattr(self, item):
setattr(self, item, defaults.get_list("builder.poetry", item))

if "builder.pip.ci.deploy" in defaults:
for item in defaults["builder.pip.ci.deploy"]:
if item in self.ci_deploy_kws:
self.ci_deploy_kws[item] = defaults.get_list("builder.pip.ci.deploy", item)

def is_detected(self, repo_path: str) -> bool:
"""Return True if this build tool is used in the target repo.

Parameters
----------
repo_path : str
The path to the target repo.

Returns
-------
bool
True if this build tool is detected, else False.
"""
package_lock_exists = ""
for file in self.package_lock:
if file_exists(repo_path, file):
package_lock_exists = file
break

for conf in self.build_configs:
# Find the paths of all pyproject.toml files.
pattern = os.path.join(repo_path, "**", conf)
files_detected = glob.glob(pattern, recursive=True)

if files_detected:
# If a package_lock file exists, and a config file is present, Poetry build tool is detected.
if package_lock_exists:
return True
# TODO: this implementation assumes one build type, so when multiple build types are supported, this
# needs to be updated.
# Take the highest level file, if there are two at the same level, take the first in the list.
file_path = min(files_detected, key=lambda x: len(Path(x).parts))
try:
# Parse the .toml file
with open(file_path, "rb") as toml_file:
try:
data = tomllib.load(toml_file)
# Check for the existence of a [tool.poetry] section.
if ("tool" in data) and ("poetry" in data["tool"]):
return True
except tomllib.TOMLDecodeError:
logger.error("Failed to read the %s file: invalid toml file.", conf)
return False
return False
except FileNotFoundError:
logger.error("Failed to read the %s file.", conf)
return False

return False

def prepare_config_files(self, wrapper_path: str, build_dir: str) -> bool:
"""Prepare the necessary wrapper files for running the build.

This method returns False on errors. Poetry doesn't require any preparation, therefore this method always
returns True.

Parameters
----------
wrapper_path : str
The path where all necessary wrapper files are located.
build_dir : str
The path of the build dir. This is where all files are copied to.

Returns
-------
bool
True if succeeds else False.
"""
return True

def get_dep_analyzer(self, repo_path: str) -> DependencyAnalyzer:
"""Create a DependencyAnalyzer for the build tool.

Parameters
----------
repo_path: str
The path to the target repo.

Returns
-------
DependencyAnalyzer
The DependencyAnalyzer object.
"""
# TODO: Implement this method.
return NoneDependencyAnalyzer()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as pip.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 9c167f2.

Loading