Skip to content

Commit

Permalink
[CI] Fix releasing pipeline adapters (#5662)
Browse files Browse the repository at this point in the history
  • Loading branch information
liranbg committed May 30, 2024
1 parent e452539 commit 7d1bfc6
Show file tree
Hide file tree
Showing 5 changed files with 50 additions and 42 deletions.
41 changes: 41 additions & 0 deletions pipeline-adapters/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# How to build and release

The Makefile simplifies the process of building and releasing MLRun KFP Python packages to PyPI.

## Prerequisites
Before using this Makefile, ensure you have the following installed:

- Python
- `twine` package (`python -m pip install twine`)


## Getting Started

1. **Build Packages**: Run the following command to build your Python packages:
```bash
make build
```

This command will build each package listed in PACKAGES.

2. **Release Packages**: After building, you can release the packages to PyPI by running:
```bash
make release
```

This command will upload the built distribution files to PyPI using twine.

3. **Cleaning Up**: If needed, you can clean up the build artifacts by running:

```bash
make clean
```

This command will remove the dist, build, and *.egg-info directories from each package directory.


## Additional Notes

- Ensure your TWINE_USERNAME and TWINE_PASSWORD are properly set before releasing to PyPI.
- Make sure your package directories contain valid setup.py files.
- Always review the release process and verify that the correct packages are being released.
43 changes: 2 additions & 41 deletions pipeline-adapters/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,46 +3,7 @@
## What is MLRun Pipeline Adapters?

MLRun Pipeline Adapters is a collection of namespaced Python packages that allow MLRun integrating with KFP pipelines.
The idea behind it is leveraging Python package namespacing to provide a clean and organized way to manage the multiple KFP pipeline versions at once.

## How to build and release

The Makefile simplifies the process of building and releasing MLRun KFP Python packages to PyPI.

### Prerequisites
Before using this Makefile, ensure you have the following installed:

- Python
- `twine` package (`python -m pip install twine`)


### Getting Started

1. **Build Packages**: Run the following command to build your Python packages:
```bash
make build
```

This command will build each package listed in PACKAGES.
## Why do we need MLRun Pipeline Adapters?

2. **Release Packages**: After building, you can release the packages to PyPI by running:
```bash
make release
```

This command will upload the built distribution files to PyPI using twine.

3. **Cleaning Up**: If needed, you can clean up the build artifacts by running:

```bash
make clean
```

This command will remove the dist, build, and *.egg-info directories from each package directory.


### Additional Notes

- Ensure your TWINE_USERNAME and TWINE_PASSWORD are properly set before releasing to PyPI.
- Make sure your package directories contain valid setup.py files.
- Always review the release process and verify that the correct packages are being released.
The idea behind it is leveraging Python package namespacing to provide a clean and organized way to manage the multiple KFP pipeline versions at once.
4 changes: 3 additions & 1 deletion pipeline-adapters/mlrun-pipelines-kfp-common/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
setup(
name="mlrun-pipelines-kfp-common",
version="0.1.0",
description="MLRun Pipelines package for providing KFP 1.8 compatibility",
description="MLRun Pipelines package for providing KFP common functionality",
author="Yaron Haviv",
author_email="yaronh@iguazio.com",
license="Apache License 2.0",
Expand All @@ -39,4 +39,6 @@
"kfp",
],
python_requires=">=3.9, <3.12",
long_description="MLRun Pipelines package for providing KFP common functionality",
long_description_content_type="text/markdown",
)
2 changes: 2 additions & 0 deletions pipeline-adapters/mlrun-pipelines-kfp-v1-8/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,4 +42,6 @@
install_requires=[
"kfp~=1.8",
],
long_description="MLRun Pipelines package for providing KFP 1.8 compatibility",
long_description_content_type="text/markdown",
)
2 changes: 2 additions & 0 deletions pipeline-adapters/mlrun-pipelines-kfp-v2/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,4 +42,6 @@
install_requires=[
"kfp[kubernetes]~=2.5.0",
],
long_description="MLRun Pipelines package for providing KFP 2.* compatibility",
long_description_content_type="text/markdown",
)

0 comments on commit 7d1bfc6

Please sign in to comment.