Skip to content

Commit

Permalink
Add scripts for building packages and publishing them to PyPi (#1915)
Browse files Browse the repository at this point in the history
* add pypi_build and pypi_publish scripts

* update release instructions

---------

Co-authored-by: Adeel Hassan <ahassan@element84.com>
  • Loading branch information
AdeelH and AdeelH committed Sep 14, 2023
1 parent b334d68 commit a8b4bff
Show file tree
Hide file tree
Showing 3 changed files with 149 additions and 40 deletions.
55 changes: 15 additions & 40 deletions docs/release.rst
Original file line number Diff line number Diff line change
Expand Up @@ -66,69 +66,44 @@ Minor or Major Version Release
pip install twine
To store settings for PyPI you can setup a ``~/.pypirc`` file containing:
To store settings for PyPI you can set up a ``~/.pypirc`` file containing:

.. code-block:: console
[pypi]
username = azavea
Once packages are published they cannot be changed so be careful. (It's possible to practice using testpypi.) Navigate to the ``raster-vision`` repo on your local filesystem. With the version branch checked out, run something like the following to publish each plugin, and then the top-level package.

.. code-block:: console
export RV="/Users/lfishgold/projects/raster-vision"
.. code-block:: console
[testpypi]
username = azavea
cd $RV/rastervision_pipeline
python setup.py sdist bdist_wheel
twine upload dist/*
Once packages are published they cannot be changed, so be careful. (It's possible to practice using TestPyPI.) Navigate to the repo's root directory on your local filesystem. With the version branch checked out, run the following scripts to build packages and publish to PyPI.

Build:

.. code-block:: console
cd $RV/rastervision_aws_batch
python setup.py sdist bdist_wheel
twine upload dist/*
scripts/pypi_build
.. code-block:: console
cd $RV/rastervision_aws_s3
python setup.py sdist bdist_wheel
twine upload dist/*
Publish to TestPyPI. (You will be prompted for the PyPI password multiple times--once for each package.)

.. code-block:: console
cd $RV/rastervision_core
python setup.py sdist bdist_wheel
twine upload dist/*
.. code-block:: console
scripts/pypi_publish --test
cd $RV/rastervision_pytorch_learner
python setup.py sdist bdist_wheel
twine upload dist/*
You can then test it with ``pip`` like so:

.. code-block:: console
cd $RV/rastervision_pytorch_backend
python setup.py sdist bdist_wheel
twine upload dist/*
.. code-block:: console
pip install --index-url https://test.pypi.org/simple/ rastervision
cd $RV/rastervision_gdal_vsi
python setup.py sdist bdist_wheel
twine upload dist/*
Finally, if everything looks okay, publish to Pypi. (You will be prompted for the PyPI password multiple times--once for each package.)

.. code-block:: console
cd $RV
python setup.py sdist bdist_wheel
twine upload dist/*
scripts/pypi_publish
#. Announce new release in our `forum <https://github.com/azavea/raster-vision/discussions>`_, and with blog post if it's a big release.
#. Make a PR to the master branch that updates the version number to the next development version. For example, if the last release was ``0.20.1``, update the version to ``0.20.2-dev``.
#. Announce the new release in our `forum <https://github.com/azavea/raster-vision/discussions>`_, and with a blog post if it's a big release.
#. Make a PR to the master branch that updates the version number to the next development version, ``X.Y.Z-dev``. For example, if the last release was ``0.20.1``, update the version to ``0.20.2-dev``.

Bug Fix Release
-----------------
Expand Down
42 changes: 42 additions & 0 deletions scripts/pypi_build
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
#!/bin/bash

# Determine the script's directory (even if it's a symbolic link)
SOURCE="${BASH_SOURCE[0]}"
while [ -h "$SOURCE" ] ; do SOURCE="$(readlink "$SOURCE")"; done
SCRIPTS_DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
SRC_DIR="$( cd -P "$( dirname "$SCRIPTS_DIR" )" && pwd )"

# List of plugins to build
plugins=("rastervision_pipeline" "rastervision_aws_batch" "rastervision_aws_s3" "rastervision_core" "rastervision_pytorch_learner" "rastervision_pytorch_backend" "rastervision_gdal_vsi")

# Usage documentation
function usage() {
echo "Usage: $(basename "$0") [--test]"
echo ""
echo "Build Raster Vision plugins and top-level package."
}

# Check for command-line arguments
if [[ "$1" == "--help" || "$1" == "-h" ]]; then
usage
exit
fi

# Function to build a plugin
function build_plugin() {
cd "$SRC_DIR/$1"
echo "Building $1 ... "
python setup.py sdist bdist_wheel
echo "Done."
cd "$SRC_DIR"
}

# Build each plugin
for plugin in "${plugins[@]}"; do
build_plugin "$plugin"
done

# Build the top-level package
echo "rastervision ... "
python setup.py sdist bdist_wheel
echo "Done."
92 changes: 92 additions & 0 deletions scripts/pypi_publish
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
#!/bin/bash

# Determine the script's directory (even if it's a symbolic link)
SOURCE="${BASH_SOURCE[0]}"
while [ -h "$SOURCE" ]; do SOURCE="$(readlink "$SOURCE")"; done
SCRIPTS_DIR="$(cd -P "$(dirname "$SOURCE")" && pwd)"
SRC_DIR="$(cd -P "$(dirname "$SCRIPTS_DIR")" && pwd)"

# List of plugins to publish
plugins=("rastervision_pipeline" "rastervision_aws_batch" "rastervision_aws_s3" "rastervision_core" "rastervision_pytorch_learner" "rastervision_pytorch_backend" "rastervision_gdal_vsi")

# Usage documentation
function usage() {
echo "Usage: $(basename "$0") [--test]"
echo ""
echo "Publish Raster Vision plugins and top-level package to PyPI or TestPyPI."
echo ""
echo "Options:"
echo " -y Automatically answer 'yes' to prompts."
echo " --test Publish to TestPyPI instead of PyPI."
}

# Function to publish a package to the specified repository
function publish_package() {
if [ "$publish_to_test" = true ]; then
echo "Publishing to TestPyPI ... "
twine upload --repository testpypi dist/*
else
echo "Publishing to PyPI ... "
twine upload dist/*
fi
echo "Done."
}

# publish a plugin
function publish_plugin() {
local plugin_name="$1"
cd "$SRC_DIR/$plugin_name"
echo "Publishing $plugin_name ... "
publish_package
cd "$SRC_DIR"
}

# publish all plugins and the top-level package
function publish_all() {
# Publish each plugin
for plugin in "${plugins[@]}"; do
publish_plugin "$plugin"
done

# Publish the top-level package
echo "Publishing rastervision ... "
publish_package
}

# Check for command-line arguments
if [[ "$1" == "--help" || "$1" == "-h" ]]; then
usage
exit
fi

# Check if the --test flag is passed
publish_to_test=false
if [[ "$1" == "--test" ]]; then
publish_to_test=true
# Remove the --test flag from the arguments
shift
fi

# If testing: publish and exit
if [ "$publish_to_test" = true ]; then
publish_all
exit
fi

# If actually publishing: prompt for confirmation
if [[ "$1" == "-y" ]]; then
response="y"
else
read -r -p "Actually publish to PyPi? (y/N): " response
fi

case "$response" in
[yY][eE][sS]|[yY])
echo "Publishing to PyPi..."
publish_all
;;
*)
echo "Aborting."
;;
esac

0 comments on commit a8b4bff

Please sign in to comment.