chore: add a new Makefile target simple-index which builds all distribution artifacts and then creates a PEP-503 compatible Simple Index in the dist/ folder

[jenstroeger]

Summary

Add a new Makefile target simple-index which builds all distribution artifacts and then creates a PEP-503 compatible Simple Index in the dist/ folder.

Description of changes

The Simple Index structure is defined in PEP 503 and this change works as follows:

~ > mkdir /tmp/macaron-index/
~ > cd /tmp/macaron-index/
/tmp/macaron-index > python -m venv .venv  # Create a new virtual environment and activate it.
/tmp/macaron-index > . .venv/bin/activate
/tmp/macaron-index > pip install --extra-index-url file:///path/to/macaron/dist/simple-index/ macaron  # Install the Macaron package.
Looking in indexes: https://pypi.org/simple, file:///path/to/macaron/dist/simple-index/
Processing /path/to/macaron/dist/simple-index/macaron/macaron-0.23.0.tar.gz
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
WARNING: Location '/path/to/macaron/dist/simple-index/requests/' is ignored: it is either a non-existing path or lacks a specific scheme.
Collecting requests<3.0.0,>=2.32.3 (from macaron)
  Using cached requests-2.33.1-py3-none-any.whl.metadata (4.8 kB)

We can ignore the warnings because the Simple Index contains only the Macaron package; all other packages fall back to the global PyPI default index.

Related issues

No related issue, but the changes are cherry-picked from PR #1355.

Checklist

• I have reviewed the contribution guide.
• My PR title and commits follow the Conventional Commits convention.
• My commits include the "Signed-off-by" line.
• I have signed my commits following the instructions provided by GitHub. Note that we run GitHub's commit verification tool to check the commit signatures. A green verified label should appear next to all of your commits on GitHub.
• I have updated the relevant documentation, if applicable.
• I have tested my changes and verified they work as expected.

[jenstroeger]

@behnazh-w following up on our conversation, here’s how to use the Simple Index to install Macaron in one go¹:

~ > mkdir macaron-index
~ > cd macaron-index/
macaron-index > python3.11 -m venv .venv
macaron-index > . .venv/bin/activate
macaron-index > pip install --no-deps --extra-index-url file:///path/to/macaron/dist/simple-index/ -r /path/to/macaron/dist/macaron-0.23.0-py3-none-macosx_x86_64-requirements.txt
...
Processing /path/to/macaron/dist/simple-index/macaron/macaron-0.23.0.tar.gz (from -r /path/to/macaron/dist/macaron-0.23.0-py3-none-macosx_x86_64-requirements.txt (line 2027))
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: macaron
  Building wheel for macaron (pyproject.toml) ... done
  Created wheel for macaron: filename=macaron-0.23.0-py3-none-any.whl size=585953 sha256=84a454bea1b7ce1268c0dec87c438661d8676151769723fd04e0b84336877236
  Stored in directory: /path/to/caches/pip/wheels/fa/e2/95/83a10db4b99104f7fd70877686c083ca53aaf06b5633a6acd5
Successfully built macaron
Installing collected packages: ...
Successfully installed ... macaron-0.23.0 ...

For example, the one line above can replace these two:

macaron/scripts/release_scripts/install_macaron_python.sh

Lines 116 to 119 in 3a4e8dd

	# Install Macaron package and dependencies.
	echo "Installing Macaron..."
	pip install --no-deps "${MACARON_WHEEL}"
	pip install --no-deps -r "${MACARON_REQUIREMENTS}"

if a Simple Index would be available locally.

Also, I noticed that building the Docker image does not use the pinned & hashed requirements but instead installs Macaron and then whatever deps pip decides at that particular point in time:

macaron/docker/Dockerfile.final

Lines 41 to 42 in 3a4e8dd

	&& pip install --no-compile --no-cache-dir --upgrade pip setuptools \
	&& find $HOME/dist -depth \( -type f \( -name "macaron-*.whl" \) \) -exec pip install --no-compile --no-cache-dir '{}' \; \

Is that intended? I had expected to see an installation using Macaron’s requirements? For example, this run and this run build a Docker image for Macaron 0.23.0 and differ in the cryptography, greenlet, and lxml package deps.²

Footnotes

1. Note that the generated requirements.txt contains the macaron pin and hashes, generated by the make requirements.txt. ↩

2. And then the types-pytz package was installed which — not through direct deps by Macaron — isn’t required for a production build? ↩

[behnazh-w]

That's cool, thanks!

[behnazh-w]

@behnazh-w following up on our conversation, here’s how to use the Simple Index to install Macaron in one go1:

~ > mkdir macaron-index
~ > cd macaron-index/
macaron-index > python3.11 -m venv .venv
macaron-index > . .venv/bin/activate
macaron-index > pip install --no-deps --extra-index-url file:///path/to/macaron/dist/simple-index/ -r /path/to/macaron/dist/macaron-0.23.0-py3-none-macosx_x86_64-requirements.txt
...
Processing /path/to/macaron/dist/simple-index/macaron/macaron-0.23.0.tar.gz (from -r /path/to/macaron/dist/macaron-0.23.0-py3-none-macosx_x86_64-requirements.txt (line 2027))
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: macaron
  Building wheel for macaron (pyproject.toml) ... done
  Created wheel for macaron: filename=macaron-0.23.0-py3-none-any.whl size=585953 sha256=84a454bea1b7ce1268c0dec87c438661d8676151769723fd04e0b84336877236
  Stored in directory: /path/to/caches/pip/wheels/fa/e2/95/83a10db4b99104f7fd70877686c083ca53aaf06b5633a6acd5
Successfully built macaron
Installing collected packages: ...
Successfully installed ... macaron-0.23.0 ...

For example, the one line above can replace these two:

macaron/scripts/release_scripts/install_macaron_python.sh

Lines 116 to 119 in 3a4e8dd

	# Install Macaron package and dependencies.
	echo "Installing Macaron..."
	pip install --no-deps "${MACARON_WHEEL}"
	pip install --no-deps -r "${MACARON_REQUIREMENTS}"

if a Simple Index would be available locally.

Also, I noticed that building the Docker image does not use the pinned & hashed requirements but instead installs Macaron and then whatever deps pip decides at that particular point in time:

macaron/docker/Dockerfile.final

Lines 41 to 42 in 3a4e8dd

	&& pip install --no-compile --no-cache-dir --upgrade pip setuptools \
	&& find $HOME/dist -depth \( -type f \( -name "macaron-*.whl" \) \) -exec pip install --no-compile --no-cache-dir '{}' \; \

Is that intended? I had expected to see an installation using Macaron’s requirements? For example, this run and this run build a Docker image for Macaron 0.23.0 and differ in the cryptography, greenlet, and lxml package deps.2

Footnotes

1. Note that the generated requirements.txt contains the macaron pin and hashes, generated by the `make requirements.txt`. [↩](#user-content-fnref-1-2894e3e4da611e9c5de8f26c929d3f23)

2. And then the `types-pytz` package was installed which — not through direct deps by Macaron — isn’t required for a production build? [↩](#user-content-fnref-2-2894e3e4da611e9c5de8f26c929d3f23)

@jenstroeger Looks like I missed this comment. Yes, I am aware of the behavior you have observed, and in fact it's been something that I've been wanting to improve, but didn't find time. Now that we have this new target, it's a good time to improve how we install Macaron in the Dockerfile and elsewhere.