1.4.0 installs a top-level `tools` package into site-packages

[Ravi2k3]

Summary

Mako 1.4.0 installs a top-level tools package (site-packages/tools/ containing toxnox.py and warn_tox.py) alongside mako/. This appears unintentional: tools/ looks like an internal repo directory that got picked up by package discovery. 1.3.x does not install it.

Because tools is an extremely common name, this shadows project-local tools packages for anyone who installs Mako, including the very large number of projects that get it transitively via Alembic. It is especially disruptive for projects whose tools is a namespace package (no __init__.py), since a regular package always wins over a namespace package regardless of sys.path order, so the usual "put your project first on the path" workaround does not help.

Reproduction

docker run --rm python:3.12-slim bash -c '
  python -m venv /tmp/v
  /tmp/v/bin/pip install -q "Mako==1.4.0"
  ls /tmp/v/lib/python3.12/site-packages/tools/
'

Output:

__init__.py
toxnox.py
warn_tox.py

The files are recorded in mako-1.4.0.dist-info/RECORD under tools/.

With Mako==1.3.12 the same commands produce no site-packages/tools/.

Impact example

A project with its own tools/ namespace package started failing at import as soon as 1.4.0 was published, via alembic -> Mako:

ImportError: cannot import name 'cell_image_cleanup' from 'tools'
  (/…/site-packages/tools/__init__.py)
ModuleNotFoundError: No module named 'tools.agent_report'

No code or dependency declaration changed on our side; only the upstream resolve did.

Suggested fix

Exclude tools* from package discovery (or move those helpers outside the packaged tree) and release a 1.4.1. As a workaround, downstream projects can pin Mako<1.4.0.

Versions

• Mako 1.4.0 (installed by pip 25.0.1, Python 3.12.13, linux/amd64, python:3.12-slim)
• Mako 1.3.12: unaffected

[zzzeek]

fixing now will release 1.4.1

[zzzeek]

OK, not sure this is that novel of an issue. we have the same tools/ directory in SQLAlchemy and alembic. It seems to do the exact same thing in Alembic right now (and has for many versions). can you see if something is different about what Mako is doing?

still trying to figure out how mako is different

[sqla-tester]

Mike Bayer has proposed a fix for this issue in the main branch:

Don't install repository tools/ directory as a package https://gerrit.sqlalchemy.org/c/sqlalchemy/mako/+/6869

[zzzeek]

1.4.1 is released please confirm issue is resolved thanks

[Ravi2k3]

Confirmed resolved: on a clean python:3.12-slim, pip install Mako==1.4.1 no longer installs site-packages/tools/. Thanks for the fast turnaround.

On what was different from Alembic, from comparing the sdists:

1. The 1.3.10 sdist doesn't contain tools/ at all; 1.4.0's does.
2. Mako 1.4.0's tools/ gained an empty tools/__init__.py; Alembic's tools/ has no __init__.py.
3. Mako 1.4.0 uses [tool.setuptools.packages.find] with only test*/examples* excluded, and setuptools discovery only collects directories that are packages (i.e. have __init__.py). Alembic's tools/ is therefore never eligible, while Mako's newly qualified and got installed.

That's why Alembic has shipped the same directory for years with no effect: it exists in the sdist only, never the wheel (a clean venv with Alembic installed has no site-packages/tools). Deleting the stray tools/__init__.py alongside the exclude would make this class impossible to regress into.

[zzzeek]

yeah alembic has include = ["alembic*"] in it, mako didn't

anyway, mako is the last of my projects that didnt have a pyproject.toml install, so hopefully I never have to deal with this again