Use of dynamic versioning with setuptools-scm causes MANIFEST.in to be ignored

[andy-maier]

When a pyproject.toml file is used with "build" as the build frontend and "setuptools" as the build backend, and when dynamic versioning is used with "setuptools-scm", then the specifications in MANIFEST.in are ignored. When using static versioning (i.e. not using "setuptools-scm" at all), the specifications in MANIFEST.in are obeyed.

Note that https://setuptools.pypa.io/en/latest/userguide/miscellaneous.html states that MANIFEST.in is used when present.

I am attaching a test_manifest.zip file with a stripped-down project that allows reproducing the issue:

test_manifest.zip

To reproduce:

• Download the test_manifest.zip file
• Create and activate a fresh Python virtualenv
• Create a new directory and cd into it
• Unzip the test_manifest.zip file into the current directory
• Prepare the development environment:

  pip install -r dev-requirements.txt
  

• Create a local git repo from the current directory, commit all the files and set a version tag:

  git init
  git add --all
  git commit -am "Initial commit"
  git tag 1.0.0
  

• Run the build script:

  ./build.sh
  

• Look at the created source archive in dist:

  tar -tvf dist/test_manifest*.tar.gz
  

It should contain the following files:

test_manifest-1.0.0/MANIFEST.in
test_manifest-1.0.0/PKG-INFO
test_manifest-1.0.0/build.sh
test_manifest-1.0.0/dev-requirements.txt
test_manifest-1.0.0/excluded/file1.py                 <- should not be included
test_manifest-1.0.0/excluded_file1.txt                <- should not be included
test_manifest-1.0.0/pyproject.toml
test_manifest-1.0.0/pyproject.toml.static_versioning  <- should not be included
test_manifest-1.0.0/requirements.txt
test_manifest-1.0.0/setup.cfg
test_manifest-1.0.0/test_manifest/__init__.py
test_manifest-1.0.0/test_manifest/_version_scm.py
test_manifest-1.0.0/test_manifest/main.py
test_manifest-1.0.0/test_manifest.egg-info/PKG-INFO
test_manifest-1.0.0/test_manifest.egg-info/SOURCES.txt
test_manifest-1.0.0/test_manifest.egg-info/dependency_links.txt
test_manifest-1.0.0/test_manifest.egg-info/requires.txt
test_manifest-1.0.0/test_manifest.egg-info/top_level.txt
test_manifest-1.0.0/test_manifest.egg-info/zip-safe

and the marked files should not have been included as per the specifications in MANIFEST.in, which are:

include pyproject.toml
include dev-requirements.txt
include MANIFEST.in
recursive-include test_manifest *

The pyproject.toml.static_versioning file is a version with static versioning, as a comparison. When renaming that to pyproject.toml and re-running the build.sh script, the generated source archive does not contain the marked files, i.e. it obeys the specifications in MANIFEST.in.

I used the following package versions on Python 3.12 on macOS:

setuptools                    75.8.2
setuptools-scm                8.2.0
wheel                         0.45.1

See also the discussion in https://discuss.python.org/t/can-i-exclude-files-in-sdist-when-using-build-frontend-with-setuptools-backend/82717/1

[RonnyPfannschmidt]

Setuptools always includes everything a file finder reports

So manifest must exclude such items

Setuptools lacks sane configuration for file finders do this had been a sore spot since more than a decade now

[andy-maier]

Ronny, you are my hero today! And thanks for responding on a sunday!

I thought I had also tested it with exclude directives, but it turns out I must have done something wrong.

I just tested it again, and with the following MANIFEST.in file, the marked files are no longer in the source archive when dynamic versioning is used:

include pyproject.toml
include dev-requirements.txt
include MANIFEST.in
recursive-include test_manifest *

# The following files should not be included in the source archive:
recursive-exclude excluded *
exclude excluded_file1.txt
exclude pyproject.toml.static_versioning

So the issue can be closed as far as I'm concerned.

The only little thing that could be done is to document the behavior that the default when MANIFEST.in is used changes between dynamic and static versioning: With static versioning, files are not included by default. With dynamic versioning based on setuptools-scm, the default changes to include all files by default. (At least according to what I found so far, maybe the situation is more complex than that).

[RonnyPfannschmidt]

I'll close this support ticket once I create a documentation issue