MANIFEST.in discrepant behavior between non-scikit-build and scikit-build projects

[kratsg]

It seems that fundamentally, scikit-build is treating the .* differently than how setuptools seems to treat it. Take for instance, this MANIFEST.in from pyhf:

prune *
graft src 
include LICENSE
exclude lower-bound-requirements.txt

global-exclude __pycache__ *.py[cod] .*

If I use something similar on a scikit-build project

prune *
graft src 
include LICENSE

global-exclude __pycache__ *.py[cod] .*

I find that none of the files are being picked up (because of this global-exclude at the end). However, if I tweak that last line instead to be

global-exclude __pycache__ *.py[cod]

then it works as expected. This is not so obvious at first glance, and it would be really nice to have something like #402 in.

[matthewfeickert]

@kratsg Seems that we don't actually need that in pyhf (I'm not sure why I first put that .* in) as if I remove it

--- a/MANIFEST.in
+++ b/MANIFEST.in
@@ -3,4 +3,4 @@ graft src
 include LICENSE
 exclude lower-bound-requirements.txt
 
-global-exclude __pycache__ *.py[cod] .*
+global-exclude __pycache__ *.py[cod]

and then check with check-manifest everything seems okay

# check-manifest .
lists of files in version control and sdist match

and the build using build works as before.

$ docker run --rm -ti python:3.8 /bin/bash
# python -m pip install --upgrade --quiet pip setuptools wheel
# python -m pip install --quiet build check-manifest
# pip list
Package        Version
-------------- -----------
build          0.3.1.post1
check-manifest 0.46
packaging      20.9
pep517         0.10.0
pip            21.1.1
pyparsing      2.4.7
setuptools     56.2.0
toml           0.10.2
wheel          0.36.2
# git clone https://github.com/scikit-hep/pyhf.git
# cd pyhf
# apt update -y && apt install -y vim
# vi MANIFEST.in 
# git diff
diff --git a/MANIFEST.in b/MANIFEST.in
index 1c9b062..e38e11c 100644
--- a/MANIFEST.in
+++ b/MANIFEST.in
@@ -3,4 +3,4 @@ graft src
 include LICENSE
 exclude lower-bound-requirements.txt
 
-global-exclude __pycache__ *.py[cod] .*
+global-exclude __pycache__ *.py[cod]
# check-manifest .
lists of files in version control and sdist match
# python -m build --sdist --wheel --outdir dist/ .
# ls -l dist/
total 252
-rw-r--r-- 1 root root 140527 May 12 03:47 pyhf-0.6.2.dev41-py3-none-any.whl
-rw-r--r-- 1 root root 110752 May 12 03:47 pyhf-0.6.2.dev41.tar.gz
# python -m pip install dist/pyhf-0.6.2.dev41-py3-none-any.whl
# pip list
Package        Version
-------------- -----------
attrs          21.2.0
build          0.3.1.post1
check-manifest 0.46
click          8.0.0
jsonpatch      1.32
jsonpointer    2.1
jsonschema     3.2.0
numpy          1.20.3
packaging      20.9
pep517         0.10.0
pip            21.1.1
pyhf           0.6.2.dev41
pyparsing      2.4.7
pyrsistent     0.17.3
PyYAML         5.4.1
scipy          1.6.3
setuptools     56.2.0
six            1.16.0
toml           0.10.2
tqdm           4.60.0
wheel          0.36.2

So I might as well go and remove that in pyhf as well.

[henryiii]

I think this might help: #402