nested projects fail since version 4

[ashic]

If I have a project structure like this:

project\.git
project\foo\setup.py
project\foo\requirementx.txt
...

in other words, putup foo is run inside the git folder so the python project is one folder down, then python setup.py --version fails with:

LookupError: setuptools-scm was unable to detect version for '/path/to/foo'.

If I copy all the contents of foo/ to the root, it works.

With pyscaffold 3, this used to work out of the box. Is there a way to get it working with v4?

In setup.py, I tried changing the setup call to:

setup(use_scm_version={"version_scheme": "no-guess-dev", "root": ".."})

This isn't working either.

Versions and main components

• PyScaffold Version: 4
• Python Version: 3.7
• Operating system: Linux / Mac
• How did you install PyScaffold: (conda/pip) pip

[FlorianWilhelm]

CC @jochengietzen as he seems to have the same problem.

[FlorianWilhelm]

Thanks, @ashic for posting this issue. If it worked for you in PyScaffold 3.3 it's a regression. In version 4.0 was a greater change in the sense that there is no longer an indirection when calling setuptools-scm. In v3.3, PyScaffold itself was called (using setup_requires and some hooks) which then called setuptools-scm. Now in v4.0, setuptools-scm is in pyproject.toml and thus it's called directly. Maybe the creator of setuptools-scm, @RonnyPfannschmidt, could give us a pointer where the problem is.

[RonnyPfannschmidt]

when using the pyproject.toml variant, it seems critical to me to also move the root option over to the tool.setuptools_scm section

if there is a bad interaction between the setup.py keyword and the toml support, we need to crosscheck that as well

i cant debug myself atm, but im happy to work on a bugfix in case a bug shows up in the combination later today

[abravalheri]

@ashic as pointed out by Ronny, it seems that you might have to do the configuration of setuptools_scm in the pyproject.toml file.

If you want to stick with the legacy behaviour though, you might also want to try the --no-pyproject flag when generating new projects or deleting pyproject.toml from the existing ones.

[FlorianWilhelm]

@ashic, could you try adding root = ".." and relative_to = "setup.py" to your pyproject.toml so that it reads:

[build-system]
# AVOID CHANGING REQUIRES: IT WILL BE UPDATED BY PYSCAFFOLD!
requires = ["setuptools>=46.1.0", "setuptools_scm[toml]>=5", "wheel"]
build-backend = "setuptools.build_meta"

[tool.setuptools_scm]
# See configuration details in https://github.com/pypa/setuptools_scm
version_scheme = "no-guess-dev"
root = ".."
relative_to = "setup.py"

also in setup.py change the call of setup(..) to:

setup(use_scm_version={"root": "..", "relative_to": __file__, "version_scheme": "no-guess-dev"})

and check if this works? At least in my test it did.

Here is how I tested it:

mkdir parent
cd parent
git init
putup child
cd child
... edited the files as described above ...
python setup.py --version
tox -e build

I wonder actually how this could work in version 3 of PyScaffold. Maybe older versions of setuptools-scm had another way of resolving the root dir? @RonnyPfannschmidt. I don't understand why setuptools-scm needs to know root and also relative_to. The git command also somehow magically figures out where the .git root directory is using git rev-parse --show-toplevel. Couldn't this way be the default for setuptools-scm?

[ashic]

@ashic, could you try adding root = ".." and relative_to = "setup.py" to your pyproject.toml so that it reads:

[build-system]
# AVOID CHANGING REQUIRES: IT WILL BE UPDATED BY PYSCAFFOLD!
requires = ["setuptools>=46.1.0", "setuptools_scm[toml]>=5", "wheel"]
build-backend = "setuptools.build_meta"

[tool.setuptools_scm]
# See configuration details in https://github.com/pypa/setuptools_scm
version_scheme = "no-guess-dev"
root = ".."
relative_to = "setup.py"

also in setup.py change the call of setup(..) to:

setup(use_scm_version={"root": "..", "relative_to": __file__, "version_scheme": "no-guess-dev"})

and check if this works? At least in my test it did.

Here is how I tested it:

mkdir parent
cd parent
putup child
cd child
... edited the files as described above ...
python setup.py --version
tox -e build

Setting root to ".." in both setup.py and pyproject.toml is enabling pip install -e . to run successfully. Thanks.

[abravalheri]

@ashic, I will close this issue now, because it looks like it was solved, but please feel free to open it again if you still have problems.

[FlorianWilhelm]

@RonnyPfannschmidt: Just out of curiosity, could you comment on that? Thank you :-)

I don't understand why setuptools-scm needs to know root and also relative_to. The git command also somehow magically figures out where the .git root directory is using git rev-parse --show-toplevel. Couldn't this way be the default for setuptools-scm?

[RonnyPfannschmidt]

@FlorianWilhelm you can say think you to redistributors and packages, as well as people that put home under git

basically people do all kinds of bad stuff with surrounding git repos and the added items are to inform the tooling of those to avoid breakage

[FlorianWilhelm]

Thanks, @RonnyPfannschmidt for explaining. So it's best if we add this to our FAQs I guess. Could you imagine why this might have worked with an older version of setuptools-scm (vendorized in PyScaffold) as @ashic says?

[RonnyPfannschmidt]

Indeed, earlier versions had a number of issues with packaging /repacking