[question] Installing conan-2.4.0 on gentoo linux goes with error

[dpronin]

What is your question?

Hi,

I have gentoo linux distribution and like to install using portage package manager, that says to me when I install conan-2.4.0:

>>> Install dev-util/conan-2.4.0 into /var/tmp/portage/dev-util/conan-2.4.0/image
 * python3_12: running distutils-r1_run_phase distutils-r1_python_install
 * The following unexpected files/directories were found top-level
 * in the site-packages directory:
 * 
 *   /usr/lib/python3.12/site-packages/test
 * 
 * This is most likely a bug in the build system.  More information
 * can be found in the Python Guide:
 * https://projects.gentoo.org/python/guide/qawarn.html#stray-top-level-files-in-site-packages
 * ERROR: dev-util/conan-2.4.0::gentoo failed (install phase):
 *   Failing install because of stray top-level files in site-packages

This error does not appear with conan-2.3.1

I noticed that you had moved conans/test to test directory that might be the reason

Do you consider installation to site-packages/test a violation some python-related building rules? Or is it only a gentoo's consideration and you see nothing harmful for the system in it?

Thank you in advance

Have you read the CONTRIBUTING guide?

• I've read the CONTRIBUTING guide

[memsharded]

Hi @dpronin

Thanks for reporting.

We indeed moved the test folder from inside the conans one, to first level. This is something most Python projects do nowadays, seems a pretty standard project layout.
We just had to change the setup.py to:

excluded_test_packages = ["test"]

And that was mostly it for releasing it into PyPI, and our other installers apparently were created without issues too.
Unfortunately I don't know enough about Gentoo, besides the fact that they system installer "builds everything from source", so it seems it is only something very specific of Gentoo, we might need some others expertise here.

[dpronin]

@memsharded thank you for the answer

[memsharded]

You can leave it open until this is solved, so it might have a bit more of visibility, did you already find a solution?

[dpronin]

You can leave it open until this is solved, so it might have a bit more of visibility, did you already find a solution?

I have concluded that you tend to use some "common" rules that python packages adhere nowadays, so for me this means that Gentoo is just stricter about the installation paths

[memsharded]

But if Gentoo users cannot install Conan, this is a valid issue and should remain open, until they can. You might be the first, but might be others (this is not the first Gentoo issue reported, I have seen some Gentoo users in the past).

I have just had a look to https://projects.gentoo.org/python/guide/qawarn.html#stray-top-level-files-in-site-packages, and it seems something that would require some updates to our setup.py or setup.cfg files, isn't it? They actually recommend fixing things "upstream", in this case this repo:

In general, it is desirable to prepare a fix for the build system and submit it upstream.

I am re-opening this, I am happy to change or improve Conan setup files if possible if that helps Gentoo users

[dpronin]

@memsharded I am not the first but sooner an ambassador of conan in gentoo distributions and support keeping conan fresh in gentoo's package manager, that's it
I very appreciate your intention to help gentoo's people, but I am not sure if this circumstance is a Gentoo's issue or conan's one. This is my concern and confusion. I do not negate the issue, the issue exists. But who is guilty I cannot tell for sure

[memsharded]

That is fine, we prefer to be sure, keep the issue open and investigate if possible, than closing prematurely as "not our fault". As long as users cannot install Conan, it is "our" issue, doesn't really matter if the solution is in our repos or in other peoples repos, it is affecting Conan users, so we will try our best to solve it 🙂

[dpronin]

@memsharded you behave very wisely when attracting more audience by undertaking all issues and not passing "not our faults" by. This scores you up. This is the reason, besides convenience and flexibility of the tool, why conan is getting more and more popular and beloved by the people

[memsharded]

This is what I have tried so far:

• Use the gentoo/stage3 official docker image
• python -m venv myvenv
• . myvenv/bin/activate
• python -m pip install conan (installs correctly)

$ conan --version
Conan version 2.4.0

Maybe is is another version? It would be nice to have some more detailed reproducible steps to try things out.

[dpronin]

@memsharded with virtual environment everything works, the issue is not in installation itself but paths those are engaged in installation. When you use pip it does not double-check paths where setup.py installs its assets, but portage (package manager of Gentoo) has system of, so-called, ebuilds with its infrastructure and environment that somewhy (maybe it is reasonable and meaningful, I do not know) checks if a package being installed is not installing its assets to site-packages/test. Probably, portage (package manager) believes that site-packages/test is a "global" directory for Gentoo's python and cannot be touched by any packages being installed

[memsharded]

What would be the instructions to tell portage to install Conan?

[dpronin]

What would be the instructions to tell portage to install Conan?

I don't think you will be happy of getting into this story, this will be one way ticket

[ravenexp]

I have the same issue with the Arch Linux package for conan-2.4.0. It suddenly started packaging all its unit tests and installing it globally in the system site-packages directory, which is something no well-behaved Python package should do. The Python package namespace is global, so there can be only one package named "test" on a system.

We indeed moved the test folder from inside the conans one, to first level. This is something most Python projects do nowadays, seems a pretty standard project layout. We just had to change the setup.py to:

excluded_test_packages = ["test"]

And that was mostly it for releasing it into PyPI, and our other installers apparently were created without issues too.

That setup.py line does not seem to work as intended though, because the "test" directory is now packaged and installed as a top-level Python package alongside with "conan" and "conans".

You can easily reproduce this by running python -m build --wheel in the top project directory. By looking at the build output, it's obvious the "test" package goes into the wheel, and the wheel is what is then installed on the user system.

UPD:

It looks like it might also be a bug in build or setuptools, because when running build with default arguments to build both the package wheel and the sdist tarball, the "test" directory is only included in the sdist, but not in the wheel.

My Python-related software versions are as follows:

• Python 3.12.3
• pip 24.0
• build 1.2.1
• setuptools 69.0.3
• wheel 0.43.0

[memsharded]

Thanks very much @ravenexp for the detailed feedback, it clarifies a lot.

It could be that there is some discrepancy in how the exclude patterns are evaluated. I am submitting #16446, that it seems it could fix it, in case you want to give it a try on your side too.

I am targeting that fix for Conan 2.4.1 next patch release, to be released soon.

[ravenexp]

I tested #16446 locally on my Arch system, and building the Arch package via python -m build --wheel now works as expected. Thank you for your prompt response.

[memsharded]

Great, thanks for the feedback!

[memsharded]

#16446 has been merged for next Conan 2.4.1, closing this ticket as solved, but please re-open or create a new one if it is not the case, thanks for the feedback!