Q: how to build install and test flit in isolated network? #406
Comments
|
I think you've made a mistake with PYTHONPATH. You've put the Try something like this: cd flit-3.2.0
cd flit_core
/usr/bin/python3 -m build --no-isolation
cd ..
PYTHONPATH=$PWD/flit_core /usr/bin/python3 -m build --no-isolation |
|
It works. Thank you :) Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.o68bet
+ umask 022
+ cd /home/tkloczko/rpmbuild/BUILD
+ cd flit-3.2.0
+ cd flit_core
+ FLIT_NO_NETWORK=1
+ /usr/bin/python3 -m build --no-isolation
+ cd -
/home/tkloczko/rpmbuild/BUILD/flit-3.2.0
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILD/flit-3.2.0/flit_core
+ /usr/bin/python3 -m build --no-isolation
+ RPM_EC=0
++ jobs -p
+ exit 0Sorry for maybe dumb questions (I'm still learning how to use |
|
OK looks like testting can be done using pytest/tox. + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-flit-3.2.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-flit-3.2.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/python3 -Bm pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.9, pytest-6.2.3, py-1.10.0, pluggy-0.13.1
rootdir: /home/tkloczko/rpmbuild/BUILD/flit-3.2.0
plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, asyncio-0.14.0, expect-1.1.0, cov-2.11.1, mock-3.5.1, httpbin-1.0.0, xdist-2.2.1, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, pyfakefs-4.4.0, freezegun-0.4.2, flaky-3.7.0, cases-3.4.6, hypothesis-6.10.0
collected 169 items
flit_core/flit_core/tests/test_build_thyself.py ... [ 1%]
flit_core/flit_core/tests/test_buildapi.py ....... [ 5%]
flit_core/flit_core/tests/test_common.py ................ [ 15%]
flit_core/flit_core/tests/test_config.py ............................................ [ 41%]
flit_core/flit_core/tests/test_sdist.py ..... [ 44%]
flit_core/flit_core/tests/test_versionno.py . [ 44%]
flit_core/flit_core/tests/test_wheel.py . [ 45%]
tests/test_build.py .... [ 47%]
tests/test_command.py .. [ 49%]
tests/test_config.py . [ 49%]
tests/test_find_python_executable.py ...F... [ 53%]
tests/test_init.py ................. [ 63%]
tests/test_install.py ................... [ 75%]
tests/test_sdist.py .......... [ 81%]
tests/test_tomlify.py . [ 81%]
tests/test_upload.py .... [ 84%]
tests/test_validate.py ............... [ 92%]
tests/test_vcs.py . [ 93%]
tests/test_wheel.py ........... [100%]
================================================================================= FAILURES =================================================================================
____________________________________________________________________________ test_find_in_path _____________________________________________________________________________
def test_find_in_path():
> assert os.path.isabs(find_python_executable("python"))
tests/test_find_python_executable.py:23:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
python = 'python'
def find_python_executable(python: Optional[str] = None) -> str:
"""Returns an absolute filepath to the executable of Python to use."""
if not python:
python = os.environ.get("FLIT_INSTALL_PYTHON")
if not python:
return sys.executable
if os.path.isabs(python): # sys.executable is absolute too
return python
# get absolute filepath of {python}
# shutil.which may give a different result to the raw subprocess call
# see https://github.com/takluyver/flit/pull/300 and https://bugs.python.org/issue38905
resolved_python = shutil.which(python)
if resolved_python is None:
> raise PythonNotFoundError("Unable to resolve Python executable {!r}".format(python))
E flit.PythonNotFoundError: Unable to resolve Python executable 'python'
flit/__init__.py:36: PythonNotFoundError
========================================================================= short test summary info ==========================================================================
FAILED tests/test_find_python_executable.py::test_find_in_path - flit.PythonNotFoundError: Unable to resolve Python executable 'python'
====================================================================== 1 failed, 168 passed in 2.67s =======================================================================I always preffer to use pytest over tox as tox has less dependencies. Nevertheless still did not figure out how to install all flibt files in some base directory tree to be able package all those files. |
|
No problem - Installation: at the moment, the canonical way is to install wheels is to use Test failures: The tests assume that |
OK will try to package installer module :)
I have only [tkloczko@barrel SPECS]$ type python
-bash: type: python: not found
[tkloczko@barrel SPECS]$ type python3
python3 is /usr/bin/python3I patch for myself that unit but maybe it is some other generic metbid finding python executable which could be usied in that unit? + /usr/bin/python3 -Bm tox --skip-missing-interpreters
py39 create: /home/tkloczko/rpmbuild/BUILD/flit-3.2.0/.tox/py39
SKIPPED: InterpreterNotFound: python3.9
py38 create: /home/tkloczko/rpmbuild/BUILD/flit-3.2.0/.tox/py38
py38 installdeps: requests, requests_download, testpath, responses, docutils, toml, pytest>=2.7.3, pytest-cov
py38 installed: attrs==20.3.0,certifi==2020.12.5,chardet==4.0.0,coverage==5.5,docutils==0.17.1,idna==2.10,iniconfig==1.1.1,packaging==20.9,pluggy==0.13.1,py==1.10.0,pyparsing==2.4.7,pytest==6.2.3,pytest-cov==2.11.1,requests==2.25.1,requests-download==0.1.2,responses==0.13.2,six==1.15.0,testpath==0.4.4,toml==0.10.2,urllib3==1.26.4
py38 run-test-pre: PYTHONHASHSEED='687411366'
py38 run-test: commands[0] | python -m pytest --cov=flit --cov=flit_core/flit_core
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.9, pytest-6.2.3, py-1.10.0, pluggy-0.13.1
cachedir: .tox/py38/.pytest_cache
rootdir: /home/tkloczko/rpmbuild/BUILD/flit-3.2.0
plugins: cov-2.11.1
collected 169 items
flit_core/flit_core/tests/test_build_thyself.py ... [ 1%]
flit_core/flit_core/tests/test_buildapi.py ....... [ 5%]
flit_core/flit_core/tests/test_common.py ................ [ 15%]
flit_core/flit_core/tests/test_config.py ............................................ [ 41%]
flit_core/flit_core/tests/test_sdist.py ..... [ 44%]
flit_core/flit_core/tests/test_versionno.py . [ 44%]
flit_core/flit_core/tests/test_wheel.py . [ 45%]
tests/test_build.py .... [ 47%]
tests/test_command.py .. [ 49%]
tests/test_config.py . [ 49%]
tests/test_find_python_executable.py ....... [ 53%]
tests/test_init.py ................. [ 63%]
tests/test_install.py ................... [ 75%]
tests/test_sdist.py .......... [ 81%]
tests/test_tomlify.py . [ 81%]
tests/test_upload.py .... [ 84%]
tests/test_validate.py ............... [ 92%]
tests/test_vcs.py . [ 93%]
tests/test_wheel.py ........... [100%]
----------- coverage: platform linux, python 3.8.9-final-0 -----------
Name Stmts Miss Cover
----------------------------------------------------------
flit/__init__.py 89 23 74%
flit/__main__.py 3 0 100%
flit/_get_dirs.py 14 9 36%
flit/build.py 45 2 96%
flit/buildapi.py 1 1 0%
flit/config.py 11 1 91%
flit/init.py 155 14 91%
flit/install.py 262 32 88%
flit/log.py 49 27 45%
flit/logo.py 2 2 0%
flit/sdist.py 120 10 92%
flit/tomlify.py 56 4 93%
flit/upload.py 115 31 73%
flit/validate.py 174 13 93%
flit/vcs/__init__.py 11 0 100%
flit/vcs/git.py 9 0 100%
flit/vcs/hg.py 23 0 100%
flit/vendorized/__init__.py 0 0 100%
flit/vendorized/readme/__init__.py 0 0 100%
flit/vendorized/readme/clean.py 1 0 100%
flit/vendorized/readme/rst.py 36 12 67%
flit/wheel.py 7 0 100%
flit_core/flit_core/__init__.py 1 0 100%
flit_core/flit_core/build_thyself.py 44 5 89%
flit_core/flit_core/buildapi.py 41 0 100%
flit_core/flit_core/common.py 229 6 97%
flit_core/flit_core/config.py 343 21 94%
flit_core/flit_core/sdist.py 122 5 96%
flit_core/flit_core/versionno.py 45 2 96%
flit_core/flit_core/wheel.py 165 28 83%
----------------------------------------------------------
TOTAL 2173 248 89%
=========================================================================== 169 passed in 4.76s ============================================================================ |
It's testing a function that finds a Python executable, so there isn't an easy way around that...
Tox creates a virtualenv to run the tests, and inside the virtualenv there will be a |
|
I think that I'll try patch that sibgle line in test unit to use If you have nothing more to add or maybe correct in flit code feel free to close that ticket. |
|
Thanks! |
|
Sorry to bother you one more time. Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.tIpYVU
+ umask 022
+ cd /home/tkloczko/rpmbuild/BUILD
+ cd flit-3.2.0
+ cd flit_core
+ FLIT_NO_NETWORK=1
+ /usr/bin/python3 -Bm build --no-isolation
+ cd -
/home/tkloczko/rpmbuild/BUILD/flit-3.2.0
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILD/flit-3.2.0/flit_core
+ /usr/bin/python3 -m build --no-isolation
ERROR Missing dependencies:
flit_core >=3.2.0,<3.3Seems only thing which I've updated in mean time is builder module to version 0.4.0. [tkloczko@barrel flit-3.2.0]$ PYTHONPATH=/home/tkloczko/rpmbuild/BUILD/flit-3.2.0/flit_core strace -fe trace=openat /usr/bin/python3 -m build --no-isolation -s 2>&1 |grep core
openat(AT_FDCWD, "/home/tkloczko/rpmbuild/BUILD/flit-3.2.0/flit_core", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
[pid 2235140] openat(AT_FDCWD, "/home/tkloczko/rpmbuild/BUILD/flit-3.2.0/flit_core", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
[pid 2235140] openat(AT_FDCWD, "/home/tkloczko/rpmbuild/BUILD/flit-3.2.0/flit_core/flit_core/__pycache__/__init__.cpython-38.pyc", O_RDONLY|O_CLOEXEC) = 3
[pid 2235140] openat(AT_FDCWD, "/home/tkloczko/rpmbuild/BUILD/flit-3.2.0/flit_core/flit_core", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
[pid 2235140] openat(AT_FDCWD, "/home/tkloczko/rpmbuild/BUILD/flit-3.2.0/flit_core/flit_core/__pycache__/buildapi.cpython-38.pyc", O_RDONLY|O_CLOEXEC) = 3
[pid 2235140] openat(AT_FDCWD, "/home/tkloczko/rpmbuild/BUILD/flit-3.2.0/flit_core/flit_core/__pycache__/common.cpython-38.pyc", O_RDONLY|O_CLOEXEC) = 3
[pid 2235140] openat(AT_FDCWD, "/home/tkloczko/rpmbuild/BUILD/flit-3.2.0/flit_core/flit_core/__pycache__/versionno.cpython-38.pyc", O_RDONLY|O_CLOEXEC) = 3
[pid 2235140] openat(AT_FDCWD, "/home/tkloczko/rpmbuild/BUILD/flit-3.2.0/flit_core/flit_core/__pycache__/config.cpython-38.pyc", O_RDONLY|O_CLOEXEC) = 3
[pid 2235140] openat(AT_FDCWD, "/home/tkloczko/rpmbuild/BUILD/flit-3.2.0/flit_core/flit_core/__pycache__/wheel.cpython-38.pyc", O_RDONLY|O_CLOEXEC) = 3
[pid 2235140] openat(AT_FDCWD, "/home/tkloczko/rpmbuild/BUILD/flit-3.2.0/flit_core/flit_core/__pycache__/sdist.cpython-38.pyc", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/home/tkloczko/rpmbuild/BUILD/flit-3.2.0/flit_core", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
flit_core >=3.2.0,<3.3 |
|
It looks like the build package now checks for dependencies, rather than attempting the build and trusting you that dependencies are in place (you're setting I can see two possible approaches to deal with that:
|
|
OK good to know that it nothing wrong with my build env :) And another though .. IIRC I saw few modules which effectively have been installing more than one directory in python tree and everything was handled by single setup.py so if I'm right IMO something like this should be possible to do using Just tested build with + /usr/bin/python3 -Bm pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.9, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: /home/tkloczko/rpmbuild/BUILD/flit-3.2.0
plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, expect-1.1.0, cov-2.11.1, httpbin-1.0.0, xdist-2.2.1, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, pyfakefs-4.4.0, freezegun-0.4.2, cases-3.4.6, case-1.5.3, isort-1.3.0, aspectlib-1.5.2, asyncio-0.15.1, toolbox-0.5, xprocess-0.17.1, flaky-3.7.0, aiohttp-0.3.0, checkdocs-2.7.0, mock-3.6.1, rerunfailures-9.1.1, requests-mock-1.9.3, hypothesis-6.13.7
collected 190 items
. . [ 0%]
flit_core/flit_core/tests/test_build_thyself.py ... [ 2%]
flit_core/flit_core/tests/test_buildapi.py ....... [ 6%]
flit_core/flit_core/tests/test_common.py ................ [ 15%]
flit_core/flit_core/tests/test_config.py ............................................ [ 41%]
flit_core/flit_core/tests/test_sdist.py ..... [ 44%]
flit_core/flit_core/tests/test_versionno.py . [ 45%]
flit_core/flit_core/tests/test_wheel.py . [ 45%]
tests/test_build.py .... [ 48%]
tests/test_command.py FF [ 49%]
tests/test_config.py . [ 50%]
tests/test_find_python_executable.py ....... [ 54%]
tests/test_init.py ................. [ 64%]
tests/test_install.py ................... [ 75%]
tests/test_sdist.py .......... [ 81%]
tests/test_tomlify.py . [ 81%]
tests/test_upload.py .... [ 84%]
tests/test_validate.py ............... [ 92%]
tests/test_vcs.py . [ 93%]
tests/test_wheel.py ........... [100%]
================================================================================= FAILURES =================================================================================
______________________________________________________________________________ test_flit_help ______________________________________________________________________________
def test_flit_help():
p = Popen([sys.executable, '-m', 'flit', '--help'], stdout=PIPE, stderr=STDOUT)
out, _ = p.communicate()
> assert 'Build wheel' in out.decode('utf-8', 'replace')
E assert 'Build wheel' in 'Traceback (most recent call last):\n File "/usr/lib64/python3.8/runpy.py", line 185, in _run_module_as_main\n mod...\n from flit_core import common\nImportError: cannot import name \'common\' from \'flit_core\' (unknown location)\n'
E + where 'Traceback (most recent call last):\n File "/usr/lib64/python3.8/runpy.py", line 185, in _run_module_as_main\n mod...\n from flit_core import common\nImportError: cannot import name \'common\' from \'flit_core\' (unknown location)\n' = <built-in method decode of bytes object at 0x561c18f8a0f0>('utf-8', 'replace')
E + where <built-in method decode of bytes object at 0x561c18f8a0f0> = b'Traceback (most recent call last):\n File "/usr/lib64/python3.8/runpy.py", line 185, in _run_module_as_main\n mo...\n from flit_core import common\nImportError: cannot import name \'common\' from \'flit_core\' (unknown location)\n'.decode
tests/test_command.py:7: AssertionError
_____________________________________________________________________________ test_flit_usage ______________________________________________________________________________
def test_flit_usage():
p = Popen([sys.executable, '-m', 'flit'], stdout=PIPE, stderr=STDOUT)
out, _ = p.communicate()
> assert 'Build wheel' in out.decode('utf-8', 'replace')
E assert 'Build wheel' in 'Traceback (most recent call last):\n File "/usr/lib64/python3.8/runpy.py", line 185, in _run_module_as_main\n mod...\n from flit_core import common\nImportError: cannot import name \'common\' from \'flit_core\' (unknown location)\n'
E + where 'Traceback (most recent call last):\n File "/usr/lib64/python3.8/runpy.py", line 185, in _run_module_as_main\n mod...\n from flit_core import common\nImportError: cannot import name \'common\' from \'flit_core\' (unknown location)\n' = <built-in method decode of bytes object at 0x561c18f90c80>('utf-8', 'replace')
E + where <built-in method decode of bytes object at 0x561c18f90c80> = b'Traceback (most recent call last):\n File "/usr/lib64/python3.8/runpy.py", line 185, in _run_module_as_main\n mo...\n from flit_core import common\nImportError: cannot import name \'common\' from \'flit_core\' (unknown location)\n'.decode
tests/test_command.py:12: AssertionError
========================================================================= short test summary info ==========================================================================
FAILED tests/test_command.py::test_flit_help - assert 'Build wheel' in 'Traceback (most recent call last):\n File "/usr/lib64/python3.8/runpy.py", line 185, in _run_mod...
FAILED tests/test_command.py::test_flit_usage - assert 'Build wheel' in 'Traceback (most recent call last):\n File "/usr/lib64/python3.8/runpy.py", line 185, in _run_mo...
====================================================================== 2 failed, 168 passed in 4.46s =======================================================================FYI to be able to use tox/pytest with /usr/bin/python3 I've added below patch: --- a/tests/test_find_python_executable.py~ 2021-03-21 21:18:18.000000000 +0000
+++ b/tests/test_find_python_executable.py 2021-05-30 19:17:45.840724117 +0100
@@ -16,14 +16,14 @@
def test_abs():
- assert find_python_executable("/usr/bin/python") == "/usr/bin/python"
+ assert find_python_executable("/usr/bin/python3") == "/usr/bin/python3"
def test_find_in_path():
- assert os.path.isabs(find_python_executable("python"))
+ assert os.path.isabs(find_python_executable("python3"))
-@pytest.mark.parametrize("bad_python_name", ["pyhton", "ls", "."])
+@pytest.mark.parametrize("bad_python_name", ["pyhton3", "ls", "."])
def test_exception(bad_python_name: str):
"""Test that an appropriate exception (that contains the error string) is raised."""
with pytest.raises(PythonNotFoundError, match=re.escape(bad_python_name)):
|
|
And yet another small question (if I can) ,. I'm packaging all my python modules adding documentation in form of roff man pages (if it is only possible). |
|
And yet another observation. + sphinx-build -b man -d python-flit doc .
Running Sphinx v4.0.2
Extension error:
Could not import extension sphinxcontrib_github_alt (exception: No module named 'sphinxcontrib_github_alt')That extension looks like has some substantial set of dependencies. As |
Adding it to PYTHONPATH makes it importable, but it doesn't show up as an installed distribution for the requirements check. That mechanism is based on
I've split it into two pieces precisely because I want it to be possible to install them separately. Every package which is built with Flit will have
That looks like when it runs
Definitely not at present, and I'd guess it's not going to be. The idea of
Using Flit to build other packages shouldn't require that Flit's docs are built first. It shouldn't even require the whole of Flit, just |
|
Thank you for your comments :)
Working +20 years on packaging software I know how important is not only building and installing (which still is not integrated part of the
So far I've added to my package patch which is removing history from the documentation (only that part requires that sphinx extension): --- a/doc/conf.py~ 2021-05-30 20:16:03.000000000 +0100
+++ b/doc/conf.py 2021-05-30 20:16:28.697651673 +0100
@@ -30,7 +30,6 @@
# ones.
extensions = [
'sphinx.ext.autodoc',
- 'sphinxcontrib_github_alt',
'sphinx_rtd_theme',
]
--- a/doc/index.rst~ 2021-03-21 21:18:18.000000000 +0000
+++ b/doc/index.rst 2021-05-30 20:16:37.874664255 +0100
@@ -23,7 +23,6 @@
:maxdepth: 1
development
- history
Indices and tables
==================
Than with that I'm able to generate man page using It is quite long list for something like only unpacking And again .. thanks for your comments because now I have better understanding of some details. |
I'm not saying they're not important, just that I don't think the
I'd rather you either built the docs entirely, or left them out and pointed people to the online docs, rather than building part of the docs with no sign that you've cut part out. Sphinx has 16 direct dependencies and likely several more indirect ones, so it's not exactly straightforward or lightweight just because you've removed one small extension. I don't think any of Sphinx's dependencies are packaged with Flit at present, but there's no reason they couldn't be.
Do you mean |
|
OK in case of sphinx dependencies you are right.
Nope rpm automatically process that file to autogenerate generated package Still I cannot install that package because I must package all what installed needs. [tkloczko@barrel .git]$ sudo dnf install 'python3dist(installer)'
Last metadata expiration check: 0:32:48 ago on Tue 01 Jun 2021 11:09:56 BST.
Error:
Problem: conflicting requests
- nothing provides python3.8dist(packagebuilder) needed by python-installer-0.1.1-2.fc35.noarch
- nothing provides python3.8dist(pip-shims) needed by python-installer-0.1.1-2.fc35.noarch
- nothing provides python3.8dist(requirementslib) needed by python-installer-0.1.1-2.fc35.noarch
- nothing provides python3.8dist(vistir) needed by python-installer-0.1.1-2.fc35.noarch
- nothing provides python3.8dist(vistir[spinner]) needed by python-installer-0.1.1-2.fc35.noarch
(try to add '--skip-broken' to skip uninstallable packages) |
|
BTW just checked Is it true that |
It looks like the name However, it is packaged with flit_core, so you'll need some workaround to get at least one of those installed.
Not really, no. Since PEP 517, we separate these tools into 'backends', like setuptools or flit, which a project chooses to define how to make packages, and 'frontends', like pip & The picture is a bit muddy because many of the tools predate PEP 517 and so do bits of both frontend & backend things. E.g. |
|
Thanks for all your help. I've manage to package all basic module and flit as well. I found that many modules which no longer have setup.py and instead they have project.toml file in sdist dist tarballs have generated in that tar ball setup.py are using distutils instead setuptools. IMO using distutils instead setuptools is kind of step back from point of functionality. Is it intentional that Sorry for asking you again but looks like you are quite close to developing all build/install/test python frameworks so you are the person which could be best suited to address that kind of question :) |
|
The |
|
(To be clear, I mean that |
I'm trying to build flit in isolated network and really struggling with doing that.
I'm trying to use
buildmodule like with few other modulke and to be hones I'm stuckMay I ask for some hints/advices or help?
The text was updated successfully, but these errors were encountered: