entrypoints/difficulty building debian pkgs of extensions

[yarikoptic]

we would definitely like to run tests while building debian pkgs of the extensions, but the problem is that extension entry points aren't found. I guess it is due to the fact that pybuild by default runs against "built" version, not installed:

   dh_auto_build -O--buildsystem=pybuild
I: pybuild base:217: /usr/bin/python3 setup.py build 
running build
running build_py
creating /home/yoh/proj/datalad/build-area/datalad-container-0.2.2/.pybuild/cpython3_3.7_datalad-container/build/datalad_container
copying datalad_container/__init__.py -> /home/yoh/proj/datalad/build-area/datalad-container-0.2.2/.pybuild/cpython3_3.7_datalad-container/build/datalad_container
copying datalad_container/containers_add.py -> /home/yoh/proj/datalad/build-area/datalad-container-0.2.2/.pybuild/cpython3_3.7_datalad-container/build/datalad_container
copying datalad_container/containers_list.py -> /home/yoh/proj/datalad/build-area/datalad-container-0.2.2/.pybuild/cpython3_3.7_datalad-container/build/datalad_container
copying datalad_container/containers_remove.py -> /home/yoh/proj/datalad/build-area/datalad-container-0.2.2/.pybuild/cpython3_3.7_datalad-container/build/datalad_container
copying datalad_container/containers_run.py -> /home/yoh/proj/datalad/build-area/datalad-container-0.2.2/.pybuild/cpython3_3.7_datalad-container/build/datalad_container
copying datalad_container/definitions.py -> /home/yoh/proj/datalad/build-area/datalad-container-0.2.2/.pybuild/cpython3_3.7_datalad-container/build/datalad_container
copying datalad_container/version.py -> /home/yoh/proj/datalad/build-area/datalad-container-0.2.2/.pybuild/cpython3_3.7_datalad-container/build/datalad_container
creating /home/yoh/proj/datalad/build-area/datalad-container-0.2.2/.pybuild/cpython3_3.7_datalad-container/build/datalad_container/adapters
copying datalad_container/adapters/__init__.py -> /home/yoh/proj/datalad/build-area/datalad-container-0.2.2/.pybuild/cpython3_3.7_datalad-container/build/datalad_container/adapters
copying datalad_container/adapters/docker.py -> /home/yoh/proj/datalad/build-area/datalad-container-0.2.2/.pybuild/cpython3_3.7_datalad-container/build/datalad_container/adapters
creating /home/yoh/proj/datalad/build-area/datalad-container-0.2.2/.pybuild/cpython3_3.7_datalad-container/build/datalad_container/tests
copying datalad_container/tests/__init__.py -> /home/yoh/proj/datalad/build-area/datalad-container-0.2.2/.pybuild/cpython3_3.7_datalad-container/build/datalad_container/tests
copying datalad_container/tests/test_containers.py -> /home/yoh/proj/datalad/build-area/datalad-container-0.2.2/.pybuild/cpython3_3.7_datalad-container/build/datalad_container/tests
copying datalad_container/tests/test_register.py -> /home/yoh/proj/datalad/build-area/datalad-container-0.2.2/.pybuild/cpython3_3.7_datalad-container/build/datalad_container/tests
copying datalad_container/tests/test_run.py -> /home/yoh/proj/datalad/build-area/datalad-container-0.2.2/.pybuild/cpython3_3.7_datalad-container/build/datalad_container/tests
copying datalad_container/tests/utils.py -> /home/yoh/proj/datalad/build-area/datalad-container-0.2.2/.pybuild/cpython3_3.7_datalad-container/build/datalad_container/tests
setup.py:40: DeprecationWarning: Due to possible ambiguity, 'convert()' is deprecated. Use 'convert_file()'  or 'convert_text()'.
  long_description = pypandoc.convert(README, 'rst')
   dh_auto_test -O--buildsystem=pybuild
I: pybuild base:217: cd /home/yoh/proj/datalad/build-area/datalad-container-0.2.2/.pybuild/cpython3_3.7_datalad-container/build; python3.7 -m nose -v 
Failure: ImportError (cannot import name 'containers_add' from 'datalad.api' (/usr/lib/python3/dist-packages/datalad/api.py)) ... ERROR
datalad_container.tests.test_register.test_register ... FAIL
Failure: ImportError (cannot import name 'containers_add' from 'datalad.api' (/usr/lib/python3/dist-packages/datalad/api.py)) ... ERROR

======================================================================
ERROR: Failure: ImportError (cannot import name 'containers_add' from 'datalad.api' (/usr/lib/python3/dist-packages/datalad/api.py))
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/nose/failure.py", line 39, in runTest
    raise self.exc_val.with_traceback(self.tb)
  File "/usr/lib/python3/dist-packages/nose/loader.py", line 417, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python3/dist-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python3/dist-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/usr/lib/python3.7/imp.py", line 234, in load_module
    return load_source(name, filename, file)
  File "/usr/lib/python3.7/imp.py", line 171, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 696, in _load
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/yoh/proj/datalad/build-area/datalad-container-0.2.2/.pybuild/cpython3_3.7_datalad-container/build/datalad_container/tests/test_containers.py", line 5, in <module>
    from datalad.api import containers_add
ImportError: cannot import name 'containers_add' from 'datalad.api' (/usr/lib/python3/dist-packages/datalad/api.py)

not yet sure what would be the best way to overcome

[bpoldrack]

Maybe this helps to provide a reasonable test environment?

--test-tox
    use tox command in test step, remember to add python-tox to Build-Depends. Requires tox.ini file 

[yarikoptic]

for now I did an ugly way:

• disable pybuild from doing tests
• run them manually https://github.com/datalad/datalad-container/blob/debian/debian/rules#L4

tox idea is interesting! I will try and report back

[yarikoptic]

although may be wouldn't -- such tox setup would require duplication of what debian/ expects (on versions of python etc) and tox.ini specifying all that. sounds also a bit fragile due to duplication

[bpoldrack]

Alternatively, according to man page pybuild provides --before-tests and --after-tests to pass commands to be run before and after. And it looks for tests under {build_dir}/tests. May be you could use those commands to have what is in {build_dir} actually installed for the testing and restore the original {build_dir} afterwards?

[bpoldrack]

Anyways, I guess it doesn't really get less "ugly" than what you have.

[yarikoptic]

ha ha -- searched up my previous question: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=900526 . and the resolution was to use debian/pybuild.testfiles to copy/install whatever is needed to be copied. But then in 007cee3 I had to revert it in favor of a simple cp of .egg-info since that machinery was not supported by older pybuilds.
Meanwhile I followed up with a more generic question, that ideally it is not just setup.py build but full setup.py install is run for the purpose of testing

[mih]

I am closing this one, as no further info is coming.

[yarikoptic]

"no further info" != "issue was resolved". "no further info" == "no further info, e.g. due to lack of time etc".
Let's not close issues merely because of their age and without any sign/confirmation of them being resolved.

In datalad-container I see that I simply disabled testing for now, which is also not a solution: https://github.com/datalad/datalad-container/blob/debian/debian/rules#L4

[mih]

Two years later, with no further updates, I am trying to close this issue again. AFAICS in https://buildd.debian.org/status/fetch.php?pkg=datalad-container&arch=all&ver=1.1.2-1&stamp=1612198173&raw=0 (some) tests run. So if my assessment is wrong again, at least an uodate of the status in this issue is due.