Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1.2.1: cannot build documentation without installing module #25766

Open
kloczek opened this issue Mar 6, 2023 · 22 comments
Open

1.2.1: cannot build documentation without installing module #25766

kloczek opened this issue Mar 6, 2023 · 22 comments
Labels
Bug Documentation Needs Decision - Include Feature Requires decision regarding including feature

Comments

@kloczek
Copy link

kloczek commented Mar 6, 2023

Describe the bug

Looks like something is wrong and I cannot build docuemtation without installing module.

Steps/Code to Reproduce

N/A

Expected Results

It should be possible to build documentation out of only what is in source tree a d withoit have installed module.

Actual Results

++ ls -1d lib.linux-x86_64-cpython-38
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILD/scikit-learn-1.2.1/build/lib.linux-x86_64-cpython-38
+ PBR_VERSION=1.2.1
+ PDM_PEP517_SCM_VERSION=1.2.1
+ SETUPTOOLS_SCM_PRETEND_VERSION=1.2.1
+ /usr/bin/sphinx-build -n -T -b man doc build/sphinx/man
Running Sphinx v6.1.3

Traceback (most recent call last):
  File "/home/tkloczko/rpmbuild/BUILD/scikit-learn-1.2.1/sklearn/__check_build/__init__.py", line 48, in <module>
    from ._check_build import check_build  # noqa
ModuleNotFoundError: No module named 'sklearn.__check_build._check_build'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/sphinx/config.py", line 351, in eval_config_file
    exec(code, namespace)  # NoQA: S102
  File "/home/tkloczko/rpmbuild/BUILD/scikit-learn-1.2.1/doc/conf.py", line 20, in <module>
    from sklearn.externals._packaging.version import parse
  File "/home/tkloczko/rpmbuild/BUILD/scikit-learn-1.2.1/sklearn/__init__.py", line 81, in <module>
    from . import __check_build  # noqa: F401
  File "/home/tkloczko/rpmbuild/BUILD/scikit-learn-1.2.1/sklearn/__check_build/__init__.py", line 50, in <module>
    raise_build_error(e)
  File "/home/tkloczko/rpmbuild/BUILD/scikit-learn-1.2.1/sklearn/__check_build/__init__.py", line 31, in raise_build_error
    raise ImportError(
ImportError: No module named 'sklearn.__check_build._check_build'
___________________________________________________________________________
Contents of /home/tkloczko/rpmbuild/BUILD/scikit-learn-1.2.1/sklearn/__check_build:
__init__.py               _check_build.pyx          _check_build.c
__pycache__
___________________________________________________________________________
It seems that scikit-learn has not been built correctly.

If you have installed scikit-learn from source, please do not forget
to build the package before using it: run `python setup.py install` or
`make` in the source directory.

If you have used an installer, please check that it is suited for your
Python version, your operating system and your platform.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/sphinx/cmd/build.py", line 279, in build_main
    app = Sphinx(args.sourcedir, args.confdir, args.outputdir,
  File "/usr/lib/python3.8/site-packages/sphinx/application.py", line 202, in __init__
    self.config = Config.read(self.confdir, confoverrides or {}, self.tags)
  File "/usr/lib/python3.8/site-packages/sphinx/config.py", line 173, in read
    namespace = eval_config_file(filename, tags)
  File "/usr/lib/python3.8/site-packages/sphinx/config.py", line 364, in eval_config_file
    raise ConfigError(msg % traceback.format_exc()) from exc
sphinx.errors.ConfigError: There is a programmable error in your configuration file:

Traceback (most recent call last):
  File "/home/tkloczko/rpmbuild/BUILD/scikit-learn-1.2.1/sklearn/__check_build/__init__.py", line 48, in <module>
    from ._check_build import check_build  # noqa
ModuleNotFoundError: No module named 'sklearn.__check_build._check_build'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/sphinx/config.py", line 351, in eval_config_file
    exec(code, namespace)  # NoQA: S102
  File "/home/tkloczko/rpmbuild/BUILD/scikit-learn-1.2.1/doc/conf.py", line 20, in <module>
    from sklearn.externals._packaging.version import parse
  File "/home/tkloczko/rpmbuild/BUILD/scikit-learn-1.2.1/sklearn/__init__.py", line 81, in <module>
    from . import __check_build  # noqa: F401
  File "/home/tkloczko/rpmbuild/BUILD/scikit-learn-1.2.1/sklearn/__check_build/__init__.py", line 50, in <module>
    raise_build_error(e)
  File "/home/tkloczko/rpmbuild/BUILD/scikit-learn-1.2.1/sklearn/__check_build/__init__.py", line 31, in raise_build_error
    raise ImportError(
ImportError: No module named 'sklearn.__check_build._check_build'
___________________________________________________________________________
Contents of /home/tkloczko/rpmbuild/BUILD/scikit-learn-1.2.1/sklearn/__check_build:
__init__.py               _check_build.pyx          _check_build.c
__pycache__
___________________________________________________________________________
It seems that scikit-learn has not been built correctly.

If you have installed scikit-learn from source, please do not forget
to build the package before using it: run `python setup.py install` or
`make` in the source directory.

If you have used an installer, please check that it is suited for your
Python version, your operating system and your platform.


Configuration error:
There is a programmable error in your configuration file:

Traceback (most recent call last):
  File "/home/tkloczko/rpmbuild/BUILD/scikit-learn-1.2.1/sklearn/__check_build/__init__.py", line 48, in <module>
    from ._check_build import check_build  # noqa
ModuleNotFoundError: No module named 'sklearn.__check_build._check_build'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/sphinx/config.py", line 351, in eval_config_file
    exec(code, namespace)  # NoQA: S102
  File "/home/tkloczko/rpmbuild/BUILD/scikit-learn-1.2.1/doc/conf.py", line 20, in <module>
    from sklearn.externals._packaging.version import parse
  File "/home/tkloczko/rpmbuild/BUILD/scikit-learn-1.2.1/sklearn/__init__.py", line 81, in <module>
    from . import __check_build  # noqa: F401
  File "/home/tkloczko/rpmbuild/BUILD/scikit-learn-1.2.1/sklearn/__check_build/__init__.py", line 50, in <module>
    raise_build_error(e)
  File "/home/tkloczko/rpmbuild/BUILD/scikit-learn-1.2.1/sklearn/__check_build/__init__.py", line 31, in raise_build_error
    raise ImportError(
ImportError: No module named 'sklearn.__check_build._check_build'
___________________________________________________________________________
Contents of /home/tkloczko/rpmbuild/BUILD/scikit-learn-1.2.1/sklearn/__check_build:
__init__.py               _check_build.pyx          _check_build.c
__pycache__
___________________________________________________________________________
It seems that scikit-learn has not been built correctly.

If you have installed scikit-learn from source, please do not forget
to build the package before using it: run `python setup.py install` or
`make` in the source directory.

If you have used an installer, please check that it is suited for your
Python version, your operating system and your platform.

Versions

1.2.1
@kloczek kloczek added Bug Needs Triage Issue requires triage labels Mar 6, 2023
@kloczek
Copy link
Author

kloczek commented Mar 6, 2023

I see what is going on.
Module code is using relative imprts whaich is causimg that DSOs must be in the source tree.
I'll try to prepare patch fro that.

@betatim betatim changed the title 1.2.1: cannot build documentation without intalling module 1.2.1: cannot build documentation without installing module Mar 6, 2023
@ogrisel
Copy link
Member

ogrisel commented Mar 9, 2023

To build the documentation including the examples, scikit-learn will need to be installed anyways. However if you can find a simple way to build the doc (without the examples) and without installing scikit-learn then why not.

@ogrisel ogrisel removed the Needs Triage Issue requires triage label Mar 9, 2023
@kloczek
Copy link
Author

kloczek commented Mar 9, 2023

To build the documentation including the examples, scikit-learn will need to be installed anyways. However if you can find a simple way to build the doc (without the examples) and without installing scikit-learn then why not.

Issue is that during typical packaging process execution there is no chance to to have module installed ... because module is build from scratch. Biggest obstacle on packaging that way are relative imports because they are forces to have DSOs in the actual source tree.
I have at he moment +170KB patch which replaces all relative imports by module name(s) and that patch still is not finished.
With such patch it is possible to build documentation with two adjustments:

  • add in conf.py sys.path.insert() which injects path to the source tree. In case of your module I have it already done and it looks like
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -12,6 +12,8 @@

 import sys
 import os
+sys.path.insert(0, os.path.abspath(".."))
+
 import warnings
 import re
 from datetime import datetime

If you will look om sphinx example copy.py from https://www.sphinx-doc.org/en/master/usage/configuration.html#example-of-configuration-file you can find that as part of that example

  • In case of modules which are using DSOs it is necessary to add second adjustment during sphinx-build command execution which looks like
[scikit-learn]$ PYTHONPATH=$PWD/build/$(cd build; ls -1d lib*) sphinx-build <params>

Those two adaptations allows open .py files from source tree and DSOs from second location where typical pep build procedure creates DSOs.

Benefit of above is that with those adaptations it is possible to build documentation without have installed it and by this have 100% guarantee that produced documentation will be generated not against what is installed (which may be different version) but what is in working source code tree.

Most of the python modules which are using DSOs already have above adaptation implemented.
Looks like scikit-learn is one of those rare cesses when that approach is not already implemented.

@kloczek
Copy link
Author

kloczek commented Apr 4, 2023

Just realised that build fails becauee incorrect module name in sklearn/__check_build/__init__.py.

The same line is causing that pytest is failing

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-scikit-learn-1.2.2-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-scikit-learn-1.2.2-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -m 'not network' --import-mode=importlib
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 774, in import_plugin
    __import__(importspec)
  File "/home/tkloczko/rpmbuild/BUILDROOT/python-scikit-learn-1.2.2-2.fc35.x86_64/usr/lib64/python3.8/site-packages/sklearn/__init__.py", line 81, in <module>
    from sklearn import __check_build  # noqa: F401
  File "/home/tkloczko/rpmbuild/BUILDROOT/python-scikit-learn-1.2.2-2.fc35.x86_64/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py", line 50, in <module>
    raise_build_error(e)
  File "/home/tkloczko/rpmbuild/BUILDROOT/python-scikit-learn-1.2.2-2.fc35.x86_64/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py", line 31, in raise_build_error
    raise ImportError(
ImportError: No module named 'sklearn._check_build'
___________________________________________________________________________
Contents of /home/tkloczko/rpmbuild/BUILDROOT/python-scikit-learn-1.2.2-2.fc35.x86_64/usr/lib64/python3.8/site-packages/sklearn/__check_build:
__init__.py               _check_build.cpython-38-x86_64-linux-gnu.so__pycache__
___________________________________________________________________________
It seems that scikit-learn has not been built correctly.

If you have installed scikit-learn from source, please do not forget
to build the package before using it: run `python setup.py install` or
`make` in the source directory.

If you have used an installer, please check that it is suited for your
Python version, your operating system and your platform.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/bin/pytest", line 8, in <module>
    sys.exit(console_main())
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 190, in console_main
    code = main()
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 148, in main
    config = _prepareconfig(args, plugins)
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 329, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
  File "/usr/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/usr/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/usr/lib/python3.8/site-packages/pluggy/_callers.py", line 55, in _multicall
    gen.send(outcome)
  File "/usr/lib/python3.8/site-packages/_pytest/helpconfig.py", line 103, in pytest_cmdline_parse
    config: Config = outcome.get_result()
  File "/usr/lib/python3.8/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1060, in pytest_cmdline_parse
    self.parse(args)
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1348, in parse
    self._preparse(args, addopts=addopts)
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1227, in _preparse
    self.pluginmanager.consider_preparse(args, exclude_only=False)
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 702, in consider_preparse
    self.consider_pluginarg(parg)
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 728, in consider_pluginarg
    self.import_plugin(arg, consider_entry_points=True)
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 776, in import_plugin
    raise ImportError(
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 774, in import_plugin
    __import__(importspec)
  File "/home/tkloczko/rpmbuild/BUILDROOT/python-scikit-learn-1.2.2-2.fc35.x86_64/usr/lib64/python3.8/site-packages/sklearn/__init__.py", line 81, in <module>
    from sklearn import __check_build  # noqa: F401
  File "/home/tkloczko/rpmbuild/BUILDROOT/python-scikit-learn-1.2.2-2.fc35.x86_64/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py", line 50, in <module>
    raise_build_error(e)
  File "/home/tkloczko/rpmbuild/BUILDROOT/python-scikit-learn-1.2.2-2.fc35.x86_64/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py", line 31, in raise_build_error
    raise ImportError(
ImportError: Error importing plugin "sklearn.tests.random_seed": No module named 'sklearn._check_build'
___________________________________________________________________________
Contents of /home/tkloczko/rpmbuild/BUILDROOT/python-scikit-learn-1.2.2-2.fc35.x86_64/usr/lib64/python3.8/site-packages/sklearn/__check_build:
__init__.py               _check_build.cpython-38-x86_64-linux-gnu.so__pycache__
___________________________________________________________________________
It seems that scikit-learn has not been built correctly.

If you have installed scikit-learn from source, please do not forget
to build the package before using it: run `python setup.py install` or
`make` in the source directory.

If you have used an installer, please check that it is suited for your
Python version, your operating system and your platform.

@lesteve
Copy link
Member

lesteve commented Apr 5, 2023

Issue is that during typical packaging process execution there is no chance to to have module installed ... because module is build from scratch.

I can't say I have a very good understanding of Linux distro packaging constraints, but would having a python-scikit-learn package and a python-scikit-learn-doc help?

It seems like this is done like this on Ubuntu (so probably Debian):

❯ apt-cache search python-sklearn     
python-sklearn-doc - documentation and examples for scikit-learn
python3-sklearn-lib - low-level implementations and bindings for scikit-learn - Python 3

@kloczek
Copy link
Author

kloczek commented Apr 5, 2023

Please instead focusing on linux packaging try to focus on output which I've provided you.

@lesteve
Copy link
Member

lesteve commented Apr 5, 2023

I am afraid I don't feel your tone is very constructive and I invite you to read our very short code of conduct in particular the part about "friendly and didactic discussions"

@betatim
Copy link
Member

betatim commented Apr 5, 2023

Could you explain a bit more what the goal is of being able to build the documentation after building the package but without installing it? What is the use-case where this is needed? What does it allow the user to do that is otherwise not possible?

Understanding those things will help avoid the X Y problem as well as give a motivation for why to make changes to support it. As an example for why this is needed is that something that one person thinks "would be nicer for me" is highly likely to be "something I consider wrong" for another user. We can't make everyone happy :-/

@kloczek
Copy link
Author

kloczek commented Apr 5, 2023

Please .. in output it is possible to find No module named 'sklearn._check_build'.
There is no such module in source tree.

@kloczek
Copy link
Author

kloczek commented Apr 5, 2023

It is module with two undersores in name but there is no with only one.

@kloczek
Copy link
Author

kloczek commented May 1, 2023

Gentle ping .. any update about fixing module name? 🤔
By that issu I'm not ble to build niaaml which is failing in pytest and shows warnings on rendering documetation

+ /usr/bin/sphinx-build -n -T -b man docs build/sphinx/man
Running Sphinx v6.2.1
making output directory... done
building [mo]: targets for 0 po files that are out of date
writing output...
building [man]: all manpages
updating environment: [new config] 15 added, 0 changed, 0 removed
reading sources... [100%] testing
WARNING: autodoc: failed to import module 'classifiers' from module 'niaaml'; the following exception was raised:
No module named 'sklearn._check_build'
___________________________________________________________________________
Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
__pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
___________________________________________________________________________
It seems that scikit-learn has not been built correctly.

If you have installed scikit-learn from source, please do not forget
to build the package before using it: run `python setup.py install` or
`make` in the source directory.

If you have used an installer, please check that it is suited for your
Python version, your operating system and your platform.
WARNING: autodoc: failed to import module 'data' from module 'niaaml'; the following exception was raised:
No module named 'sklearn._check_build'
___________________________________________________________________________
Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
__pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
___________________________________________________________________________
It seems that scikit-learn has not been built correctly.

If you have installed scikit-learn from source, please do not forget
to build the package before using it: run `python setup.py install` or
`make` in the source directory.

If you have used an installer, please check that it is suited for your
Python version, your operating system and your platform.
WARNING: autodoc: failed to import module 'fitness' from module 'niaaml'; the following exception was raised:
No module named 'sklearn._check_build'
___________________________________________________________________________
Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
__pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
___________________________________________________________________________
It seems that scikit-learn has not been built correctly.

If you have installed scikit-learn from source, please do not forget
to build the package before using it: run `python setup.py install` or
`make` in the source directory.

If you have used an installer, please check that it is suited for your
Python version, your operating system and your platform.
WARNING: autodoc: failed to import module 'niaaml'; the following exception was raised:
No module named 'sklearn._check_build'
___________________________________________________________________________
Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
__pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
___________________________________________________________________________
It seems that scikit-learn has not been built correctly.

If you have installed scikit-learn from source, please do not forget
to build the package before using it: run `python setup.py install` or
`make` in the source directory.

If you have used an installer, please check that it is suited for your
Python version, your operating system and your platform.
WARNING: autodoc: failed to import module 'preprocessing' from module 'niaaml'; the following exception was raised:
No module named 'sklearn._check_build'
___________________________________________________________________________
Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
__pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
___________________________________________________________________________
It seems that scikit-learn has not been built correctly.

If you have installed scikit-learn from source, please do not forget
to build the package before using it: run `python setup.py install` or
`make` in the source directory.

If you have used an installer, please check that it is suited for your
Python version, your operating system and your platform.
WARNING: autodoc: failed to import module 'preprocessing.feature_selection' from module 'niaaml'; the following exception was raised:
No module named 'sklearn._check_build'
___________________________________________________________________________
Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
__pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
___________________________________________________________________________
It seems that scikit-learn has not been built correctly.

If you have installed scikit-learn from source, please do not forget
to build the package before using it: run `python setup.py install` or
`make` in the source directory.

If you have used an installer, please check that it is suited for your
Python version, your operating system and your platform.
WARNING: autodoc: failed to import module 'preprocessing.feature_transform' from module 'niaaml'; the following exception was raised:
No module named 'sklearn._check_build'
___________________________________________________________________________
Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
__pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
___________________________________________________________________________
It seems that scikit-learn has not been built correctly.

If you have installed scikit-learn from source, please do not forget
to build the package before using it: run `python setup.py install` or
`make` in the source directory.

If you have used an installer, please check that it is suited for your
Python version, your operating system and your platform.
WARNING: autodoc: failed to import module 'preprocessing.encoding' from module 'niaaml'; the following exception was raised:
No module named 'sklearn._check_build'
___________________________________________________________________________
Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
__pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
___________________________________________________________________________
It seems that scikit-learn has not been built correctly.

If you have installed scikit-learn from source, please do not forget
to build the package before using it: run `python setup.py install` or
`make` in the source directory.

If you have used an installer, please check that it is suited for your
Python version, your operating system and your platform.
WARNING: autodoc: failed to import module 'preprocessing.imputation' from module 'niaaml'; the following exception was raised:
No module named 'sklearn._check_build'
___________________________________________________________________________
Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
__pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
___________________________________________________________________________
It seems that scikit-learn has not been built correctly.

If you have installed scikit-learn from source, please do not forget
to build the package before using it: run `python setup.py install` or
`make` in the source directory.

If you have used an installer, please check that it is suited for your
Python version, your operating system and your platform.
WARNING: autodoc: failed to import module 'niaaml'; the following exception was raised:
No module named 'sklearn._check_build'
___________________________________________________________________________
Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
__pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
___________________________________________________________________________
It seems that scikit-learn has not been built correctly.

If you have installed scikit-learn from source, please do not forget
to build the package before using it: run `python setup.py install` or
`make` in the source directory.

If you have used an installer, please check that it is suited for your
Python version, your operating system and your platform.
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
writing... python-niaaml.3 { getting_started changelog installation testing documentation api/index api/niaaml api/data api/classifiers api/preprocessing api/fitness about contributing code_of_conduct } /home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/docs/getting_started.rst:151: WARNING: py:class reference target not found: niaaml.preprocessing.feature_selection.SelectKBest
/home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/docs/api/niaaml.rst:1: WARNING: py:mod reference target not found: niaaml
/home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/docs/api/data.rst:1: WARNING: py:mod reference target not found: niaaml.data
/home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/docs/api/classifiers.rst:1: WARNING: py:mod reference target not found: niaaml.classifiers
/home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/docs/api/preprocessing.rst:1: WARNING: py:mod reference target not found: niaaml.preprocessing
/home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/docs/api/preprocessing.rst:9: WARNING: py:mod reference target not found: niaaml.preprocessing.feature_selection
/home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/docs/api/preprocessing.rst:17: WARNING: py:mod reference target not found: niaaml.preprocessing.feature_transform
/home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/docs/api/preprocessing.rst:25: WARNING: py:mod reference target not found: niaaml.preprocessing.encoding
/home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/docs/api/preprocessing.rst:33: WARNING: py:mod reference target not found: niaaml.preprocessing.imputation
/home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/docs/api/fitness.rst:1: WARNING: py:mod reference target not found: niaaml.fitness
done
build succeeded, 20 warnings.

and pytest

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-niaaml-1.1.12-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-niaaml-1.1.12-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -m 'not network'
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.8.16, pytest-7.3.1, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12
collected 0 items / 17 errors

========================================================================================== ERRORS ===========================================================================================
_____________________________________________________________________ ERROR collecting tests/test_basic_data_reader.py ______________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/tests/test_basic_data_reader.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:48: in <module>
    from sklearn._check_build import check_build  # noqa
E   ModuleNotFoundError: No module named 'sklearn._check_build'

During handling of the above exception, another exception occurred:
/usr/lib64/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_basic_data_reader.py:2: in <module>
    from niaaml.data import BasicDataReader
niaaml/__init__.py:1: in <module>
    from niaaml import classifiers
niaaml/classifiers/__init__.py:2: in <module>
    from niaaml.classifiers.random_forest import RandomForest
niaaml/classifiers/random_forest.py:2: in <module>
    from niaaml.utilities import MinMax
niaaml/utilities.py:1: in <module>
    from sklearn.metrics import accuracy_score, precision_score, cohen_kappa_score, f1_score
/usr/lib64/python3.8/site-packages/sklearn/__init__.py:81: in <module>
    from sklearn import __check_build  # noqa: F401
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:50: in <module>
    raise_build_error(e)
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:31: in raise_build_error
    raise ImportError(
E   ImportError: No module named 'sklearn._check_build'
E   ___________________________________________________________________________
E   Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
E   __pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
E   ___________________________________________________________________________
E   It seems that scikit-learn has not been built correctly.
E
E   If you have installed scikit-learn from source, please do not forget
E   to build the package before using it: run `python setup.py install` or
E   `make` in the source directory.
E
E   If you have used an installer, please check that it is suited for your
E   Python version, your operating system and your platform.
_____________________________________________________________________ ERROR collecting tests/test_classifier_factory.py _____________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/tests/test_classifier_factory.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:48: in <module>
    from sklearn._check_build import check_build  # noqa
E   ModuleNotFoundError: No module named 'sklearn._check_build'

During handling of the above exception, another exception occurred:
/usr/lib64/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_classifier_factory.py:2: in <module>
    from niaaml.classifiers import ClassifierFactory, Classifier
niaaml/__init__.py:1: in <module>
    from niaaml import classifiers
niaaml/classifiers/__init__.py:2: in <module>
    from niaaml.classifiers.random_forest import RandomForest
niaaml/classifiers/random_forest.py:2: in <module>
    from niaaml.utilities import MinMax
niaaml/utilities.py:1: in <module>
    from sklearn.metrics import accuracy_score, precision_score, cohen_kappa_score, f1_score
/usr/lib64/python3.8/site-packages/sklearn/__init__.py:81: in <module>
    from sklearn import __check_build  # noqa: F401
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:50: in <module>
    raise_build_error(e)
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:31: in raise_build_error
    raise ImportError(
E   ImportError: No module named 'sklearn._check_build'
E   ___________________________________________________________________________
E   Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
E   __pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
E   ___________________________________________________________________________
E   It seems that scikit-learn has not been built correctly.
E
E   If you have installed scikit-learn from source, please do not forget
E   to build the package before using it: run `python setup.py install` or
E   `make` in the source directory.
E
E   If you have used an installer, please check that it is suited for your
E   Python version, your operating system and your platform.
________________________________________________________________________ ERROR collecting tests/test_classifiers.py _________________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/tests/test_classifiers.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:48: in <module>
    from sklearn._check_build import check_build  # noqa
E   ModuleNotFoundError: No module named 'sklearn._check_build'

During handling of the above exception, another exception occurred:
/usr/lib64/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_classifiers.py:2: in <module>
    import niaaml.classifiers as c
niaaml/__init__.py:1: in <module>
    from niaaml import classifiers
niaaml/classifiers/__init__.py:2: in <module>
    from niaaml.classifiers.random_forest import RandomForest
niaaml/classifiers/random_forest.py:2: in <module>
    from niaaml.utilities import MinMax
niaaml/utilities.py:1: in <module>
    from sklearn.metrics import accuracy_score, precision_score, cohen_kappa_score, f1_score
/usr/lib64/python3.8/site-packages/sklearn/__init__.py:81: in <module>
    from sklearn import __check_build  # noqa: F401
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:50: in <module>
    raise_build_error(e)
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:31: in raise_build_error
    raise ImportError(
E   ImportError: No module named 'sklearn._check_build'
E   ___________________________________________________________________________
E   Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
E   __pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
E   ___________________________________________________________________________
E   It seems that scikit-learn has not been built correctly.
E
E   If you have installed scikit-learn from source, please do not forget
E   to build the package before using it: run `python setup.py install` or
E   `make` in the source directory.
E
E   If you have used an installer, please check that it is suited for your
E   Python version, your operating system and your platform.
______________________________________________________________________ ERROR collecting tests/test_csv_data_reader.py _______________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/tests/test_csv_data_reader.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:48: in <module>
    from sklearn._check_build import check_build  # noqa
E   ModuleNotFoundError: No module named 'sklearn._check_build'

During handling of the above exception, another exception occurred:
/usr/lib64/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_csv_data_reader.py:2: in <module>
    from niaaml.data import CSVDataReader
niaaml/__init__.py:1: in <module>
    from niaaml import classifiers
niaaml/classifiers/__init__.py:2: in <module>
    from niaaml.classifiers.random_forest import RandomForest
niaaml/classifiers/random_forest.py:2: in <module>
    from niaaml.utilities import MinMax
niaaml/utilities.py:1: in <module>
    from sklearn.metrics import accuracy_score, precision_score, cohen_kappa_score, f1_score
/usr/lib64/python3.8/site-packages/sklearn/__init__.py:81: in <module>
    from sklearn import __check_build  # noqa: F401
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:50: in <module>
    raise_build_error(e)
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:31: in raise_build_error
    raise ImportError(
E   ImportError: No module named 'sklearn._check_build'
E   ___________________________________________________________________________
E   Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
E   __pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
E   ___________________________________________________________________________
E   It seems that scikit-learn has not been built correctly.
E
E   If you have installed scikit-learn from source, please do not forget
E   to build the package before using it: run `python setup.py install` or
E   `make` in the source directory.
E
E   If you have used an installer, please check that it is suited for your
E   Python version, your operating system and your platform.
______________________________________________________________________ ERROR collecting tests/test_encoder_factory.py _______________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/tests/test_encoder_factory.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:48: in <module>
    from sklearn._check_build import check_build  # noqa
E   ModuleNotFoundError: No module named 'sklearn._check_build'

During handling of the above exception, another exception occurred:
/usr/lib64/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_encoder_factory.py:2: in <module>
    from niaaml.preprocessing.encoding import EncoderFactory, FeatureEncoder
niaaml/__init__.py:1: in <module>
    from niaaml import classifiers
niaaml/classifiers/__init__.py:2: in <module>
    from niaaml.classifiers.random_forest import RandomForest
niaaml/classifiers/random_forest.py:2: in <module>
    from niaaml.utilities import MinMax
niaaml/utilities.py:1: in <module>
    from sklearn.metrics import accuracy_score, precision_score, cohen_kappa_score, f1_score
/usr/lib64/python3.8/site-packages/sklearn/__init__.py:81: in <module>
    from sklearn import __check_build  # noqa: F401
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:50: in <module>
    raise_build_error(e)
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:31: in raise_build_error
    raise ImportError(
E   ImportError: No module named 'sklearn._check_build'
E   ___________________________________________________________________________
E   Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
E   __pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
E   ___________________________________________________________________________
E   It seems that scikit-learn has not been built correctly.
E
E   If you have installed scikit-learn from source, please do not forget
E   to build the package before using it: run `python setup.py install` or
E   `make` in the source directory.
E
E   If you have used an installer, please check that it is suited for your
E   Python version, your operating system and your platform.
______________________________________________________________________ ERROR collecting tests/test_feature_encoder.py _______________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/tests/test_feature_encoder.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:48: in <module>
    from sklearn._check_build import check_build  # noqa
E   ModuleNotFoundError: No module named 'sklearn._check_build'

During handling of the above exception, another exception occurred:
/usr/lib64/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_feature_encoder.py:2: in <module>
    from niaaml.preprocessing.encoding import OneHotEncoder, encode_categorical_features
niaaml/__init__.py:1: in <module>
    from niaaml import classifiers
niaaml/classifiers/__init__.py:2: in <module>
    from niaaml.classifiers.random_forest import RandomForest
niaaml/classifiers/random_forest.py:2: in <module>
    from niaaml.utilities import MinMax
niaaml/utilities.py:1: in <module>
    from sklearn.metrics import accuracy_score, precision_score, cohen_kappa_score, f1_score
/usr/lib64/python3.8/site-packages/sklearn/__init__.py:81: in <module>
    from sklearn import __check_build  # noqa: F401
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:50: in <module>
    raise_build_error(e)
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:31: in raise_build_error
    raise ImportError(
E   ImportError: No module named 'sklearn._check_build'
E   ___________________________________________________________________________
E   Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
E   __pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
E   ___________________________________________________________________________
E   It seems that scikit-learn has not been built correctly.
E
E   If you have installed scikit-learn from source, please do not forget
E   to build the package before using it: run `python setup.py install` or
E   `make` in the source directory.
E
E   If you have used an installer, please check that it is suited for your
E   Python version, your operating system and your platform.
_____________________________________________________________________ ERROR collecting tests/test_feature_selection.py ______________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/tests/test_feature_selection.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:48: in <module>
    from sklearn._check_build import check_build  # noqa
E   ModuleNotFoundError: No module named 'sklearn._check_build'

During handling of the above exception, another exception occurred:
/usr/lib64/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_feature_selection.py:2: in <module>
    import niaaml.preprocessing.feature_selection as fs
niaaml/__init__.py:1: in <module>
    from niaaml import classifiers
niaaml/classifiers/__init__.py:2: in <module>
    from niaaml.classifiers.random_forest import RandomForest
niaaml/classifiers/random_forest.py:2: in <module>
    from niaaml.utilities import MinMax
niaaml/utilities.py:1: in <module>
    from sklearn.metrics import accuracy_score, precision_score, cohen_kappa_score, f1_score
/usr/lib64/python3.8/site-packages/sklearn/__init__.py:81: in <module>
    from sklearn import __check_build  # noqa: F401
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:50: in <module>
    raise_build_error(e)
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:31: in raise_build_error
    raise ImportError(
E   ImportError: No module named 'sklearn._check_build'
E   ___________________________________________________________________________
E   Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
E   __pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
E   ___________________________________________________________________________
E   It seems that scikit-learn has not been built correctly.
E
E   If you have installed scikit-learn from source, please do not forget
E   to build the package before using it: run `python setup.py install` or
E   `make` in the source directory.
E
E   If you have used an installer, please check that it is suited for your
E   Python version, your operating system and your platform.
____________________________________________________________ ERROR collecting tests/test_feature_selection_algorithm_factory.py _____________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/tests/test_feature_selection_algorithm_factory.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:48: in <module>
    from sklearn._check_build import check_build  # noqa
E   ModuleNotFoundError: No module named 'sklearn._check_build'

During handling of the above exception, another exception occurred:
/usr/lib64/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_feature_selection_algorithm_factory.py:2: in <module>
    from niaaml.preprocessing.feature_selection import (
niaaml/__init__.py:1: in <module>
    from niaaml import classifiers
niaaml/classifiers/__init__.py:2: in <module>
    from niaaml.classifiers.random_forest import RandomForest
niaaml/classifiers/random_forest.py:2: in <module>
    from niaaml.utilities import MinMax
niaaml/utilities.py:1: in <module>
    from sklearn.metrics import accuracy_score, precision_score, cohen_kappa_score, f1_score
/usr/lib64/python3.8/site-packages/sklearn/__init__.py:81: in <module>
    from sklearn import __check_build  # noqa: F401
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:50: in <module>
    raise_build_error(e)
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:31: in raise_build_error
    raise ImportError(
E   ImportError: No module named 'sklearn._check_build'
E   ___________________________________________________________________________
E   Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
E   __pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
E   ___________________________________________________________________________
E   It seems that scikit-learn has not been built correctly.
E
E   If you have installed scikit-learn from source, please do not forget
E   to build the package before using it: run `python setup.py install` or
E   `make` in the source directory.
E
E   If you have used an installer, please check that it is suited for your
E   Python version, your operating system and your platform.
_____________________________________________________________________ ERROR collecting tests/test_feature_transform.py ______________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/tests/test_feature_transform.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:48: in <module>
    from sklearn._check_build import check_build  # noqa
E   ModuleNotFoundError: No module named 'sklearn._check_build'

During handling of the above exception, another exception occurred:
/usr/lib64/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_feature_transform.py:2: in <module>
    import niaaml.preprocessing.feature_transform as ft
niaaml/__init__.py:1: in <module>
    from niaaml import classifiers
niaaml/classifiers/__init__.py:2: in <module>
    from niaaml.classifiers.random_forest import RandomForest
niaaml/classifiers/random_forest.py:2: in <module>
    from niaaml.utilities import MinMax
niaaml/utilities.py:1: in <module>
    from sklearn.metrics import accuracy_score, precision_score, cohen_kappa_score, f1_score
/usr/lib64/python3.8/site-packages/sklearn/__init__.py:81: in <module>
    from sklearn import __check_build  # noqa: F401
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:50: in <module>
    raise_build_error(e)
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:31: in raise_build_error
    raise ImportError(
E   ImportError: No module named 'sklearn._check_build'
E   ___________________________________________________________________________
E   Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
E   __pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
E   ___________________________________________________________________________
E   It seems that scikit-learn has not been built correctly.
E
E   If you have installed scikit-learn from source, please do not forget
E   to build the package before using it: run `python setup.py install` or
E   `make` in the source directory.
E
E   If you have used an installer, please check that it is suited for your
E   Python version, your operating system and your platform.
____________________________________________________________ ERROR collecting tests/test_feature_transform_algorithm_factory.py _____________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/tests/test_feature_transform_algorithm_factory.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:48: in <module>
    from sklearn._check_build import check_build  # noqa
E   ModuleNotFoundError: No module named 'sklearn._check_build'

During handling of the above exception, another exception occurred:
/usr/lib64/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_feature_transform_algorithm_factory.py:2: in <module>
    from niaaml.preprocessing.feature_transform import (
niaaml/__init__.py:1: in <module>
    from niaaml import classifiers
niaaml/classifiers/__init__.py:2: in <module>
    from niaaml.classifiers.random_forest import RandomForest
niaaml/classifiers/random_forest.py:2: in <module>
    from niaaml.utilities import MinMax
niaaml/utilities.py:1: in <module>
    from sklearn.metrics import accuracy_score, precision_score, cohen_kappa_score, f1_score
/usr/lib64/python3.8/site-packages/sklearn/__init__.py:81: in <module>
    from sklearn import __check_build  # noqa: F401
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:50: in <module>
    raise_build_error(e)
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:31: in raise_build_error
    raise ImportError(
E   ImportError: No module named 'sklearn._check_build'
E   ___________________________________________________________________________
E   Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
E   __pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
E   ___________________________________________________________________________
E   It seems that scikit-learn has not been built correctly.
E
E   If you have installed scikit-learn from source, please do not forget
E   to build the package before using it: run `python setup.py install` or
E   `make` in the source directory.
E
E   If you have used an installer, please check that it is suited for your
E   Python version, your operating system and your platform.
__________________________________________________________________________ ERROR collecting tests/test_fitness.py ___________________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/tests/test_fitness.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:48: in <module>
    from sklearn._check_build import check_build  # noqa
E   ModuleNotFoundError: No module named 'sklearn._check_build'

During handling of the above exception, another exception occurred:
/usr/lib64/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_fitness.py:3: in <module>
    import niaaml.fitness as f
niaaml/__init__.py:1: in <module>
    from niaaml import classifiers
niaaml/classifiers/__init__.py:2: in <module>
    from niaaml.classifiers.random_forest import RandomForest
niaaml/classifiers/random_forest.py:2: in <module>
    from niaaml.utilities import MinMax
niaaml/utilities.py:1: in <module>
    from sklearn.metrics import accuracy_score, precision_score, cohen_kappa_score, f1_score
/usr/lib64/python3.8/site-packages/sklearn/__init__.py:81: in <module>
    from sklearn import __check_build  # noqa: F401
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:50: in <module>
    raise_build_error(e)
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:31: in raise_build_error
    raise ImportError(
E   ImportError: No module named 'sklearn._check_build'
E   ___________________________________________________________________________
E   Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
E   __pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
E   ___________________________________________________________________________
E   It seems that scikit-learn has not been built correctly.
E
E   If you have installed scikit-learn from source, please do not forget
E   to build the package before using it: run `python setup.py install` or
E   `make` in the source directory.
E
E   If you have used an installer, please check that it is suited for your
E   Python version, your operating system and your platform.
______________________________________________________________________ ERROR collecting tests/test_fitness_factory.py _______________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/tests/test_fitness_factory.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:48: in <module>
    from sklearn._check_build import check_build  # noqa
E   ModuleNotFoundError: No module named 'sklearn._check_build'

During handling of the above exception, another exception occurred:
/usr/lib64/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_fitness_factory.py:2: in <module>
    from niaaml.fitness import FitnessFactory, FitnessFunction
niaaml/__init__.py:1: in <module>
    from niaaml import classifiers
niaaml/classifiers/__init__.py:2: in <module>
    from niaaml.classifiers.random_forest import RandomForest
niaaml/classifiers/random_forest.py:2: in <module>
    from niaaml.utilities import MinMax
niaaml/utilities.py:1: in <module>
    from sklearn.metrics import accuracy_score, precision_score, cohen_kappa_score, f1_score
/usr/lib64/python3.8/site-packages/sklearn/__init__.py:81: in <module>
    from sklearn import __check_build  # noqa: F401
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:50: in <module>
    raise_build_error(e)
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:31: in raise_build_error
    raise ImportError(
E   ImportError: No module named 'sklearn._check_build'
E   ___________________________________________________________________________
E   Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
E   __pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
E   ___________________________________________________________________________
E   It seems that scikit-learn has not been built correctly.
E
E   If you have installed scikit-learn from source, please do not forget
E   to build the package before using it: run `python setup.py install` or
E   `make` in the source directory.
E
E   If you have used an installer, please check that it is suited for your
E   Python version, your operating system and your platform.
__________________________________________________________________________ ERROR collecting tests/test_imputer.py ___________________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/tests/test_imputer.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:48: in <module>
    from sklearn._check_build import check_build  # noqa
E   ModuleNotFoundError: No module named 'sklearn._check_build'

During handling of the above exception, another exception occurred:
/usr/lib64/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_imputer.py:2: in <module>
    from niaaml.preprocessing.imputation import SimpleImputer, impute_features
niaaml/__init__.py:1: in <module>
    from niaaml import classifiers
niaaml/classifiers/__init__.py:2: in <module>
    from niaaml.classifiers.random_forest import RandomForest
niaaml/classifiers/random_forest.py:2: in <module>
    from niaaml.utilities import MinMax
niaaml/utilities.py:1: in <module>
    from sklearn.metrics import accuracy_score, precision_score, cohen_kappa_score, f1_score
/usr/lib64/python3.8/site-packages/sklearn/__init__.py:81: in <module>
    from sklearn import __check_build  # noqa: F401
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:50: in <module>
    raise_build_error(e)
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:31: in raise_build_error
    raise ImportError(
E   ImportError: No module named 'sklearn._check_build'
E   ___________________________________________________________________________
E   Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
E   __pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
E   ___________________________________________________________________________
E   It seems that scikit-learn has not been built correctly.
E
E   If you have installed scikit-learn from source, please do not forget
E   to build the package before using it: run `python setup.py install` or
E   `make` in the source directory.
E
E   If you have used an installer, please check that it is suited for your
E   Python version, your operating system and your platform.
______________________________________________________________________ ERROR collecting tests/test_imputer_factory.py _______________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/tests/test_imputer_factory.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:48: in <module>
    from sklearn._check_build import check_build  # noqa
E   ModuleNotFoundError: No module named 'sklearn._check_build'

During handling of the above exception, another exception occurred:
/usr/lib64/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_imputer_factory.py:2: in <module>
    from niaaml.preprocessing.imputation import Imputer, ImputerFactory
niaaml/__init__.py:1: in <module>
    from niaaml import classifiers
niaaml/classifiers/__init__.py:2: in <module>
    from niaaml.classifiers.random_forest import RandomForest
niaaml/classifiers/random_forest.py:2: in <module>
    from niaaml.utilities import MinMax
niaaml/utilities.py:1: in <module>
    from sklearn.metrics import accuracy_score, precision_score, cohen_kappa_score, f1_score
/usr/lib64/python3.8/site-packages/sklearn/__init__.py:81: in <module>
    from sklearn import __check_build  # noqa: F401
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:50: in <module>
    raise_build_error(e)
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:31: in raise_build_error
    raise ImportError(
E   ImportError: No module named 'sklearn._check_build'
E   ___________________________________________________________________________
E   Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
E   __pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
E   ___________________________________________________________________________
E   It seems that scikit-learn has not been built correctly.
E
E   If you have installed scikit-learn from source, please do not forget
E   to build the package before using it: run `python setup.py install` or
E   `make` in the source directory.
E
E   If you have used an installer, please check that it is suited for your
E   Python version, your operating system and your platform.
__________________________________________________________________________ ERROR collecting tests/test_pipeline.py __________________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/tests/test_pipeline.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:48: in <module>
    from sklearn._check_build import check_build  # noqa
E   ModuleNotFoundError: No module named 'sklearn._check_build'

During handling of the above exception, another exception occurred:
/usr/lib64/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_pipeline.py:2: in <module>
    from niaaml import Pipeline, OptimizationStats
niaaml/__init__.py:1: in <module>
    from niaaml import classifiers
niaaml/classifiers/__init__.py:2: in <module>
    from niaaml.classifiers.random_forest import RandomForest
niaaml/classifiers/random_forest.py:2: in <module>
    from niaaml.utilities import MinMax
niaaml/utilities.py:1: in <module>
    from sklearn.metrics import accuracy_score, precision_score, cohen_kappa_score, f1_score
/usr/lib64/python3.8/site-packages/sklearn/__init__.py:81: in <module>
    from sklearn import __check_build  # noqa: F401
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:50: in <module>
    raise_build_error(e)
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:31: in raise_build_error
    raise ImportError(
E   ImportError: No module named 'sklearn._check_build'
E   ___________________________________________________________________________
E   Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
E   __pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
E   ___________________________________________________________________________
E   It seems that scikit-learn has not been built correctly.
E
E   If you have installed scikit-learn from source, please do not forget
E   to build the package before using it: run `python setup.py install` or
E   `make` in the source directory.
E
E   If you have used an installer, please check that it is suited for your
E   Python version, your operating system and your platform.
_____________________________________________________________________ ERROR collecting tests/test_pipeline_optimizer.py _____________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/tests/test_pipeline_optimizer.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:48: in <module>
    from sklearn._check_build import check_build  # noqa
E   ModuleNotFoundError: No module named 'sklearn._check_build'

During handling of the above exception, another exception occurred:
/usr/lib64/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_pipeline_optimizer.py:2: in <module>
    from niaaml import PipelineOptimizer, Pipeline
niaaml/__init__.py:1: in <module>
    from niaaml import classifiers
niaaml/classifiers/__init__.py:2: in <module>
    from niaaml.classifiers.random_forest import RandomForest
niaaml/classifiers/random_forest.py:2: in <module>
    from niaaml.utilities import MinMax
niaaml/utilities.py:1: in <module>
    from sklearn.metrics import accuracy_score, precision_score, cohen_kappa_score, f1_score
/usr/lib64/python3.8/site-packages/sklearn/__init__.py:81: in <module>
    from sklearn import __check_build  # noqa: F401
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:50: in <module>
    raise_build_error(e)
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:31: in raise_build_error
    raise ImportError(
E   ImportError: No module named 'sklearn._check_build'
E   ___________________________________________________________________________
E   Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
E   __pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
E   ___________________________________________________________________________
E   It seems that scikit-learn has not been built correctly.
E
E   If you have installed scikit-learn from source, please do not forget
E   to build the package before using it: run `python setup.py install` or
E   `make` in the source directory.
E
E   If you have used an installer, please check that it is suited for your
E   Python version, your operating system and your platform.
_________________________________________________________________________ ERROR collecting tests/test_utilities.py __________________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/NiaAML-1.1.12/tests/test_utilities.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:48: in <module>
    from sklearn._check_build import check_build  # noqa
E   ModuleNotFoundError: No module named 'sklearn._check_build'

During handling of the above exception, another exception occurred:
/usr/lib64/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_utilities.py:2: in <module>
    from niaaml import ParameterDefinition, MinMax, OptimizationStats, get_bin_index
niaaml/__init__.py:1: in <module>
    from niaaml import classifiers
niaaml/classifiers/__init__.py:2: in <module>
    from niaaml.classifiers.random_forest import RandomForest
niaaml/classifiers/random_forest.py:2: in <module>
    from niaaml.utilities import MinMax
niaaml/utilities.py:1: in <module>
    from sklearn.metrics import accuracy_score, precision_score, cohen_kappa_score, f1_score
/usr/lib64/python3.8/site-packages/sklearn/__init__.py:81: in <module>
    from sklearn import __check_build  # noqa: F401
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:50: in <module>
    raise_build_error(e)
/usr/lib64/python3.8/site-packages/sklearn/__check_build/__init__.py:31: in raise_build_error
    raise ImportError(
E   ImportError: No module named 'sklearn._check_build'
E   ___________________________________________________________________________
E   Contents of /usr/lib64/python3.8/site-packages/sklearn/__check_build:
E   __pycache__               __init__.py               _check_build.cpython-38-x86_64-linux-gnu.so
E   ___________________________________________________________________________
E   It seems that scikit-learn has not been built correctly.
E
E   If you have installed scikit-learn from source, please do not forget
E   to build the package before using it: run `python setup.py install` or
E   `make` in the source directory.
E
E   If you have used an installer, please check that it is suited for your
E   Python version, your operating system and your platform.
================================================================================== short test summary info ==================================================================================
ERROR tests/test_basic_data_reader.py
ERROR tests/test_classifier_factory.py
ERROR tests/test_classifiers.py
ERROR tests/test_csv_data_reader.py
ERROR tests/test_encoder_factory.py
ERROR tests/test_feature_encoder.py
ERROR tests/test_feature_selection.py
ERROR tests/test_feature_selection_algorithm_factory.py
ERROR tests/test_feature_transform.py
ERROR tests/test_feature_transform_algorithm_factory.py
ERROR tests/test_fitness.py
ERROR tests/test_fitness_factory.py
ERROR tests/test_imputer.py
ERROR tests/test_imputer_factory.py
ERROR tests/test_pipeline.py
ERROR tests/test_pipeline_optimizer.py
ERROR tests/test_utilities.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 17 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
==================================================================================== 17 errors in 0.47s =====================================================================================

@kloczek
Copy link
Author

kloczek commented Jul 7, 2023

I was able to build documentation for 1.2.2 with patch which removes all relative imports and perform build procedure like below:

/usr/bin/python3 -sBm build -w --no-isolation
PYTHONPATH=$PWD/build/$(cd build; ls -1d lib*) \
/usr/bin/sphinx-build -n -T -b man doc build/sphinx/man

However with 1.3.0 that patch which was +170KB needs to be rediffed 😞

Remove all relative imports allow take .py files from sklearn/ and DSOs from $PYTHONPATH.
Using relative imports forces to have all DSOs in the sklearn/.

The same issue is with test suite.
In this case additionally all test suite bits are spread under subdirectories of sklearn/*/.
IMO it would be good to move all test suite files under tests/ however before that all relative imports needs to be removed and replaced by full name imports.
Currently because all test suite parts are spread across sklearn/ and they are not excluded from install all those files ae not excluded in pyproject.tom.

If you want I cen submit PR which will replace relative imports bu full name imports.

One more time. It would be rally good to:

  • remove use relative imports
  • separate test suite into tests/

As side effect it will solve building documentation without have sklearn module installed and as well will make whole code way easier to move between directories.

@thomasjpfan
Copy link
Member

Could you explain a bit more what the goal is of being able to build the documentation after building the package but without installing it? What is the use-case where this is needed? What does it allow the user to do that is otherwise not possible?

We need an answer to this question before considering a PR.

From what I've seen, numerical Python libraries, such as NumPy's docs and SciPy's docs, require the library to be built and installed before building the documentation.

As it stands, I am -1 on enabling the documentation to built without installing scikit-learn.

@thomasjpfan thomasjpfan added Documentation Needs Decision - Include Feature Requires decision regarding including feature labels Jul 8, 2023
@kloczek
Copy link
Author

kloczek commented Jul 8, 2023

From what I've seen, numerical Python libraries, such as NumPy's docs and SciPy's docs, require the library to be built and installed before building the documentation.

You are 100% right in case of numpy .. but it is only part of the whole picture.
To build numpy documentation you need to have installed numpydoc. That module requires mathplot and that module requires numpy.
In other words if you want ot have consistent build env to build numpy documentation you must have installed some version of the numpy (not exactly the same which is build).
However .. to prevent build that documentation against installed numpy (not that one which you have currently in source tree)
it needs to be added PYTHONPATH=$PWD/build/$(cd build; ls -d1 lib*) to sphinx-build execution env to hint python to take that numpy version first from that location.
Just checked my python-numpy.spec file and here what I have in that file

BuildRequires:  flexiblas-devel
BuildRequires:  gcc
BuildRequires:  gcc-c++
BuildRequires:  gcc-gfortran
BuildRequires:  lapack-devel
BuildRequires:  python3-devel
BuildRequires:  python3dist(build)
BuildRequires:  python3dist(cython)     >= 0.29.24
BuildRequires:  python3dist(installer)
BuildRequires:  python3dist(ipython)
BuildRequires:  python3dist(setuptools) >= 59.2
BuildRequires:  python3dist(matplotlib)
%{?with_loopy_deps:
BuildRequires:  python3dist(numpydoc)
BuildRequires:  python3dist(scipy)
BuildRequires:  python3dist(pydata-sphinx-theme)
BuildRequires:  python3dist(sphinx)
BuildRequires:  python3dist(sphinx-design)
BuildRequires:  python3dist(sphinxcontrib-restbuilder)
BuildRequires:  python3dist(sphinx-panels)}
BuildRequires:  python3dist(wheel)

[..]

%build
%{set_build_flags} \
BLAS=%{_libdir} \
LAPACK=%{_libdir} \
OPENBLAS=%{_libdir} \
SETUPTOOLS_USE_DISTUTILS=stdlib \
%pyproject_wheel
%{?with_loopy_deps:
PYTHONPATH=$PWD/build/$(cd build; ls -d1 lib*) \
%sphinx_build_man doc/source}

To boostrap numpy as you see I have special build mode which (without loopy_deps) which allows skip building documentation. That boostraping affects as wll list of BuildRequires.

On building scipy none of that module build dependencies requires scipy so in this case you don't need to have installed scipy to build documentation and that trick with:

PYTHONPATH=$PWD/build/$(cd build; ls -d1 lib*) \
%sphinx_build_man doc/source}

it is all what is necessary to build that module documentation.
Additionally there is no any problems with loading DSOs of that module from $PYTHONPATH because ..

[tkloczko@pers-jacek SPECS]$ grep -r "from  \."; grep -r "import \."| grep -v /doc/
[tkloczko@pers-jacek SPECS]$
[tkloczko@pers-jacek scipy-1.11.1]$ grep -r "from  \."; grep -r "import \."| grep -v /doc/
doc/source/release/1.5.0-notes.rst:* `#11623 <https://github.com/scipy/scipy/pull/11623>`__: MAINT: Remove unnecessary 'from __future__ import ...' statements
doc/source/release/1.5.0-notes.rst:* `#11646 <https://github.com/scipy/scipy/pull/11646>`__: MAINT: Remove unnecessary 'from __future__ import ...' statements
scipy/_lib/boost_math/build/Jamfile.v2:import ../../config/checks/config : requires ;
scipy/_lib/boost_math/example/Jamfile.v2:import ../../config/checks/config : requires ;
scipy/_lib/boost_math/reporting/accuracy/Jamfile.v2:import ../../../config/checks/config : requires ;
scipy/_lib/boost_math/reporting/performance/Jamfile.v2:import ../../../config/checks/config : requires ;
scipy/_lib/boost_math/reporting/performance/Jamfile.v2:import ../../../predef/check/predef
scipy/_lib/boost_math/test/Jamfile.v2:import ../../config/checks/config : requires ;
scipy/_lib/boost_math/tools/Jamfile.v2:import ../../config/checks/config : requires ;
scipy/fftpack/tests/test_import.py:the form `from scipy.fftpack import ...`, so that a simple replacement
scipy/fftpack/tests/test_import.py:        regexp = r"\s*from.+\.fftpack import .*\n"
scipy/linalg/_generate_pyx.py:# cimport ..linalg.cython_blas as cython_blas
scipy/linalg/_generate_pyx.py:# cimport ..linalg.cython_lapack as cython_lapack
scipy/linalg/_generate_pyx.py:# cimport ..linalg.cython_blas as cython_blas
scipy/linalg/_generate_pyx.py:# cimport ..linalg.cython_lapack as cython_lapack
scipy/linalg/_matfuncs.py:    # Avoid circular import ... this is OK, right?

If you will look on above closer you will find that actually there is no here relative imports.

From that ponit of view scikit-learn has no properties like numpy and its documentation can be build without have that module installed.

It is yet another aspect of using relative imports.
Theoretically in case import .foo it replaces import sklearn.foo which is 7 bytes shorter in source code however in this case it must be made another call to python routine which takes current directory (and additional OS syscall to take that path) and that string needs to be concatenated with foo to produce from path from which module or its code (.pyc/.pyo files) needs to be loaded.
I'm sure that it causes that actual JIT code is by this longer and additionally it causes additional syscall to take current directory path.
In other words using relative imports is only apparently better.

I have already packaged

[tkloczko@pers-jacek SPECS]$ ls -l python-*spec|wc -l
1170

python modules as rpm packages. I can tell you that using in the code relative imports was one of the top 5 cause of some issues found on packaging especially on testing stage in case of modules with DSOs.
This is why IMO using relative imports saves nothing and is nothing more than kind of just asking for troubles 😋

@betatim
Copy link
Member

betatim commented Jul 10, 2023

There isn't any new information on why this would be worth doing. So I propose we close this issue. New topics/questions are best discussed in a new issue.

@kloczek
Copy link
Author

kloczek commented Jul 10, 2023

So you don;t want to support "test as installed" methodology which is now widely used in Fedora and Debian? 🤔
Saving time is not the argument?
Saving JIT code size is not the argument? 🤔

@betatim
Copy link
Member

betatim commented Jul 11, 2023

I don't know if it is worth making a change to support these things or not. This is because:

Could you explain a bit more what the goal is of being able to build the documentation after building the package but without installing it? What is the use-case where this is needed? What does it allow the user to do that is otherwise not possible?

We need an answer to this question before considering a PR.

I still can't find a clear answer to this question. Without knowing why you want to do what you are trying to do (build the docs without installing scikit-learn, at least I assume this is what you are trying to do) it is impossible to make a judgement on whether this is something we want to support and how much effort it is possible to invest in it.

As far as I can tell you are asking for at least two changes: (1) moving all the tests to a new location and (2) not using relative imports in the code base. These are two very big asks that would require a lot of effort to implement and potentially quite disruptive. This means without a good understanding of why someone might desire this, and how many others there are like them it is impossible to make a decision.

@kloczek
Copy link
Author

kloczek commented Jul 11, 2023

I still can't find a clear answer to this question. Without knowing why you want to do what you are trying to do (build the docs without installing scikit-learn, at least I assume this is what you are trying to do) it is impossible to make a judgement on whether this is something we want to support and how much effort it is possible to invest in it.

Answer is very simple .. I'm packaging your module as rpm package.
In such case build process is done from non-root account and it is not possible to install module.
More .. from packaging process point of view installing is completely not necessary process.

As far as I can tell you are asking for at least two changes: (1) moving all the tests to a new location and (2) not using relative imports in the code base. These are two very big asks that would require a lot of effort to implement and potentially quite disruptive.

If you will look closer on spreaded in different locations tests/ subdirectories you will find that NONE of the code in that directories uses relative imports -> code in those directories can be freely moved around without additional changes.
The same property will have actual module code after remove relative imports.

@betatim
Copy link
Member

betatim commented Jul 12, 2023

Thanks for explaining. Let's see what others from @scikit-learn/core-devs think about moving tests and not using relative imports to make it easier to package scikit-learn as a RPM. Please correct me if I am wrong regarding what you are asking for.

@kloczek
Copy link
Author

kloczek commented Jul 12, 2023

Thanks for explaining. Let's see what others from @scikit-learn/core-devs think about moving tests and not using relative imports to make it easier to package scikit-learn as a RPM. Please correct me if I am wrong regarding what you are asking for.

In case for example distributions like Debian or Fedora testing and produce documentation in some unified form still is not obligatory.
In case of distribution on which I'm working:

[tkloczko@pers-jacek SPECS]$ ls -1 python-*.spec | wc -l; grep ^%pytest python-*.spec | wc -l; grep %sphinx_build_man python-*.spec | wc -l
1171
1154
603

Which means that In 1171 packaged python modules 1154 are using pytest and 603 produces and packages sphinx based documentation as man page.
Result (from my current devel env):

[tkloczko@pers-jacek SPECS]$ man python-<tab><tab>
python-asttokens                  python-jedi                       python-pyproject-hooks
python-babel                      python-jeepney                    python-pytest
python-backcall                   python-jinja2                     python-pytest-mock
python-build                      python-jsonschema                 python-rapidfuzz
python-cachecontrol               python-jsonschema-specifications  python-referencing
python-cffi                       python-keyring                    python-requests
python-charset-normalizer         python-markupsafe                 python-requests_toolbelt
python-cleo                       python-more-itertools             python-secretstorage
python-dateutil                   python-msgpack                    python-setuptools
python-decorator                  python-numpy                      python-six
python-distlib                    python-parso                      python-sphinx-theme-alabaster
python-dulwich                    python-pdfminer                   python-tomlkit
python-filelock                   python-pkginfo                    python-traitlets
python-html5lib                   python-platformdirs               python-urllib3
python-importlib-metadata         python-pluggy                     python-wcwidth
python-importlib_resources        python-prompt_toolkit             python-webencodings
python-installer                  python-ptyprocess                 python-wheel
python-ipython                    python-pygments                   python-zipp
python-jaraco-classes             python-pypdf

@adrinjalali
Copy link
Member

Reading the thread, I'll try to summarize points which I think are relevant to us

Packaging

It is possible to package the library w/o any changes, you just need to change the way you build and create the packages, e.g. create a separate env for each package build. So this is not a motivation for a change. Other distros already package scikit-learn.

Relative Imports

Some interesting points are raised in #25766 (comment) regarding extra syscalls. I'm not sure if they're true, I'd need to check with how the module finder works exactly.

I know many other libraries use absolute imports, but I find it confusing, as the imports will end up importing from a different location if you're not careful about where you start your python process. Apparently this is not an issue for those packages.

I might be convinced with the right argument to move to absolute imports (I do find them cleaner to read at least).

Moving Tests

Moving tests outside of the source tree might have marginal benefits like not including them in our distributions and have a cleaner source tree. But for any project the size of our project these things are not trivial to do since they introduce a lot of friction for a while. I personally don't have a strong preference on whether we should move them or not.

Misc

In such case build process is done from non-root account and it is not possible to install module.

One can always create an environment and install there, so being a non-root user is a non-issue in python. In fact, I personally strongly discourage people from installing any of the packages in the ecosystem as root anyway. They really SHOULD have a different environment for each project they have.

If you will look closer on spreaded in different locations tests/ subdirectories you will find that NONE of the code in that directories uses relative imports -> code in those directories can be freely moved around without additional changes.
The same property will have actual module code after remove relative imports.

In rare occasions we import from those tests in other tests, so that'd need to be changed, but not a big deal I'd say. I personally don't mind it either way, except that packaging w/o the tests included would be easier if we move them somewhere else.

@kloczek
Copy link
Author

kloczek commented Jul 12, 2023

LGTM 👍

I can only add few additional pieces to that picture that python modules which are building DSOs (Dynamic Shared Objects) are kind of special cases. Only way to build module and documentation during which sphinx will be loading module DSOs is to use setyp.py build --inplace.
Use setup.py currently is deprecated and it still does not provide proper solution for testing as long as pytest is not called as python -m pytest (in case use python -m foo interpreter automatically adds current directory to sys.path).
Additionally use python -m pytest is known that it should not be used that way (there is well known list of possible side effects of use pytest that way). This is why pytest module provides pytest wrapper script.
In above scenario only solution is to have tested module with its DSO installed in python tree which is not possible to have during typical non-root packaging process.

PS. If you don't have time to prepare patch which will remove replace use imports I can submit PR for that 😋

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Documentation Needs Decision - Include Feature Requires decision regarding including feature
Projects
None yet
Development

No branches or pull requests

6 participants