Cannot run tests when installed from PyPI

[davidbrochart]

Description

pip install jupyter_server[test] doesn't allow to run the tests.

Reproduce

1. mamba create -n jupyter_server jupyter_server
2. conda activate jupyter_server
3. pytest --pyargs jupyter_server
4. Tests are failing because the fixtures cannot be found.

conftest.py is at the root of the repository and is not included in the package.

Expected behavior

The tests are part of the package, so we should be able to run them.

Context

• Operating System and version: Ubuntu 21.04
• Browser and version: N/A
• Jupyter Server version: 1.10.1

[oliver-sanders]

@davidbrochart

conftest.py is at the root of the repository and is not included in the package.

The required pytest plugin is now defined in a top-level contest file:

jupyter_server/conftest.py

Lines 4 to 6 in 52e9467

	pytest_plugins = [
	"jupyter_server.pytest_plugin"
	]

Due to a deprecation in pytest which requires plugins to be defined in a top-level conftest file:

https://docs.pytest.org/en/6.2.x/deprecations.html#pytest-plugins-in-non-top-level-conftest-files

• We might be able to set the required plugins in a conftest or on the CLI in the downstream tests?
• We might be able to reroute the examples/ dir (contains a conftest file) inside jupyter_server/, then try to get pytest to accept the jupyter_server/ dir as the top-level dir so that we could package the conftest file that includes the plugin definition?
  • I'm not sure if the existence of files like pyproject.toml at the root-level of the project would prohibit this?
  • https://docs.pytest.org/en/6.2.x/customize.html#initialization-determining-rootdir-and-configfile

[Zsailer]

My suggestion is that we move the conftest.py into the jupyter_server/tests location and we require that tests to be invoked using pytest jupyter_server from the root directory. This means we need to add a conftest.py to the examples/ directory, too, and invoke those tests with pytest examples. That way, we can run the tests from source and the PyPI package in a similar way, pytest --pyargs jupyter_server.

The only downside, of course, is that the tests will raise an error when run from source using just pytest. I don't think this is a big deal. We can document this in our CONTRIBUTING docs.

I don't think we should move the examples folder into the package—this would make the examples less discoverable. They are really only useful for people looking at the source code in the Github repo. If they aren't at the top level, I don't think anyone will see them.