Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Explicitly install nose to unbreak tests #64

Merged
merged 1 commit into from
Jan 15, 2022

Conversation

mbrukman
Copy link
Collaborator

The command nosetests appears to have disappeared from the GitHub Actions
images for Python 3.8 and 3.9 on all versions of Ubuntu and macOS, so let's try
installing it manually to fix the tests in the short-term.

Closes #63

@mbrukman
Copy link
Collaborator Author

Strange, now we're getting yet a different error on both Python 3.8 and 3.9 (3.7 seems to be fine):

Run nosetests
E
======================================================================
ERROR: Failure: ModuleNotFoundError (No module named 'IPython.config')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/nose/failure.py", line 39, in runTest
    raise self.exc_val.with_traceback(self.tb)
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/nose/loader.py", line 417, in loadTestsFromName
    module = self.importer.importFromPath(
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/imp.py", line 234, in load_module
    return load_source(name, filename, file)
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/imp.py", line 171, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 702, in _load
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/runner/work/ipycache/ipycache/test_ipycache.py", line 20, in <module>
    from ipycache import (save_vars, load_vars, clean_var, clean_vars, do_save,
  File "/home/runner/work/ipycache/ipycache/ipycache.py", line 14, in <module>
    from IPython.config.configurable import Configurable
ModuleNotFoundError: No module named 'IPython.config'

----------------------------------------------------------------------
Ran 1 test in 0.952s

FAILED (errors=1)
Error: Process completed with exit code 1.

@mbrukman
Copy link
Collaborator Author

Looks like this is an issue with recently-released IPython 8.0.0 (https://pypi.org/project/ipython/#history shows it was released on Jan 12, 2022) — since we specify ipython[all] in requirements.txt, we're getting the latest-available and compatible IPython library.

Looks like IPython 8.0.0 is not compatible with Python 3.7, so there we get an earlier version:

Run pip install -r requirements.txt
Collecting ipython[all]
  Downloading ipython-7.31.0-py3-none-any.whl (792 kB)
[...]

whereas for Python 3.8 and 3.9, we're getting IPython 8.0:

Run pip install -r requirements.txt
Collecting ipython[all]
  Downloading ipython-8.0.0-py3-none-any.whl (747 kB)
[...]

Thus, the solution here is to update requirements.txt and explicitly specify that ipycache is compatible with IPython prior to 8.0.0.

mbrukman added a commit to mbrukman/ipycache that referenced this pull request Jan 14, 2022
The command `nosetests` appears to have disappeared from the GitHub Actions
images for Python 3.8 and 3.9 on all versions of Ubuntu and macOS, so let's try
installing it manually to fix the tests in the short-term.

During the development of [this change][1], we've learned that ipycache is not
compatible with recently-released IPython 8.0.0, so we've updated
`requirements.txt` to require a version of IPython prior to 8.0.0; additionally,
we re-enabled testing with Python 2.7 since we'll need to update our code in the
future to be compatible with `nose2`, so until we drop support for Python 2.7,
we should be testing with that version as well.

Closes rossant#63

[1]: rossant#64
@mbrukman
Copy link
Collaborator Author

The new error is:

Run nosetests
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/IPython/config.py:13: ShimWarning: The `IPython.config` package has been deprecated since IPython 4.0. You should import from traitlets.config instead.
  "You should import from traitlets.config instead.", ShimWarning)
/home/runner/work/ipycache/ipycache/ipycache.py:17: UserWarning: IPython.utils.traitlets has moved to a top-level traitlets package.
  from IPython.utils.traitlets import Unicode
..........
----------------------------------------------------------------------
Ran 10 tests in 0.617s

OK
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/setuptools/dist.py:454: UserWarning: Normalizing '0.1.5dev' to '0.1.5.dev0'
running check
  warnings.warn(tmpl.format(**locals()))
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/IPython/nbformat.py:13: ShimWarning: The `IPython.nbformat` package has been deprecated since IPython 4.0. You should import from nbformat instead.
  "You should import from nbformat instead.", ShimWarning)
Traceback (most recent call last):
  File "ipynb_runner.py", line 5, in <module>
    from IPython.nbformat.current import read
ModuleNotFoundError: No module named 'IPython.nbformat.current'
Error: Process completed with exit code 1.

An earlier test run included the following helpful guidance:

- use nbformat for read/write/validate public API
- use nbformat.vX directly to composing notebooks of a particular version

and https://nbformat.readthedocs.io/en/latest/api.html shows that there should be a nbformat.read() method, so I think we can update ipynb_runner.py as follows:

-from IPython.nbformat.current import read
+from IPython.nbformat import read

to address this issue.

mbrukman added a commit to mbrukman/ipycache that referenced this pull request Jan 14, 2022
The command `nosetests` appears to have disappeared from the GitHub Actions
images for Python 3.8 and 3.9 on all versions of Ubuntu and macOS, so let's try
installing it manually to fix the tests in the short-term.

During the development of [this change][1], we've learned that ipycache is not
compatible with recently-released IPython 8.0.0, so we've updated
`requirements.txt` to require a version of IPython prior to 8.0.0; additionally,
we re-enabled testing with Python 2.7 since we'll need to update our code in the
future to be compatible with `nose2`, so until we drop support for Python 2.7,
we should be testing with that version as well.

Closes rossant#63

[1]: rossant#64
mbrukman added a commit to mbrukman/ipycache that referenced this pull request Jan 14, 2022
The command `nosetests` appears to have disappeared from the GitHub Actions
images for Python 3.8 and 3.9 on all versions of Ubuntu and macOS, so let's try
installing it manually to fix the tests in the short-term.

During the development of [this change][1], we've learned that ipycache is not
compatible with recently-released IPython 8.0.0, so we've updated
`requirements.txt` to require a version of IPython prior to 8.0.0; additionally,
we re-enabled testing with Python 2.7 since we'll need to update our code in the
future to be compatible with `nose2`, so until we drop support for Python 2.7,
we should be testing with that version as well.

We're also manually installing `nbformat` and changed the import
`IPython.nbformat.current` to use top-level `nbformat` instead.

Closes rossant#63

[1]: rossant#64
@mbrukman
Copy link
Collaborator Author

Nope, that didn't work either:

Run nosetests
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/IPython/config.py:13: ShimWarning: The `IPython.config` package has been deprecated since IPython 4.0. You should import from traitlets.config instead.
  "You should import from traitlets.config instead.", ShimWarning)
/home/runner/work/ipycache/ipycache/ipycache.py:17: UserWarning: IPython.utils.traitlets has moved to a top-level traitlets package.
  from IPython.utils.traitlets import Unicode
..........
----------------------------------------------------------------------
Ran 10 tests in 0.790s

OK
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/setuptools/dist.py:454: UserWarning: Normalizing '0.1.5dev' to '0.1.5.dev0'
running check
  warnings.warn(tmpl.format(**locals()))
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/IPython/nbformat.py:13: ShimWarning: The `IPython.nbformat` package has been deprecated since IPython 4.0. You should import from nbformat instead.
  "You should import from nbformat instead.", ShimWarning)
Traceback (most recent call last):
  File "ipynb_runner.py", line 5, in <module>
    from IPython.nbformat import read
ModuleNotFoundError: No module named 'nbformat'
Error: Process completed with exit code 1.

Next approach to try is:

-from IPython.nbformat import read
+from nbformat import read

and manually install nbformat in test_requirements.txt to make sure it's present in the environment.

@mbrukman
Copy link
Collaborator Author

mbrukman commented Jan 14, 2022

We're making progress, but it still doesn't work:

Run nosetests
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/IPython/config.py:13: ShimWarning: The `IPython.config` package has been deprecated since IPython 4.0. You should import from traitlets.config instead.
  "You should import from traitlets.config instead.", ShimWarning)
/home/runner/work/ipycache/ipycache/ipycache.py:17: UserWarning: IPython.utils.traitlets has moved to a top-level traitlets package.
  from IPython.utils.traitlets import Unicode
..........
----------------------------------------------------------------------
Ran 10 tests in 0.610s

OK
running check
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/setuptools/dist.py:454: UserWarning: Normalizing '0.1.5dev' to '0.1.5.dev0'
  warnings.warn(tmpl.format(**locals()))
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/IPython/kernel/__init__.py:13: ShimWarning: The `IPython.kernel` package has been deprecated since IPython 4.0.You should import from ipykernel or jupyter_client instead.
  "You should import from ipykernel or jupyter_client instead.", ShimWarning)
Traceback (most recent call last):
  File "ipynb_runner.py", line 6, in <module>
    from IPython.kernel import KernelManager
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/IPython/kernel/__init__.py", line 31, in <module>
    from ipykernel import comm, inprocess
ModuleNotFoundError: No module named 'ipykernel'
Error: Process completed with exit code 1.

We will next attempt the following change:

-from IPython.kernel import KernelManager
+from jupyter_client.manager import KernelManager

and install jupyter-client.

mbrukman added a commit to mbrukman/ipycache that referenced this pull request Jan 14, 2022
The command `nosetests` appears to have disappeared from the GitHub Actions
images for Python 3.8 and 3.9 on all versions of Ubuntu and macOS, so let's try
installing it manually to fix the tests in the short-term.

During the development of [this change][1], we've learned that ipycache is not
compatible with recently-released IPython 8, so we updated `requirements.txt` to
require a version of IPython prior to 8; additionally, we re-enabled testing
with Python 2.7 since we'll need to update our code in the future to be
compatible with `nose2`, so until we drop support for Python 2.7, we should be
testing with that version as well.

Replace import `IPython.nbformat.current` with `nbformat` and install `nbformat`
module.

Replace `from IPython.kernel import KernelManager` with
`from jupyter_client.manager import KernelManager` and install `jupyter-client`.

Closes rossant#63

[1]: rossant#64
mbrukman added a commit to mbrukman/ipycache that referenced this pull request Jan 14, 2022
The command `nosetests` appears to have disappeared from the GitHub Actions
images for Python 3.8 and 3.9 on all versions of Ubuntu and macOS, so let's try
installing it manually to fix the tests in the short-term.

During the development of [this change][1], we've learned that ipycache is not
compatible with recently-released IPython 8, so we updated `requirements.txt` to
require a version of IPython prior to 8; additionally, we re-enabled testing
with Python 2.7 since we'll need to update our code in the future to be
compatible with `nose2`, so until we drop support for Python 2.7, we should be
testing with that version as well.

Replace import `IPython.nbformat.current` with `nbformat` and install `nbformat`
module.

Replace `from IPython.kernel import KernelManager` with
`from jupyter_client.manager import KernelManager` and install `jupyter-client`.

Closes rossant#63

[1]: rossant#64
@mbrukman
Copy link
Collaborator Author

Looks like the new problem: is:

Run nosetests
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/IPython/config.py:13: ShimWarning: The `IPython.config` package has been deprecated since IPython 4.0. You should import from traitlets.config instead.
  "You should import from traitlets.config instead.", ShimWarning)
/home/runner/work/ipycache/ipycache/ipycache.py:17: UserWarning: IPython.utils.traitlets has moved to a top-level traitlets package.
  from IPython.utils.traitlets import Unicode
..........
----------------------------------------------------------------------
Ran 10 tests in 1.003s

OK
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/setuptools/dist.py:454: UserWarning: Normalizing '0.1.5dev' to '0.1.5.dev0'
running check
  warnings.warn(tmpl.format(**locals()))
Checking: examples/example.ipynb
Traceback (most recent call last):
  File "ipynb_runner.py", line 41, in <module>
    nb = read(open(notebook), 'json')
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/nbformat/__init__.py", line 143, in read
    return reads(buf, as_version, **kwargs)
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/nbformat/__init__.py", line 75, in reads
    nb = convert(nb, as_version)
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/nbformat/converter.py", line 54, in convert
    "version doesn't exist" % (to_version))
TypeError: %d format: a number is required, not str
Error: Process completed with exit code 1.

Specifically, the following line of code is problematic with this version of nbformat:

nb = read(open(notebook), 'json')

because nbformat.read() takes as a second parameter the notebook version (int) rather than a string such as 'json'.

Let's see the versions of our notebooks:

$ grep nbformat examples/*
examples/capture_output.ipynb: "nbformat": 3,
examples/capture_output.ipynb: "nbformat_minor": 0,
examples/example.ipynb: "nbformat": 3,
examples/example.ipynb: "nbformat_minor": 0,
examples/example_outputs.ipynb: "nbformat": 3,
examples/example_outputs.ipynb: "nbformat_minor": 0,

Looks like they're all version 3, so we can make the following change:

-nb = read(open(notebook), 'json')
+nb = read(open(notebook), as_version=3)

@mbrukman
Copy link
Collaborator Author

Failing again. Python 2.7 tests failed with:

ERROR: Could not find a version that satisfies the requirement nbformat~=5.1.3 (from -r test_requirements.txt (line 3)) (from versions: 4.0.0, 4.0.1, 4.1.0, 4.2.0, 4.3.0, 4.4.0)
ERROR: No matching distribution found for nbformat~=5.1.3 (from -r test_requirements.txt (line 3))
Error: Process completed with exit code 1.

So we could go with 4.4.0 (which may resolve other issues as well).

Python 3.7 failed with:

Run nosetests
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/IPython/config.py:13: ShimWarning: The `IPython.config` package has been deprecated since IPython 4.0. You should import from traitlets.config instead.
  "You should import from traitlets.config instead.", ShimWarning)
/home/runner/work/ipycache/ipycache/ipycache.py:17: UserWarning: IPython.utils.traitlets has moved to a top-level traitlets package.
  from IPython.utils.traitlets import Unicode
..........
----------------------------------------------------------------------
Ran 10 tests in 0.809s

OK
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/setuptools/dist.py:454: UserWarning: Normalizing '0.1.5dev' to '0.1.5.dev0'
running check
  warnings.warn(tmpl.format(**locals()))
Checking: examples/example.ipynb
Traceback (most recent call last):
  File "ipynb_runner.py", line 45, in <module>
    km.start_kernel()
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/jupyter_client/utils.py", line 26, in wrapped
    raise e
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/jupyter_client/utils.py", line 23, in wrapped
    return loop.run_until_complete(future)
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/nest_asyncio.py", line 81, in run_until_complete
    return f.result()
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/asyncio/futures.py", line 181, in result
    raise self._exception
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/asyncio/tasks.py", line 249, in __step
    result = coro.send(None)
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/jupyter_client/manager.py", line 79, in wrapper
    raise e
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/jupyter_client/manager.py", line 71, in wrapper
    out = await method(self, *args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/jupyter_client/manager.py", line 376, in _async_start_kernel
    kernel_cmd, kw = await ensure_async(self.pre_start_kernel(**kw))
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/jupyter_client/utils.py", line 26, in wrapped
    raise e
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/jupyter_client/utils.py", line 23, in wrapped
    return loop.run_until_complete(future)
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/nest_asyncio.py", line 81, in run_until_complete
    return f.result()
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/asyncio/futures.py", line 181, in result
    raise self._exception
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/asyncio/tasks.py", line 249, in __step
    result = coro.send(None)
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/jupyter_client/manager.py", line 339, in _async_pre_start_kernel
    self.kernel_spec,
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/jupyter_client/manager.py", line 169, in kernel_spec
    self._kernel_spec = self.kernel_spec_manager.get_kernel_spec(self.kernel_name)
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/jupyter_client/kernelspec.py", line 292, in get_kernel_spec
    raise NoSuchKernel(kernel_name)
jupyter_client.kernelspec.NoSuchKernel: No such kernel named python3
Error: Process completed with exit code 1.

which is curious, as we don't specify python3 anywhere, so looks like that's the default.

Per this SO answer, it seems that we need to manually install the ipykernel module, which makes sense, so let's try that next. We'll also run jupyter kernelspec list before running the notebook tests to see what's actually installed, and split up the tests into separate phases to get shorter error messages per phase.

mbrukman added a commit to mbrukman/ipycache that referenced this pull request Jan 15, 2022
The command `nosetests` appears to have disappeared from the GitHub Actions
images for Python 3.8 and 3.9 on all versions of Ubuntu and macOS, so let's try
installing it manually to fix the tests in the short-term.

During the development of [this change][1], we've learned that ipycache is not
compatible with recently-released IPython 8, so we updated `requirements.txt` to
require a version of IPython prior to 8; additionally, we re-enabled testing
with Python 2.7 since we'll need to update our code in the future to be
compatible with `nose2`, so until we drop support for Python 2.7, we should be
testing with that version as well.

Replace import `IPython.nbformat.current` with `nbformat` and install `nbformat`
module.

Replace `from IPython.kernel import KernelManager` with
`from jupyter_client.manager import KernelManager` and install `jupyter-client`.

Also install `ipykernel` package to address the error message
"jupyter_client.kernelspec.NoSuchKernel: No such kernel named python3".

Closes rossant#63

[1]: rossant#64
@mbrukman
Copy link
Collaborator Author

We're making some progress! Tests now pass on Python 3.7 and 3.8.

Python 2.7 fails with:

ERROR: Could not find a version that satisfies the requirement jupyter-client~=7.1.1 (from -r test_requirements.txt (line 5)) (from versions: 4.0.0, 4.1.0, 4.1.1, 4.2.0, 4.2.1, 4.2.2, 4.3.0, 4.4.0, 5.0.0, 5.0.1, 5.1.0, 5.2.0, 5.2.1, 5.2.2, 5.2.3, 5.2.4, 5.3.0, 5.3.1, 5.3.2, 5.3.3, 5.3.4, 5.3.5)
ERROR: No matching distribution found for jupyter-client~=7.1.1 (from -r test_requirements.txt (line 5))

We could use jupyter-client ~= 5.3.5 for now, until we drop support for Python 2.

Python 3.9 fails with:

Run jupyter kernelspec list
Available kernels:
  python3    /opt/hostedtoolcache/Python/3.9.9/x64/share/jupyter/kernels/python3
Traceback (most recent call last):
  File "/home/runner/work/ipycache/ipycache/ipynb_runner.py", line 5, in <module>
    from nbformat import read
  File "/opt/hostedtoolcache/Python/3.9.9/x64/lib/python3.9/site-packages/nbformat/__init__.py", line 14, in <module>
    from . import v1
  File "/opt/hostedtoolcache/Python/3.9.9/x64/lib/python3.9/site-packages/nbformat/v1/__init__.py", line 19, in <module>
    from .nbjson import reads as reads_json, writes as writes_json
  File "/opt/hostedtoolcache/Python/3.9.9/x64/lib/python3.9/site-packages/nbformat/v1/nbjson.py", line 19, in <module>
    from base64 import encodestring
ImportError: cannot import name 'encodestring' from 'base64' (/opt/hostedtoolcache/Python/3.9.9/x64/lib/python3.9/base64.py)
Error: Process completed with exit code 1.

Which seems to be a failure purely due to jupyter kernelspec list, which we don't need anymore, now that we know ipykernel provided us with a working python3 kernel, so let's just drop it and see if that fixes things for us.

mbrukman added a commit to mbrukman/ipycache that referenced this pull request Jan 15, 2022
The command `nosetests` appears to have disappeared from the GitHub Actions
images for Python 3.8 and 3.9 on all versions of Ubuntu and macOS, so let's try
installing it manually to fix the tests in the short-term.

During the development of [this change][1], we've learned that ipycache is not
compatible with recently-released IPython 8, so we updated `requirements.txt` to
require a version of IPython prior to 8; additionally, we re-enabled testing
with Python 2.7 since we'll need to update our code in the future to be
compatible with `nose2`, so until we drop support for Python 2.7, we should be
testing with that version as well.

Replace import `IPython.nbformat.current` with `nbformat` and install `nbformat`
module.

Replace `from IPython.kernel import KernelManager` with
`from jupyter_client.manager import KernelManager` and install `jupyter-client`.

Also install `ipykernel` package to address the error message
"jupyter_client.kernelspec.NoSuchKernel: No such kernel named python3".

Closes rossant#63

[1]: rossant#64
@mbrukman
Copy link
Collaborator Author

Python 2.7 now fails with:

ERROR: Could not find a version that satisfies the requirement ipykernel~=6.7.0 (from -r test_requirements.txt (line 6)) (from versions: 4.0.1, 4.0.2, 4.0.3, 4.1.0, 4.1.1, 4.2.0, 4.2.1, 4.2.2, 4.3.0, 4.3.1, 4.4.0, 4.4.1, 4.5.0, 4.5.1, 4.5.2, 4.6.0, 4.6.1, 4.7.0, 4.8.0, 4.8.1, 4.8.2, 4.9.0, 4.10.0, 4.10.1)
ERROR: No matching distribution found for ipykernel~=6.7.0 (from -r test_requirements.txt (line 6))

so let's try ipykernel ~= 4.10.1.

Python 3.9 is failing with:

Run python ipynb_runner.py -v -s examples/example.ipynb
Traceback (most recent call last):
  File "/home/runner/work/ipycache/ipycache/ipynb_runner.py", line 5, in <module>
    from nbformat import read
  File "/opt/hostedtoolcache/Python/3.9.9/x64/lib/python3.9/site-packages/nbformat/__init__.py", line 14, in <module>
    from . import v1
  File "/opt/hostedtoolcache/Python/3.9.9/x64/lib/python3.9/site-packages/nbformat/v1/__init__.py", line 19, in <module>
    from .nbjson import reads as reads_json, writes as writes_json
  File "/opt/hostedtoolcache/Python/3.9.9/x64/lib/python3.9/site-packages/nbformat/v1/nbjson.py", line 19, in <module>
    from base64 import encodestring
ImportError: cannot import name 'encodestring' from 'base64' (/opt/hostedtoolcache/Python/3.9.9/x64/lib/python3.9/base64.py)
Error: Process completed with exit code 1.

which is the same error we saw previously, so it's not due to jupyter kernelspec list.

Per jupyter/nbformat#215, this is due to our use of nbformat ~= 4.4.0; we need to use nbformat 5.1.3 for Python 3.9 support, but that means we won't be able to support Python 2.7. It turns out that the reason our tests were passing before is because we disabled Python 2.7 in #55 when we fixed the CI the first time (after it hasn't run for quite some time).

Thus, we can either drop support for Python 2.7 immediately or we can create a separate requirements.txt such that we can run Python 2.7 tests in a separate environment with different versions of dependencies, until such time that we delete support for Python 2.7 entirely.

Let's try creating a separate test_requirements_python27.txt with downgraded dependencies, and upgrade our dependency versions in test_requirements.txt for the latest versions to support the newest Python 3 versions.

mbrukman added a commit to mbrukman/ipycache that referenced this pull request Jan 15, 2022
The command `nosetests` appears to have disappeared from the GitHub Actions
images for Python 3.8 and 3.9 on all versions of Ubuntu and macOS, so let's try
installing it manually to fix the tests in the short-term.

During the development of [this change][1], we've learned that ipycache is not
compatible with recently-released IPython 8, so we updated `requirements.txt` to
require a version of IPython prior to 8; additionally, we re-enabled testing
with Python 2.7 since we'll need to update our code in the future to be
compatible with `nose2`, so until we drop support for Python 2.7, we should be
testing with that version as well.

Replace import `IPython.nbformat.current` with `nbformat` and install `nbformat`
module.

Replace `from IPython.kernel import KernelManager` with
`from jupyter_client.manager import KernelManager` and install `jupyter-client`.

Also install `ipykernel` package to address the error message
"jupyter_client.kernelspec.NoSuchKernel: No such kernel named python3".

Create a separate `test_requirements_python27.txt` with old versions of
dependencies to support Python 2.7, and upgrade dependencies in
`test_requirements.txt` to the latest versions for the newest Python 3.x
releases.

Closes rossant#63

[1]: rossant#64
@mbrukman
Copy link
Collaborator Author

Looks like Python 3.7, 3.8, and 3.9 are passing! But both Python 2.7 and 3.x are being installed in all cases, so Python 2.7 is failing. We'll need to fix the YAML syntax of GitHub Actions if statement to make them mutually exclusive.

The command `nosetests` appears to have disappeared from the GitHub Actions
images for Python 3.8 and 3.9 on all versions of Ubuntu and macOS, so let's try
installing it manually to fix the tests in the short-term.

During the development of [this change][1], we've learned that ipycache is not
compatible with recently-released IPython 8, so we updated `requirements.txt` to
require a version of IPython prior to 8; additionally, we re-enabled testing
with Python 2.7 since we'll need to update our code in the future to be
compatible with `nose2`, so until we drop support for Python 2.7, we should be
testing with that version as well.

Replace import `IPython.nbformat.current` with `nbformat` and install `nbformat`
module.

Replace `from IPython.kernel import KernelManager` with
`from jupyter_client.manager import KernelManager` and install `jupyter-client`.

Also install `ipykernel` package to address the error message
"jupyter_client.kernelspec.NoSuchKernel: No such kernel named python3".

Create a separate `test_requirements_python27.txt` with old versions of
dependencies to support Python 2.7, and upgrade dependencies in
`test_requirements.txt` to the latest versions for the newest Python 3.x
releases.

Closes rossant#63

[1]: rossant#64
@mbrukman
Copy link
Collaborator Author

Success!! 🎉

All tests finally pass across the full matrix of Python and OS versions.

@mbrukman mbrukman merged commit 431b510 into rossant:master Jan 15, 2022
@mbrukman mbrukman deleted the install-nose-explicitly branch January 15, 2022 02:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Install nose explicitly as a quick fix to unblock license clarification
1 participant