Question about running in Jypiter notebooks

[tsemana]

@chmp - from you comment in the issue logged in the pytest repo:
pytest-dev/pytest#2268 (comment)

In both cases, the asserts will be pytest flavoured. You can execute test_a either in the cell it was defined in or anywhere else.

I'm new to using Python / Jupyiter and found this entry, but I'm sure I'm doing something wrong because I can't get the simple examples to work. Using the first example, I get stuck after the second cell with

unknown magic command '%rewrite_asserts'
UnknownMagic: unknown magic command '%rewrite_asserts'

More generally - I'm not sure how to 'execute' the test. Whether this means just calling the function normally, since using %%run_pytest will result in the same error above, but calling the function does not return especially helpful results.

I found the Example.ipynb file and running them provides the a different error when I run ipytest.run('-qq')

ERROR: not found: /mnt/tmp/8347656836271596575
(no name '/mnt/tmp/8347656836271596575' in any of [])

For context, I'm running the jupyter notebook on and AWS EMR instance running a PySpark kernel. Relevant installed versions:

ipytest                    0.7.1  
ipython                    7.11.1 
ipython-genutils           0.2.0  
pytest                     5.3.4 

Thanks for your help.

[chmp]

Hi @tsemana.

The problems problems you describe sound very much, like config was not called. After that everything should work. The basic setup is:

import ipytest
ipytest.config(rewrite_asserts=True, magics=True)

__file__ = "INSERT YOUR NOTEBOOK FILENAME HERE"

Hopfefully that fixes the issues.

[tsemana]

@chmp thank you for the response. Unfortunately I did have that block. Here is the entirety of the cells I'm trying to get it to run:

cell1 (adding libraries to emr cluster running the notebook)

sc.install_pypi_package("pytest")
sc.install_pypi_package("ipytest")
sc.install_pypi_package("ipython")

cell2

import pytest
import ipytest

ipytest.config(rewrite_asserts=True, magics=True)

__file__ = "udacity-tony-test-emr.ipynb"

cell3

def my_func(x):
    return x // 2 * 2

cell4

# define the tests

def test_my_func():
    assert my_func(0) == 0
    assert my_func(1) == 0
    assert my_func(2) == 2
    assert my_func(3) == 2

cell5

# execute the tests via pytest, arguments are passed to pytest
ipytest.run('-qq')

response

ERROR: not found: /mnt/tmp/1008237126276784108
(no name '/mnt/tmp/1008237126276784108' in any of [])

cell6

%%run_pytest[clean] -qq

response

unknown magic command '%run_pytest[clean]'
UnknownMagic: unknown magic command '%run_pytest[clean]'

With each attempt at running the cells from a restart of the notebook, the following series of errors always show on running Cell2 - where I import ipytest and set the config:

The first time the cell is run:

'NoneType' object has no attribute 'ast_transformers'
Traceback (most recent call last):
  File "/tmp/1580125054377-0/local/lib/python3.6/site-packages/ipytest/_config.py", line 80, in __call__
    context.__enter__()
  File "/tmp/1580125054377-0/local/lib/python3.6/site-packages/ipytest/_config.py", line 145, in __enter__
    setattr(self.config, k, v)
  File "/tmp/1580125054377-0/local/lib/python3.6/site-packages/ipytest/_config.py", line 29, in __set__
    self.func(instance, value)
  File "/tmp/1580125054377-0/local/lib/python3.6/site-packages/ipytest/_config.py", line 91, in rewrite_asserts
    self._rewrite_context.__enter__()
  File "/tmp/1580125054377-0/local/lib/python3.6/site-packages/ipytest/_pytest_support.py", line 163, in __enter__
    self.register()
  File "/tmp/1580125054377-0/local/lib/python3.6/site-packages/ipytest/_pytest_support.py", line 173, in register
    self.shell.ast_transformers.append(self.transformer)
AttributeError: 'NoneType' object has no attribute 'ast_transformers'

When I rerun the cell, the error message changes:

'NoneType' object has no attribute 'register_magics'
Traceback (most recent call last):
  File "/tmp/1580124131462-0/local/lib/python3.6/site-packages/ipytest/_config.py", line 80, in __call__
    context.__enter__()
  File "/tmp/1580124131462-0/local/lib/python3.6/site-packages/ipytest/_config.py", line 145, in __enter__
    setattr(self.config, k, v)
  File "/tmp/1580124131462-0/local/lib/python3.6/site-packages/ipytest/_config.py", line 29, in __set__
    self.func(instance, value)
  File "/tmp/1580124131462-0/local/lib/python3.6/site-packages/ipytest/_config.py", line 104, in magics
    get_ipython().register_magics(IPyTestMagics)
AttributeError: 'NoneType' object has no attribute 'register_magics'

When I re-run the cell a third time, there are is no error. This series happens consistently, regardless of how long I wait between running cell1 and cell2 (though I know that should not matter).

When I call ipytest.config after this I get the following output:

<Config rewrite_asserts=True, magics=True, tempfile_fallback=False, clean='[Tt]est*', addopts=(), raise_on_error=False, run_in_thread=False>

Hope this gives enough details to assess what's going wrong.
Thanks again.

[chmp]

Hm. Strange. From above's error message, my guess would be, you're not running inside a standard Jupyter notebook with an IPython kernel. Otherwise, I would expect that get_ipython() gives you the IPython shell.

What is the environment you're running in?

To set expectations: at the moment ipytest is pretty much hardwired around the IPython kernel, not sure it's easy to support other environments.

[tsemana]

Ok, that would explain it then if that's a hard limitation. I'm running on the notebook on an AWS EMR cluster running on PySpark kernel.

I was hoping to be able to test in the notebook to validate ETL processes.

[chmp]

Maybe nbval works for your usecase. I, for example, test the notebooks in this repo with nbval. The corresponding code can be found here.

Not sure though, whether they make any assumptions about the kernel.

Given our discussion, I will close the issue.