Using Coconut w/IPython (in a Python3 virtualenv) as described in the tutorial doesn't work

[x10an14]

Tutorial link; Here.
Full attempt; Here.

Relevant error-messages and commands invoking them:

x10an14@x10Desktop2016:~/Documents/coconut$ venv/bin/coconut --ipython console
Traceback (most recent call last):
  File "venv/bin/coconut", line 11, in <module>
    sys.exit(main())
  File "/home/x10an14/Documents/coconut/venv/lib/python3.5/site-packages/coconut/__main__.py", line 32, in main
    cli().start()
  File "/home/x10an14/Documents/coconut/venv/lib/python3.5/site-packages/coconut/command.py", line 286, in start
    self.cmd(self.commandline.parse_args())
  File "/home/x10an14/Documents/coconut/venv/lib/python3.5/site-packages/coconut/command.py", line 367, in cmd
    self.start_jupyter(args.jupyter)
  File "/home/x10an14/Documents/coconut/venv/lib/python3.5/site-packages/coconut/command.py", line 598, in start_jupyter
    install_func(check_args)
  File "/home/x10an14/Documents/coconut/venv/lib/python3.5/site-packages/coconut/command.py", line 592, in <lambda>
    install_func = lambda args: subprocess.check_output(args, stderr=subprocess.STDOUT) # stdout is returned and ignored
  File "/usr/lib/python3.5/subprocess.py", line 626, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.5/subprocess.py", line 693, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.5/subprocess.py", line 947, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.5/subprocess.py", line 1551, in _execute_child
    raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: 'jupyter'
x10an14@x10Desktop2016:~/Documents/coconut$ 

PIP packages installed:

x10an14@x10Desktop2016:~/Documents/coconut$ venv/bin/pip freeze
coconut==1.1.1
decorator==4.0.10
ipython==5.1.0
ipython-genutils==0.1.0
pexpect==4.2.1
pickleshare==0.7.4
pkg-resources==0.0.0
prompt-toolkit==1.0.7
ptyprocess==0.5.1
Pygments==2.1.3
pyparsing==2.1.5
simplegeneric==0.8.1
six==1.10.0
traitlets==4.2.2
wcwidth==0.1.7
x10an14@x10Desktop2016:~/Documents/coconut$ 

It is (as far as I could tell while skimming over, excited to try coconut) that jupyter needs to be installed, for the successful invocation of an IPython console with Coconut invoked with the following command:

coconut --ipython console

Is this the case? If so, I would suggest perhaps editing the tutorial/documentation to reflect this.

Please let me know if there's anything else I've misunderstood, or any other information I can supply to clarify what's going wrong.
(As the 2nd link at the top shows, it works to execute coconut code from inside an ipython console, but that seems a bit cumbersome, no? At least if what you're (I'm) after is an interactive console).

[evhub]

@x10an14 Yes, the dependency is on Jupyter not IPython. By itself IPython no longer provides support for alternate kernels such as Coconut--that support was moved to Jupyter. The problem should be resolved if you pip install jupyter. I'll try to update the documentation to make that more clear.

[x10an14]

@evhub Hi! And thanks for the prompt reply!

I did attempt the command after installing Jupyter via PIP, but I got a very similar (if not the same as the one below) error message.

I'm currently at work, where I've only got access to Windows, but here's an attempt using Windows, an attempt which includes installing Jupyter in the same fashion as IPython:

C:\coconut> .\venv\Scripts\coconut --verbose --ipython console
> jupyter --version
4.1.1
> jupyter kernelspec install c:\coconut\venv\lib\site-packages\coconut\icoconut\coconut --replace
'jupyter-kernelspec' is not recognized as an internal or external command,
operable program or batch file.
> jupyter kernelspec install c:\coconut\venv\lib\site-packages\coconut\icoconut\coconut --replace --user
'jupyter-kernelspec' is not recognized as an internal or external command,
operable program or batch file.
Traceback (most recent call last):
  File "c:\coconut\venv\lib\site-packages\coconut\command.py", line 607, in start_jupyter
    install_func(install_args)
  File "c:\coconut\venv\lib\site-packages\coconut\command.py", line 594, in <lambda>
    install_func = lambda args: subprocess.check_call(args)
  File "C:\Program Files\Python35\Lib\subprocess.py", line 581, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['jupyter', 'kernelspec', 'install', 'c:\\coconut\\venv\\lib\\site-packages\\coconut\\icoconut\\coconut', '--replace']' returned non-zero exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\coconut\venv\lib\site-packages\coconut\command.py", line 612, in start_jupyter
    install_func(user_install_args)
  File "c:\coconut\venv\lib\site-packages\coconut\command.py", line 594, in <lambda>
    install_func = lambda args: subprocess.check_call(args)
  File "C:\Program Files\Python35\Lib\subprocess.py", line 581, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['jupyter', 'kernelspec', 'install', 'c:\\coconut\\venv\\lib\\site-packages\\coconut\\icoconut\\coconut', '--replace', '--user']' returned non-zero exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\coconut\venv\lib\site-packages\coconut\compiler.py", line 1450, in warn
    raise warning
coconut.compiler.CoconutWarning: unable to install Jupyter kernel specification file (failed command "jupyter kernelspec install c:\coconut\venv\lib\site-packages\coconut\icoconut\coconut --replace")
> jupyter kernelspec install c:\coconut\venv\lib\site-packages\coconut\icoconut\coconut2 --replace
'jupyter-kernelspec' is not recognized as an internal or external command,
operable program or batch file.
> jupyter kernelspec install c:\coconut\venv\lib\site-packages\coconut\icoconut\coconut2 --replace --user
'jupyter-kernelspec' is not recognized as an internal or external command,
operable program or batch file.
Traceback (most recent call last):
  File "c:\coconut\venv\lib\site-packages\coconut\command.py", line 607, in start_jupyter
    install_func(install_args)
  File "c:\coconut\venv\lib\site-packages\coconut\command.py", line 594, in <lambda>
    install_func = lambda args: subprocess.check_call(args)
  File "C:\Program Files\Python35\Lib\subprocess.py", line 581, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['jupyter', 'kernelspec', 'install', 'c:\\coconut\\venv\\lib\\site-packages\\coconut\\icoconut\\coconut2', '--replace']' returned non-zero exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\coconut\venv\lib\site-packages\coconut\command.py", line 612, in start_jupyter
    install_func(user_install_args)
  File "c:\coconut\venv\lib\site-packages\coconut\command.py", line 594, in <lambda>
    install_func = lambda args: subprocess.check_call(args)
  File "C:\Program Files\Python35\Lib\subprocess.py", line 581, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['jupyter', 'kernelspec', 'install', 'c:\\coconut\\venv\\lib\\site-packages\\coconut\\icoconut\\coconut2', '--replace', '--user']' returned non-zero exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\coconut\venv\lib\site-packages\coconut\compiler.py", line 1450, in warn
    raise warning
coconut.compiler.CoconutWarning: unable to install Jupyter kernel specification file (failed command "jupyter kernelspec install c:\coconut\venv\lib\site-packages\coconut\icoconut\coconut2 --replace")
> jupyter kernelspec install c:\coconut\venv\lib\site-packages\coconut\icoconut\coconut3 --replace
'jupyter-kernelspec' is not recognized as an internal or external command,
operable program or batch file.
> jupyter kernelspec install c:\coconut\venv\lib\site-packages\coconut\icoconut\coconut3 --replace --user
'jupyter-kernelspec' is not recognized as an internal or external command,
operable program or batch file.
Traceback (most recent call last):
  File "c:\coconut\venv\lib\site-packages\coconut\command.py", line 607, in start_jupyter
    install_func(install_args)
  File "c:\coconut\venv\lib\site-packages\coconut\command.py", line 594, in <lambda>
    install_func = lambda args: subprocess.check_call(args)
  File "C:\Program Files\Python35\Lib\subprocess.py", line 581, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['jupyter', 'kernelspec', 'install', 'c:\\coconut\\venv\\lib\\site-packages\\coconut\\icoconut\\coconut3', '--replace']' returned non-zero exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\coconut\venv\lib\site-packages\coconut\command.py", line 612, in start_jupyter
    install_func(user_install_args)
  File "c:\coconut\venv\lib\site-packages\coconut\command.py", line 594, in <lambda>
    install_func = lambda args: subprocess.check_call(args)
  File "C:\Program Files\Python35\Lib\subprocess.py", line 581, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['jupyter', 'kernelspec', 'install', 'c:\\coconut\\venv\\lib\\site-packages\\coconut\\icoconut\\coconut3', '--replace', '--user']' returned non-zero exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\coconut\venv\lib\site-packages\coconut\compiler.py", line 1450, in warn
    raise warning
coconut.compiler.CoconutWarning: unable to install Jupyter kernel specification file (failed command "jupyter kernelspec install c:\coconut\venv\lib\site-packages\coconut\icoconut\coconut3 --replace")
> python3 -m coconut --version
Traceback (most recent call last):
  File "C:\Program Files\Python35\Lib\runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Program Files\Python35\Lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\coconut\venv\Scripts\coconut.exe\__main__.py", line 9, in <module>
  File "c:\coconut\venv\lib\site-packages\coconut\__main__.py", line 32, in main
    cli().start()
  File "c:\coconut\venv\lib\site-packages\coconut\command.py", line 286, in start
    self.cmd(self.commandline.parse_args())
  File "c:\coconut\venv\lib\site-packages\coconut\command.py", line 367, in cmd
    self.start_jupyter(args.jupyter)
  File "c:\coconut\venv\lib\site-packages\coconut\command.py", line 625, in start_jupyter
    install_func(check_args)
  File "c:\coconut\venv\lib\site-packages\coconut\command.py", line 594, in <lambda>
    install_func = lambda args: subprocess.check_call(args)
  File "C:\Program Files\Python35\Lib\subprocess.py", line 576, in check_call
    retcode = call(*popenargs, **kwargs)
  File "C:\Program Files\Python35\Lib\subprocess.py", line 557, in call
    with Popen(*popenargs, **kwargs) as p:
  File "C:\Program Files\Python35\Lib\subprocess.py", line 947, in __init__
    restore_signals, start_new_session)
  File "C:\Program Files\Python35\Lib\subprocess.py", line 1224, in _execute_child
    startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
C:\coconut>

Another gist showing the full attempt (including install and pip freeze); Here.

[x10an14]

If I could be so bold as to hazard a guess, it looks to me like it's presumed that Jupyter is available through the 'machine' PATH, when this is all installed in a virtualenvironment, and an "non-activated" one at that.

[evhub]

@x10an14 Ah! Yup, I think you're exactly correct. From the --verbose output, you can see that the real problem is right here:

'jupyter-kernelspec' is not recognized as an internal or external command,
operable program or batch file.

[evhub]

@x10an14 There's not much I can do to really fix this error, since coconut --jupyter console is always going to have to dispatch calls to the command-line, but I'll at least try to make the error message nicer in the next release.

[x10an14]

@evhub Ahh, allright. Well, then I'd recommend editing the documentation to reflect this limitation when using Coconut in a virtualenv =)

Good job, I'm very impressed with this project, and look forward to getting familiar with it!

[x10an14]

Hmmm... Even after installing Jupyter on the 'machine PATH', I can't get the console to open successfully (on Windows).

I get further, but I still don't get an interactive console opened, which supports Coconut:

C:\coconut> .\venv\Scripts\coconut --verbose --ipython console
> jupyter --version
4.1.1
> jupyter kernelspec install c:\coconut\venv\lib\site-packages\coconut\icoconut\coconut --replace
[InstallKernelSpec] Installed kernelspec coconut in C:\ProgramData\jupyter\kernels\coconut
> jupyter kernelspec install c:\coconut\venv\lib\site-packages\coconut\icoconut\coconut2 --replace
[InstallKernelSpec] Installed kernelspec coconut2 in C:\ProgramData\jupyter\kernels\coconut2
> jupyter kernelspec install c:\coconut\venv\lib\site-packages\coconut\icoconut\coconut3 --replace
[InstallKernelSpec] Installed kernelspec coconut3 in C:\ProgramData\jupyter\kernels\coconut3
> python3 -m coconut --version
Traceback (most recent call last):
  File "C:\Program Files\Python35\Lib\runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Program Files\Python35\Lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\coconut\venv\Scripts\coconut.exe\__main__.py", line 9, in <module>
  File "c:\coconut\venv\lib\site-packages\coconut\__main__.py", line 32, in main
    cli().start()
  File "c:\coconut\venv\lib\site-packages\coconut\command.py", line 286, in start
    self.cmd(self.commandline.parse_args())
  File "c:\coconut\venv\lib\site-packages\coconut\command.py", line 367, in cmd
    self.start_jupyter(args.jupyter)
  File "c:\coconut\venv\lib\site-packages\coconut\command.py", line 625, in start_jupyter
    install_func(check_args)
  File "c:\coconut\venv\lib\site-packages\coconut\command.py", line 594, in <lambda>
    install_func = lambda args: subprocess.check_call(args)
  File "C:\Program Files\Python35\Lib\subprocess.py", line 576, in check_call
    retcode = call(*popenargs, **kwargs)
  File "C:\Program Files\Python35\Lib\subprocess.py", line 557, in call
    with Popen(*popenargs, **kwargs) as p:
  File "C:\Program Files\Python35\Lib\subprocess.py", line 947, in __init__
    restore_signals, start_new_session)
  File "C:\Program Files\Python35\Lib\subprocess.py", line 1224, in _execute_child
    startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
C:\coconut>

Continuation of previous link in above comment can be found here:
https://gist.github.com/x10an14/74ea6cb9e16e4b9564c55b84ac192507

I have no experience with Jupyter, only used IPython before, so I feel that I've got to ask:

• Have I misunderstood how to install/use this?
• Am I perhaps lacking/forgetting some step(s)?

Obviously I did manage to install it on the 'machine PATH' (as I've been referring to it), since I got further than the last reported attempt, but I'm starting to feel pretty stumped here.

Should I perhaps go ask Jupyter if they can help me?

(I followed the instructions here to install Jupyter through PIP, putitng the resulting executable on the machine PATH).

[evhub]

@x10an14 I think your problem is that you aren't activating your virtualenv, since not only jupyter, but also coconut itself, needs to be accessible on the command line for coconut --jupyter console to work. You can see that since the failing command is:

python3 -m coconut --version

which is Coconut testing to make sure it can call itself on the command-line.

The following worked for me on windows:

virtualenv .\coconut-test
.\coconut-test\Scripts\activate
pip install jupyter coconut
coconut --ipython console

And thanks! Feel free to stop by Coconut's Gitter if you have any other questions you want to ask.

[evhub]

@x10an14 I've been experimenting with this more, and I think I've found an actual bug now as well--it seems that Coconut is working nicely with Jupyter console 4.1.1, but breaking with Jupyter console 5.0.0. I'm keeping this open to try to investigate that breakage, and make sure that in the future pip install coconut[jupyter] will always install a working Jupyter version.

For now, if you still can't get the Jupyter console to work, try:

pip install --upgrade jupyter-console==4.1.1

[x10an14]

@evhub Thanks so much for your indulging my newbish questions, and your time and efforts!

I didn't get your steps listed here to work, but I'll make an extra effort to check if I somehow messed up (I tried your steps on Linux).

If I can't, I'll get on Gitter as requested! =) Thanks again for all your help and patience!

[x10an14]

@evhub Thanks for your continued help at your previously linked Gitter channel!
After our discussion there, I've found the following steps to work (so far only tested on Linux), in the quest for an interactive prompt already-loaded with Coconut (IPython style), in a Python3 virtual environment:

1. Install a virtualenv

   • virtualenv -p python3 venv

2. Perform the following steps as described below (I think order matters! -Especially b) and c)...), or activate the virtualenv and use its PATH binaries (same either way) instead now, and skip step 3 :

   a) venv/bin/pip install coconut jupyter
   b) venv/bin/pip install -U jupyter-console==4.1.1
   c) venv/bin/pip install -U jupyter==1.0.0

3. Activate the virtualenv if you haven't already (this is mandatory for step 4):

   • On Linux: source venv/bin/activate

4. Open interactive, IPython-style prompt with:

   • coconut --jupyter console

5. Success! (yay! =D)

Edit 1: Step 4 can also read coconut --ipython console! Just tested! (Not really sure of the difference, though).