xonsh Kernel #274

WardThomas · 2019-10-23T08:02:52Z

Hello Bo Peng, thanks for sharing Sos.
Would it be possible to add Xonsh as an SoS sub kernel?

I see you use the metakernel/Calysto bash as a base for your bash sub kernel and that there is a metakernel/Calysto xonsh as well, so maybe it would be fairly easy to add xonsh?

https://nbviewer.jupyter.org/github/Calysto/xonsh_kernel/tree/master/

regards

BoPeng · 2019-10-23T13:11:35Z

I installed xonsh and xonsh-kernel, then

$ python -m xonsh_kernel install
/Users/bpeng1/anaconda3/envs/sos/lib/python3.7/site-packages/ipykernel/iostream.py:14: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  from imp import lock_held as import_lock_held
Installed kernelspec python3 in /usr/local/share/jupyter/kernels/python3

I do not see a xonsh kernel in jupyter. What did I do wrong?

BoPeng · 2019-10-23T13:43:11Z

If xonsh-kernel is compatible with Python, it is likely that you can enable variable exchange with

%use xonsh --language Python3

Let me know if it works.

WardThomas · 2019-10-23T13:43:38Z

Hi BoPeng,

for info - I do see calysto_xonsh kernel in jupyter ( its a vanilla pipenv install of jupyterlab (no SoS installed yet), but it throws errors if they are used

pipenv install xonsh_kernel
pipenv shell
python -m xonsh_kernel.install

both jupyterlab and ipython recognise the calysto_xonsh but they both throw an errors

output from jupyterlab and ipython below

jupyter lab --debug
......stuff deleted...
[D 14:33:26.158 LabApp] Found kernel calysto_xonsh in /home//.local/share/jupyter/kernels
[D 14:33:26.158 LabApp] Found kernel python3 in /home/.local/share/virtualenvs/bk1jwvoh/share/jupyter/kernels
[D 14:33:26.158 LabApp] Found kernel python2 in /usr/share/jupyter/kernels
[D 14:33:26.158 LabApp] Found kernel xonsh in /usr/share/jupyter/kernels

ipython console --kernel calysto_xonsh
......stuff deleted...
Xonsh - the Python-ish, BASHwards-looking shell
In [1]:

more complete output below

/home/fairlead/.local/share/virtualenvs/fairlead-bk1jwvoh/lib/python3.7/site-packages/ipykernel/iostream.py:14: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  from imp import lock_held as import_lock_held
/home/fairlead/.local/share/virtualenvs/fairlead-bk1jwvoh/lib/python3.7/site-packages/ipykernel/kernelapp.py:523: DeprecationWarning: zmq.eventloop.ioloop is deprecated in pyzmq 17. pyzmq now works with default tornado and asyncio eventloops.
  zmq_ioloop.install()
/home/fairlead/.local/share/virtualenvs/fairlead-bk1jwvoh/lib/python3.7/site-packages/jupyter_client/connect.py:330: DeprecationWarning: IPKernelApp._ip_changed is deprecated in traitlets 4.1: use @observe and @unobserve instead.
  def _ip_changed(self, name, old, new):
Jupyter console 6.0.0

Xonsh - the Python-ish, BASHwards-looking shell

/home/fairlead/.local/share/virtualenvs/fairlead-bk1jwvoh/lib/python3.7/site-packages/traitlets/config/configurable.py:73: DeprecationWarning: Passing unrecoginized arguments to super(XonshKernel).__init__(user_ns=None).
object.__init__() takes exactly one argument (the instance to initialize)
This is deprecated in traitlets 4.2.This error will be raised in a future release of traitlets.
  super(Configurable, self).__init__(**kwargs)
/home/fairlead/.local/share/virtualenvs/fairlead-bk1jwvoh/lib/python3.7/site-packages/traitlets/config/configurable.py:73: DeprecationWarning: Passing unrecoginized arguments to super(CommManager).__init__(shell=None).
object.__init__() takes exactly one argument (the instance to initialize)
This is deprecated in traitlets 4.2.This error will be raised in a future release of traitlets.
  super(Configurable, self).__init__(**kwargs)
In [1]: echo "ddd" 
      :                                                                                                                                                                                                                                                                                                                      
[IPKernelApp] ERROR | Exception in message handler:
Traceback (most recent call last):
  File "/home/fairlead/.local/share/virtualenvs/fairlead-bk1jwvoh/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 268, in dispatch_shell
    yield gen.maybe_future(handler(stream, idents, msg))
  File "/home/fairlead/.local/share/virtualenvs/fairlead-bk1jwvoh/lib/python3.7/site-packages/tornado/gen.py", line 735, in run
    value = future.result()
  File "/home/fairlead/.local/share/virtualenvs/fairlead-bk1jwvoh/lib/python3.7/site-packages/tornado/gen.py", line 209, in wrapper
    yielded = next(result)
  File "/home/fairlead/.local/share/virtualenvs/fairlead-bk1jwvoh/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 541, in execute_request
    user_expressions, allow_stdin,
  File "/home/fairlead/.local/share/virtualenvs/fairlead-bk1jwvoh/lib/python3.7/site-packages/metakernel/_metakernel.py", line 395, in do_execute
    retval = self.do_execute_direct(code)
  File "/home/fairlead/.local/share/virtualenvs/fairlead-bk1jwvoh/lib/python3.7/site-packages/xonsh_kernel/kernel.py", line 45, in do_execute_direct
    out, err, interrupted = self._do_execute_direct(code)
  File "/home/fairlead/.local/share/virtualenvs/fairlead-bk1jwvoh/lib/python3.7/site-packages/xonsh_kernel/kernel.py", line 60, in _do_execute_direct
    shell = builtins.__xonsh_shell__
AttributeError: module 'builtins' has no attribute '__xonsh_shell__'

WardThomas · 2019-10-23T13:46:27Z

Hi BoPeng

%use xonsh --language Python3

xonsh runs all python commands and it can also run most bash type commands
so looks like it should work
I will try it in about two weeks time

Many thanks for you help
Thomas

BoPeng · 2019-10-23T13:50:41Z

Instead of

python -m xonsh_kernel.install

I used

python -m xonsh_kernel install

which somehow worked and installed a python3 kernel. This might be a bug with MetaKernel.

BoPeng · 2019-10-23T13:57:02Z

The kernel does not work on mac with python 3.7,

(sos) [bpeng1@BCBMF5KWK0F8F69:~]$ jupyter notebook
[I 08:54:44.069 NotebookApp] Loading IPython parallel extension
[I 08:54:44.132 NotebookApp] JupyterLab extension loaded from /Users/bpeng1/anaconda3/envs/sos/lib/python3.7/site-packages/jupyterlab
[I 08:54:44.132 NotebookApp] JupyterLab application directory is /Users/bpeng1/anaconda3/envs/sos/share/jupyter/lab
[I 08:54:44.134 NotebookApp] Serving notebooks from local directory: /Users/bpeng1
[I 08:54:44.134 NotebookApp] The Jupyter Notebook is running at:
[I 08:54:44.134 NotebookApp] http://localhost:8888/?token=6647c730294047f7d79a64c8b087d5832cee855637864490
[I 08:54:44.134 NotebookApp]  or http://127.0.0.1:8888/?token=6647c730294047f7d79a64c8b087d5832cee855637864490
[I 08:54:44.134 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 08:54:44.140 NotebookApp] 
    
    To access the notebook, open this file in a browser:
        file:///Users/bpeng1/Library/Jupyter/runtime/nbserver-27518-open.html
    Or copy and paste one of these URLs:
        http://localhost:8888/?token=6647c730294047f7d79a64c8b087d5832cee855637864490
     or http://127.0.0.1:8888/?token=6647c730294047f7d79a64c8b087d5832cee855637864490
[W 08:54:44.415 NotebookApp] 404 GET /api/kernels/7a71f3f6-f409-4273-bcbb-1f0ce6128301/channels?session_id=8ed21707a11f43178b548090e2920614 (::1): Kernel does not exist: 7a71f3f6-f409-4273-bcbb-1f0ce6128301
[W 08:54:44.461 NotebookApp] 404 GET /api/kernels/7a71f3f6-f409-4273-bcbb-1f0ce6128301/channels?session_id=8ed21707a11f43178b548090e2920614 (::1) 65.30ms referer=None
[W 08:54:48.594 NotebookApp] 404 GET /api/kernels/e021d546-2991-4334-af88-2122dfec84e7/channels?session_id=038c76d00aa448c895f11303794d9730 (::1): Kernel does not exist: e021d546-2991-4334-af88-2122dfec84e7
[W 08:54:48.595 NotebookApp] 404 GET /api/kernels/e021d546-2991-4334-af88-2122dfec84e7/channels?session_id=038c76d00aa448c895f11303794d9730 (::1) 2.69ms referer=None
[W 08:54:48.608 NotebookApp] Replacing stale connection: 7a71f3f6-f409-4273-bcbb-1f0ce6128301:8ed21707a11f43178b548090e2920614
[I 08:54:49.689 NotebookApp] Creating new notebook in 
[I 08:54:50.527 NotebookApp] Kernel started: c23b21ba-2ee7-4462-9695-5a583bb39a38
/Users/bpeng1/anaconda3/envs/sos/lib/python3.7/site-packages/ipykernel/iostream.py:14: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  from imp import lock_held as import_lock_held
/Users/bpeng1/anaconda3/envs/sos/lib/python3.7/site-packages/ipykernel/kernelapp.py:523: DeprecationWarning: zmq.eventloop.ioloop is deprecated in pyzmq 17. pyzmq now works with default tornado and asyncio eventloops.
  zmq_ioloop.install()
/Users/bpeng1/anaconda3/envs/sos/lib/python3.7/site-packages/jupyter_client/connect.py:326: DeprecationWarning: IPKernelApp._ip_changed is deprecated in traitlets 4.1: use @observe and @unobserve instead.
  def _ip_changed(self, name, old, new):
[IPKernelApp] ERROR | Exception in message handler:
Traceback (most recent call last):
  File "/Users/bpeng1/anaconda3/envs/sos/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 272, in dispatch_shell
    yield gen.maybe_future(handler(stream, idents, msg))
  File "/Users/bpeng1/anaconda3/envs/sos/lib/python3.7/site-packages/tornado/gen.py", line 735, in run
    value = future.result()
  File "/Users/bpeng1/anaconda3/envs/sos/lib/python3.7/site-packages/tornado/gen.py", line 209, in wrapper
    yielded = next(result)
  File "/Users/bpeng1/anaconda3/envs/sos/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 542, in execute_request
    user_expressions, allow_stdin,
  File "/Users/bpeng1/anaconda3/envs/sos/lib/python3.7/site-packages/metakernel/_metakernel.py", line 395, in do_execute
    retval = self.do_execute_direct(code)
  File "/Users/bpeng1/anaconda3/envs/sos/lib/python3.7/site-packages/xonsh_kernel/kernel.py", line 45, in do_execute_direct
    out, err, interrupted = self._do_execute_direct(code)
  File "/Users/bpeng1/anaconda3/envs/sos/lib/python3.7/site-packages/xonsh_kernel/kernel.py", line 60, in _do_execute_direct
    shell = builtins.__xonsh_shell__
AttributeError: module 'builtins' has no attribute '__xonsh_shell__'
[W 08:54:56.999 NotebookApp] Replacing stale connection: e021d546-2991-4334-af88-2122dfec84e7:038c76d00aa448c895f11303794d9730
[I 08:55:09.628 NotebookApp] Saving file at /Untitled.ipynb

BoPeng · 2019-10-23T14:06:12Z

@WardThomas Let me know how to fix Xonsh Kernel and I will see what I can do on the SoS side. In the easiest case I only need to add xonsh as one of the supported languages for sos-python using the following patch

(sos) [bpeng1@BCBMF5KWK0F8F69:~/sos/sos-python (master *)]$ git diff
diff --git a/src/sos_python/kernel.py b/src/sos_python/kernel.py
index f299059..0374a2e 100644
--- a/src/sos_python/kernel.py
+++ b/src/sos_python/kernel.py
@@ -119,8 +119,8 @@ def __preview_var(item):
 
 
 class sos_Python:
-    supported_kernels = {'Python3': ['python3'], 'Python2': ['python2']}
-    background_color = {'Python2': '#FFF177', 'Python3': '#FFD91A'}
+    supported_kernels = {'Python3': ['python3'], 'Python2': ['python2'], 'Xonsh': ['calysto_xonsh']}
+    background_color = {'Python2': '#FFF177', 'Python3': '#FFD91A', 'Xonsh': 'red'}
     options = {
         'variable_pattern': r'^\s*[_A-Za-z0-9\.]+\s*$',
         'assignment_pattern': r'^\s*([_A-Za-z0-9\.]+)\s*=.*$',

BoPeng · 2019-10-23T16:25:20Z

Submitted a bug report to upstream Calysto/xonsh_kernel#13

WardThomas · 2019-10-24T17:22:50Z

Hello BoPeng, thanks for following that up

I have manged to get the Calysto/xonsh_kernel to work with Jupyter lab - see post on the Calysto git thread which you started

I then followed your instructions to install SoS
https://vatlab.github.io/sos-docs/running.html#content

and calysto_xonsh works as an SoS sub kernel

I am not familair with SoS yet so I can't test if data exchange etc works,

WardThomas · 2019-10-24T17:47:28Z

Hello BoPeng,

I think data exchange doesn't work yet -see screenshot below

but next week I can try to modify the SoS kernel as you suggested.
I had a brief look at the SoS code just now and saw some other places which might need modifying, for example

packages/lib/codemirror.js

do you think they will need updating as well?

BoPeng · 2019-10-24T18:00:33Z

Did you run

%use calysto_xonsh --language Python3

to specify language?

As I have suggested in another thread, it is better for you to submit a PR to fix calysto_xonsh so that whatever we do here can benefit the entire xonsh community.

WardThomas · 2019-10-29T14:19:14Z

Hi BoPeng

Did you run
%use calysto_xonsh --language Python3

yes I did

it is better for you to submit a PR to fix calysto_xonsh so that whatever we do here can benefit the entire xonsh community.

Yes I will try but I don't even now what "submit a PR", I will work it out but it will take a while before I have the time - don't hold your breath - probably next week

regards

Thomas

BoPeng · 2020-09-17T16:43:46Z

@WardThomas I am closing this ticket since the upstream has not been fixed. I will re-open this ticket if Calysto/xonsh_kernel#13 is fixed.

BoPeng transferred this issue from vatlab/sos Nov 19, 2019

BoPeng closed this as completed Sep 17, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

xonsh Kernel #274

xonsh Kernel #274

WardThomas commented Oct 23, 2019

BoPeng commented Oct 23, 2019

BoPeng commented Oct 23, 2019

WardThomas commented Oct 23, 2019 •

edited by BoPeng

Loading

WardThomas commented Oct 23, 2019

BoPeng commented Oct 23, 2019

BoPeng commented Oct 23, 2019

BoPeng commented Oct 23, 2019 •

edited

Loading

BoPeng commented Oct 23, 2019

WardThomas commented Oct 24, 2019

WardThomas commented Oct 24, 2019

BoPeng commented Oct 24, 2019

WardThomas commented Oct 29, 2019

BoPeng commented Sep 17, 2020

xonsh Kernel #274

xonsh Kernel #274

Comments

WardThomas commented Oct 23, 2019

BoPeng commented Oct 23, 2019

BoPeng commented Oct 23, 2019

WardThomas commented Oct 23, 2019 • edited by BoPeng Loading

WardThomas commented Oct 23, 2019

BoPeng commented Oct 23, 2019

BoPeng commented Oct 23, 2019

BoPeng commented Oct 23, 2019 • edited Loading

BoPeng commented Oct 23, 2019

WardThomas commented Oct 24, 2019

WardThomas commented Oct 24, 2019

BoPeng commented Oct 24, 2019

WardThomas commented Oct 29, 2019

BoPeng commented Sep 17, 2020

WardThomas commented Oct 23, 2019 •

edited by BoPeng

Loading

BoPeng commented Oct 23, 2019 •

edited

Loading