Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Jupyter (IPython 3) kernel spec #224

Closed
stevengj opened this issue Aug 22, 2014 · 40 comments · Fixed by JuliaLang/METADATA.jl#2191
Closed

Support Jupyter (IPython 3) kernel spec #224

stevengj opened this issue Aug 22, 2014 · 40 comments · Fixed by JuliaLang/METADATA.jl#2191
Assignees

Comments

@stevengj
Copy link
Member

This should be a very simple matter of writing out a couple of extra files to the .ipython directory.

cc: @ellisonbg, @Carreau

@Carreau Carreau self-assigned this Aug 22, 2014
@Carreau
Copy link
Contributor

Carreau commented Aug 22, 2014

Yep, that's in progress in #210 we'll need a few more hooks in IPython for IJulia custom javascript.

Assigning to me. Wont have time I guess to do it for your keynote, but see you at EuroScipy anyway.

@Carreau
Copy link
Contributor

Carreau commented Aug 22, 2014

Technically speaking the hook missing in IPython/Jupyter are

  • transparently handle language in MD
  • Patch un-patch various ,things (implementation is in my head)
  • switch Logo
  • Append custom css.
  • Switch favicon

@stevengj
Copy link
Member Author

Ah, I forgot about #210.

Support for switching the logo should really happen on the Jupyter side; @ellisonbg said this was planned. I don't think it's necessary to switch the favicon as long as the favicon is for Jupyter and not Python.

What is missing for handling the language in the metadata? Isn't this already in Jupyter?

@Carreau
Copy link
Contributor

Carreau commented Aug 22, 2014

Most of current kernels to add a metadata.language field to the notebook, we do not (yet) use this info to switch to know defined kernelspec created by @takluyver. This mean the the IJulia user would have to select manually "Julia" at least once for each of their notebook and then re-save. (which I don't think is acceptable).

We also do need a way to "un-monkeypatch" the javascript. Eg, I haskell patch for tooltip to show on space. if you switch "Python"->"Haskell"->"Julia" this would lead to tooltip triggering on space for julia too.

I propose that the kernelspec custom.js be a module that define (at least) to methods, lets call them patch and unpatch. When switching to Julia julia.custom.patch(args) is guarantied to be called to patch/configure what it wants. When switching "away" from Julia julia.custom.unpatch(args) is called and should return patched things to the original state. This is not (yet) in IPython Jupyter.

If there is a custom.css file, it is not yet either injected in the notebook.

@stevengj
Copy link
Member Author

The custom Javascript patches that we currently use, if I remember correctly, were mainly:

  • Loading and using the proper Codemirror mode. Shouldn't this be in Jupyter?
  • Changing the tooltip code to recognize Julia identifiers. This seems obsoleted by the new inspect_request protocol, because now IPython will just send us the entire code cell and let us decide on the relevant identifier ourselves.

I feel like whether the tooltip is triggered by space or shift-tab or whatever should be a global keybinding preference, not a per-language thing. I don't see why this is language-specific.

@stevengj
Copy link
Member Author

@Carreau, regarding adding and using the metadata.language field, is there a Jupyter issue for this? This is not something the kernel should need to hack Javascript for.

@Carreau
Copy link
Contributor

Carreau commented Aug 22, 2014

Loading and using the proper Codemirror mode. Shouldn't this be in Jupyter?

Yes it is

Changing the tooltip code to recognize Julia identifiers. This seems obsoleted by the new inspect_request protocol, because now IPython will just send us the entire code cell and let us decide on the relevant identifier ourselves.

Not sure the tooltip code has been updated to use that yet. But you are right

I feel like whether the tooltip is triggered by space or shift-tab or whatever should be a global keybinding preference, not a per-language thing. I don't see why this is language-specific.

Haskell don't use parents. in python or julia ( will trigger a pending request that pop up the tooltip if nit key is pressed in XXX ms. In haskell you want the same behavior on map $ <cursor waiting> shift tab will still trigger. F# also trigger on every . as they do purely static completion and are sure of not having side effeect

regarding adding and using the metadata.language field, is there a Jupyter issue for this?

No, but it's on the list of things we know we should do.

This is not something the kernel should need to hack Javascript for.

Yes agreed

@Carreau
Copy link
Contributor

Carreau commented Aug 22, 2014

No, but it's on the list of things we know we should do.

My bad, yes there is ipython/ipython#6217

@StefanKarpinski
Copy link
Sponsor Member

Since the plan, as I understand it, is to serve multiple different types of notebooks from a single Jupyter server, each according its kind, and favicon's are per-server, I don't think that changing favicons even really makes sense.

@stevengj
Copy link
Member Author

@StefanKarpinski, actually it is a bit more ambitious than that. @ellisonbg showed me a demo today in which there was a menu item on the notebook allowing you to switch languages inside a single notebook.

(Unfortunately, switching languages kills the old kernel. So, if you switch from Julia to Python and then back to Julia, you lose all of your Julia variables. Also, "run all" no longer works because Jupyter doesn't know which cells were in which languages. So, I'm skeptical that this feature is particularly useful, as opposed to having a fixed per-notebook kernel/language.)

@StefanKarpinski
Copy link
Sponsor Member

Hmm. That seems kind of pointless. @Carreau, @ellisonbg, what's the motivation?

@minrk
Copy link
Contributor

minrk commented Aug 23, 2014

I don't think the kernel is likely to be a function of time for a given notebook, but we need UI somewhere to select the kernel for a given notebook. It's currently on the notebook page itself. It's not likely to be changed much more often than at notebook creation time, even though the UI as it is technically allows people to fiddle with it in ways that don't make much sense.

@stevengj
Copy link
Member Author

@minrk, wouldn't it make more sense for the UI to be on the dashboard? e.g. add a drop-down menu by "new notebook" to set the current kernel preference.

The problem with allowing people to change it while editing the notebook is that you then need to support that functionality, even if it doesn't make much sense. For example, see @Carreau's comments above about having to support both patching and unpatching of the Javascript, which seems like unnecessary pain. And I'm guessing that this is only one of many implementation annoyances.

@ellisonbg
Copy link

I agree we need to rethink the UI - I have been demoing this all week and
Everyone I show it to is horribly confused about how the kernel state
relates to the ntoebook state.

On Fri, Aug 22, 2014 at 9:12 PM, Steven G. Johnson <notifications@github.com

wrote:

@minrk https://github.com/minrk, wouldn't it make more sense for the UI
to be on the dashboard? i.e. add a drop-down menu by "new notebook".


Reply to this email directly or view it on GitHub
#224 (comment).

Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger@calpoly.edu and ellisonbg@gmail.com

@minrk
Copy link
Contributor

minrk commented Aug 23, 2014

@stevengj sure, that's a possibility. It would need to be triggered somehow in the notebook UI anyway, as part of the 'New Notebook' action in the File menu, or as a pop-up when you open a notebook with no associated kernel.

Plus, it still makes sense to be able to switch the kernel at some point, because one could reasonably have several Python kernels or several Julia kernels available. So switching kernels is definitely required.

@Carreau
Copy link
Contributor

Carreau commented Aug 23, 2014

Agreed with min, we had the discussion of changing only at creation time, or using a "copy this notebook and make it XXX". let's say you got an assignment it requires you to copy the notebook. Update from Julia 0.2 to Julia 0.3 (cause you are a dev), make a copy. In terminal it makes sens to be able to do julia setup.py [even if it does not work :-)]

The reasoning is we want first an "indicator" of the language, while we are relatively advance user that can in less than 100ms distinguish Julia for perl, from C, from Python 2 or 3 code, most user cannot. So there is an indicator. And if you ever want to switch the most natural place is this indicator.

After we can make the fact that it restart the kernel more obvious by having the switch also clear all output and scroll to top.

@Carreau
Copy link
Contributor

Carreau commented Aug 23, 2014

Since the plan, as I understand it, is to serve multiple different types of notebooks from a single Jupyter server, each according its kind, and favicon's are per-server, I don't think that changing favicons even really makes sense.

Yes once we have a Jupyter favicon, people actually using IJulia now would be confuse to have an IPython icon, but agree low priority (and might not even works on all browsers)

@stevengj
Copy link
Member Author

@Carreau, allowing people to change kernels in a live notebook, with all of the complexities that entails (e.g. unpatching) is a high price to pay just to have an indicator. Why not just have a static indicator? (In addition to logo changes.)

@minrk, the reason I suggested a drop-down menu next to the "New Notebook" button was that most users will probably be using the same language most of the time, so it is more efficient for them to just set the preference once and then be able to click "new notebook". Having a pop-up (or your current scheme), in contrast, imposes a cost every single time you create a new notebook. (And yes, I know there is a config file where you can change this, but forcing users to edit a config file is also a high cost.)

@Carreau
Copy link
Contributor

Carreau commented Aug 23, 2014

the reason I suggested a drop-down menu next to the "New Notebook" button was that most users will probably be using the same language most of the time, so it is more efficient for them to just set the preference once and then be able to click "new notebook".

There will be a way to set the default. Then "New notebook" will always behave without having the user to switch.

The problem with switching from dashboard is that it adds a strong coupling between dashboard and notebook page, which shouldn't be much easier that unpatching.

If unpatching becomes too hard, we can still have the change of kernel reload the page after saving the notebook, but unpatching shouldn't be much harder than saving a copy of the function at patch time and replace with the old one after. Also reload would require saving the notebook, and I woudl prefer people to be able to test different languages without saving files.

@stevengj
Copy link
Member Author

@Carreau, @ellisonbg already told me there will be a way to set a default by changing some config file somewhere, but that's not good enough. Not only is this painful from a usability perspective—most users will not want to edit a config file manually, and in fact will not even be aware of config files (they are not discoverable)—but it is also doesn't address the case where I'm working with Python on Thursday and Julia on Friday, and I want to change the default easily without restarting Jupyter. And it might not even be possible to edit the config file if I'm running Jupyter on a remote server (authenticated via webcerts) and viewing it locally in my browser.

Switching from the dashboard does not add a strong coupling between the dashboard and notebook. My proposed switch in the dashboard only affects new notebooks that are opened subsequently. That's why it's next to the "new notebook" button. The whole point (well, part of the point) of this is to avoid changing the kernel in a running notebook.

I know that unpatching is technically possible, but you are adding a whole level of complexity for no useful purpose. (Not to mention that I think unpatching will be only the beginning of our troubles when we start to use this in practice for complicated notebooks that have lots of embedded JS in output cells.)

@Carreau
Copy link
Contributor

Carreau commented Aug 23, 2014

Oh, but we will have "new notebook" with drop down in the dashboard at some point. We just have a limiting manpower willing to work on JS side.

Envoyé de mon iPhone

Le 23 août 2014 à 15:41, "Steven G. Johnson" notifications@github.com a écrit :

@Carreau, @ellisonbg already said there will be a way to set a default, by changing some config file somewhere. Not only is this painful from a usability perspective—most users will not want to edit a config file manually—but it is also doesn't address the case where I'm working with Python on Thursday and Julia on Friday, and I want to change the config easily without restarting Jupyter.

Switching from the dashboard does not add a strong coupling between the dashboard and notebook. My proposed switch in the dashboard only affects new notebooks that are opened subsequently. The whole point (well, part of the point) of this is to avoid changing the kernel in a running notebook.


Reply to this email directly or view it on GitHub.

@rgbkrk
Copy link
Contributor

rgbkrk commented Nov 25, 2014

Header logo and favicon for now is at least Jupyter rather than IPython: ipython/ipython#7004, ipython/ipython#7031

@stevengj
Copy link
Member Author

Okay, no problem.

@rgbkrk
Copy link
Contributor

rgbkrk commented Nov 25, 2014

However, the kernel logo is going to be part of the kernel spec. The layout is still TBD. ipython/ipython#6537

@stevengj
Copy link
Member Author

stevengj commented Feb 2, 2015

@Carreau, since ipython/ipython#6537 was merged, is there anything else holding this up?

@Carreau
Copy link
Contributor

Carreau commented Feb 2, 2015

@Carreau, since ipython#6537 was merged, is there anything else holding this up?

Me getting time to finis #210 among other. I should be definitively in berkeley in something like 48h.
probably will take a day or two to get office space and badges, and I resume my work.
(prey for delta airline to get power outlets, and I'll geek in the plane)

@stevengj
Copy link
Member Author

stevengj commented Feb 2, 2015

Cool, thanks! So we are not waiting on any changes on the IPython side anymore.

@Carreau
Copy link
Contributor

Carreau commented Feb 2, 2015

Except for last minute raptor, no.

@jiahao
Copy link
Member

jiahao commented Feb 23, 2015

It looks like both 0.3.6+ and 0.4-dev work quite nicely with 3.0.0rc1:

screen shot 2015-02-22 at 9 14 12 pm

Really liking the improved scrolling behavior!

@jiahao
Copy link
Member

jiahao commented Feb 23, 2015

It looks like the only thing left to do is to fix this deprecation warning:

/Library/Python/2.7/site-packages/IPython/kernel/manager.py:95: UserWarning: Setting kernel_cmd is deprecated, use kernel_spec to start different kernels.

Otherwise it looks like all systems go.

@Carreau
Copy link
Contributor

Carreau commented Feb 23, 2015

Just don't use --profile=julia and you are good to go.
(unless you get some other custom things in your profile)

@jiahao
Copy link
Member

jiahao commented Feb 23, 2015

I don't need --profile=julia anymore??!! (mind blown)

@Carreau
Copy link
Contributor

Carreau commented Feb 23, 2015

We do out best so that you do not have to:

capture d ecran 2015-02-22 a 18 37 12

Still a bit of work, but all languages should be equal citizen soon.

@jiahao
Copy link
Member

jiahao commented Feb 23, 2015

I would really like to have both Julia 0.3 and 0.4 accessible from Jupyter. Is that possible now or is that still WIP?

@Carreau
Copy link
Contributor

Carreau commented Feb 23, 2015

I would really like to have both Julia 0.3 and 0.4 accessible from Jupyter. Is that possible now or is that still WIP?

I guess changing this line to install in julia${major_version} instead of only Julia should work so that you get 2 kernelspecs. That how we do it for python.

@jiahao
Copy link
Member

jiahao commented Feb 23, 2015

I'll give it a try.

Sadly, 3.0 breaks one of my favorite packages, @shashi's Interact.jl for accessing IPython widgets:

screen shot 2015-02-22 at 9 57 28 pm

but I suppose this was inevitable.

@Carreau
Copy link
Contributor

Carreau commented Feb 23, 2015

:-( Sorry about that, That's also why we split in Jupyter, we hope to have non-python contributor, at least in the governance of the project, to minimally know what the breakage in differents kernels might be and get early feedback.

Side question as you are around: are you aware that we have https://try.jupyter.org ?

@jiahao
Copy link
Member

jiahao commented Feb 23, 2015

Indeed, making essentially that change and rebuilding Pkg.build("IJulia") on both julia versions produces the desired result:

screen shot 2015-02-22 at 10 16 18 pm

Side question as you are around: are you aware that we have https://try.jupyter.org ?

Not until you mentioned it!

@Carreau
Copy link
Contributor

Carreau commented Feb 23, 2015

Not until you mentioned it!

Well, it's still in progress it spawns Docker container without network access
that are torn don after 2 hours for people to try (Hosted graciously by rackspace).
If you like to add Julia demo, that would be great. And if Julia community would like
to help with it that would be great.

@jiahao
Copy link
Member

jiahao commented Feb 23, 2015

You're welcome to use all the notebooks I have posted publicly at

https://github.com/jiahao/ijulia-notebooks

and

https://github.com/jiahao/ijulia-notebooks-assorted

jiahao added a commit that referenced this issue Feb 23, 2015
This change allows multiple Julia versions to be registered as
IPython/Jupyter 3.0 kernels.

You will get entries like

- Julia 0.3.6
- Julia 0.4.0-dev
- Julia 0.4.0-dev debug

depending on which Julia binary is used to build IJulia.jl.

To get multiple Julia versions available as IPython kernels, you will
need to rebuild IJulia from within each Julia binary.

Ref: #224
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants