New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial support for D3.js and vega.js plots in the IP notebook. #2120

Closed
wants to merge 1 commit into
base: master
from

Conversation

Projects
None yet
4 participants
@Krastanov
Member

Krastanov commented May 17, 2013

This builds up on the prototype that I presented in a blog post.

It works only in IPython notebook. I have added a notebook with examples.

The notebook must be evaluated for the plots to appear (due to the hacking around with publish_javascript).

There are rumors that this same publish_javascript is getting deprecated and a new _repr_json_ is being developed. In any case, this will be known in the second half of 2013.

Sometimes it is necessary to evaluate a cell twice to get the plots (again due to the hacks with publish_javascript).

It works only for lines (plot and plot_parametric), Custom aesthetics (like color dependent on coordinates) do not work yet, nor most of the options, but they would be very easy to add.

Hovering over a line makes it bigger so one can see it beneath others if the plot contains a lot of information.

Do you want to merge this as it is, or would you prefer that we wait for IPython to implement the necessary dependencies.

screenshot from 2013-05-17 22 16 19
screenshot from 2013-05-17 22 15 41

@asmeurer

This comment has been minimized.

Member

asmeurer commented May 17, 2013

Let's see if we can get some @ipython guys to look this over.

@asmeurer

This comment has been minimized.

Member

asmeurer commented May 17, 2013

(I don't know if @ipython actually pings them. If it doesn't seem to, just ask on their mailing list)

@asmeurer

This comment has been minimized.

Member

asmeurer commented May 17, 2013

You need to set the backend in the notebook or something. It just opens plots in new windows for me.

@asmeurer

This comment has been minimized.

Member

asmeurer commented May 17, 2013

Also, if the fully rendered notebook isn't that big, just push that. Then we can view it with nbviewer.

@Krastanov

This comment has been minimized.

Member

Krastanov commented May 17, 2013

Indeed you need to change the backend (there is "officially" no way for the python kernel to know that it is evoked from the notebook frontend).

See in http://nbviewer.ipython.org/urls/raw.github.com/Krastanov/sympy/08d7872df47005bac295d9ae6562b5fe59513b53/examples/beginner/plotting_with_d3js.ipynb for how to change the backend.

This notebook is actually the rendered version, however as I mentioned above, due to the hacks done to inject javascript in the notebook one has to reevaluate it in order to render the javascript plots.

There will be canonical nonhackish tools for doing this in the post-1.0 version of IPython (_repr_json_ I believe).

@asmeurer

This comment has been minimized.

Member

asmeurer commented May 17, 2013

Am I missing something? That's the same notebook in this pull request, right? When I executed it, I didn't get d3 plots.

@asmeurer

This comment has been minimized.

Member

asmeurer commented May 17, 2013

Why the [::-1]? Just write the tuple backwards.

@asmeurer

This comment has been minimized.

Member

asmeurer commented May 17, 2013

Or write one (x*sin(x), x*cos(x)) and a little function to compute all the combinations of it.

@Krastanov

This comment has been minimized.

Member

Krastanov commented May 17, 2013

If you are doing the plotting_backends['default']=... you are not missing anything. If it does not work it is some problem with the pull request or maybe with the ipython version (I am on 0.14.dev from one or two months ago).

Are there any errors? As I mentioned above sometimes (the first time) it is necessary to execute the cell twice because of problems with the javascript injection.

And about the plot command and [::-1] - I was copy pasting while testing and this was the quickest thing to write/copy. I should change it if it is going to be a part of the examples.

@asmeurer

This comment has been minimized.

Member

asmeurer commented May 17, 2013

I am not doing anything. Just executing the notebook.

I am using IPython 0.13.2. I can't get the newer pyzmq installed for IPython dev to work.

There are no errors. It just gives the plot in a window instead of using d3.

@asmeurer

This comment has been minimized.

Member

asmeurer commented May 17, 2013

OK, I guess there is an error

KeyError                                  Traceback (most recent call last)
<ipython-input-3-43418bee1652> in <module>()
----> 1 plot_backends['default'] = plot_backends['ipythonD3']

KeyError: 'ipythonD3'
@asmeurer

This comment has been minimized.

Member

asmeurer commented May 17, 2013

OK, I think that it is not picking up the git sympy because it is running the notebook from the examples directory. Let me play with it.

@asmeurer

This comment has been minimized.

Member

asmeurer commented May 17, 2013

OK, it does work. As you noted, I had to reexecute each cell two or three times to get it to show.

@Krastanov

This comment has been minimized.

Member

Krastanov commented May 18, 2013

I have played around trying to fix the issue with the required reevaluations. There is no way to do it from within the plotting module as far as I know.

It can be done in a manner similar to what we do for printing but this seems invasive and contrary to the _repr_*_ methodology of IPython.

So there are three options:

  • accepting this as it is with its flaws (which will disappear in IPython 1.0+)
  • waiting for 6 months for IPython 1.0+ which will support _repr_json_.
  • hacking up some solution now that would create a lot of technical debt in IPython 1.0+
@asmeurer

This comment has been minimized.

Member

asmeurer commented May 18, 2013

Supposing we release SymPy before IPython 1.0, should we hold off this fix (i.e., don't put this in an actual release until it is done right with IPython 1.0)? I doubt we will do more than one release before IPython 1.0.

@Krastanov

This comment has been minimized.

Member

Krastanov commented May 18, 2013

OK. I will wait a bit for any answers from the IPython team (I asked on their mailing list), and them probably I will close this PR and create an issue tracking the progress on the IPython side.

@Krastanov

This comment has been minimized.

Member

Krastanov commented May 18, 2013

Sorry, I misread and did not notice it was actually a question. If we release before 1.0 with this pull request we will have working but flawed d3 plotting (the need to evaluate the first cell twice or a hackish workaround for that need). As my previous comment alluded, I think it is more appropriate to wait for it to be "done right".

@Carreau

This comment has been minimized.

Contributor

Carreau commented May 19, 2013

Hey,
So no @ipython don't ping us, or at least not me.
I personally would suggest waiting a little for a few reason, you already underlined some of them.

  1. we will at some point activate output sanitizing which will strip 'script' and maybe 'style' tag from displayed HTML.
    display_js will still work though. Not sure this will be done for 1.0 though.

  2. This make the d3 plot incompatible with nbviewer, indead the display_js is not stored/not executed a load time in the notebook, which mean that the graph will be lost across session and on nbviewer. using a script tag in repr_html go around that, but will be disable by 1.

  3. With js plugin etc, you will with high probability not have to worry about injecting D3/other library, and the plugin will most likely be availlable on nbviewer.

Note that we'll probably have a meeting about that and other things at end of July. We'll release a more detailed roadmap then.

@Carreau

This comment has been minimized.

Contributor

Carreau commented May 19, 2013

Oh, another comment, try to avoid as much as possible the use of display_foo/publish_foo
try to return an object that have different repr when possible (or call display on it). Even if only 1 repr is show in the notebook all representation are stored and used by nbviewer/nbconvert

So you can display js in the notebook, and return PNG so that when converting to PDF for example PNG is used.

@Krastanov

This comment has been minimized.

Member

Krastanov commented May 21, 2013

@asmeurer, I am closing this for the moment. An issue is created here https://code.google.com/p/sympy/issues/detail?id=3841

@Krastanov Krastanov closed this May 21, 2013

@asmeurer

This comment has been minimized.

Member

asmeurer commented May 21, 2013

Hopefully you can reuse most of the code when this is doable the right way.

@Krastanov

This comment has been minimized.

Member

Krastanov commented May 21, 2013

The only part that deals with IPython is publish_... The change will be a straightforward switch to _repr_*.

@z-m-k

This comment has been minimized.

z-m-k commented Jun 7, 2013

For static things you can also do external rendering as I did here: http://nbviewer.ipython.org/4484816/ipyD3sample.ipynb

@Carreau

This comment has been minimized.

Contributor

Carreau commented Jun 7, 2013

Le 7 juin 2013 à 14:16, z-m-k a écrit :

For static things you can also do external rendering as I did here: http://nbviewer.ipython.org/4484816/ipyD3sample.ipynb

The 1) is not a problem :

def x():
from IPython.core.display import Javascript
Javascript('alert("a")')
x()

would you expect

def x():
a=1
a
x()

to produce output ?

would you expect
def x():
a=1
return a # add a return
x()

to produce output ?

would you expect

def x():
a=1
return a # add a return
x()
pass

to produce output ?

replace a by HTML/Javascript and you get your answer.

display() is the equivalent of print

Matthias


Reply to this email directly or view it on GitHub.

@z-m-k

This comment has been minimized.

z-m-k commented Jun 7, 2013

This is beyond the point there... It is how I got started when I did not know of display(). What it does there is takes the js/css you construct in notebook, starts a PhantomJs instance lets it render it and displays the final html with styling. So like I said, if you just want to use d3js for static (not necessarily basic) graphs than you do not have to worry about publish_javascript being depreciated. The benefit is that you do not have to rerun anything on refresh.

@Carreau

This comment has been minimized.

Contributor

Carreau commented Jun 7, 2013

Yes, I understand, I was just underlining the point you wrote in obstacle 1:

I cite :

Of course, just like Javascript(), HTML() does not work from withing a function as well.

Which is completly false, it does work as expected. It would be like writing the following


"you can't use addition in a function", example:

def myAdd(a,b):
   c=a+b
   c
myAdd(1,2)

"return nothing, so addition are not working in function. Wherease"

In [1]: a=1
   ...: b=2
   ...: c=a+b
   ...: c
   ...: 
Out[1]: 3

Which is a completly false example as there is no return statement.

This add to the confusion that Javascript is to display javascript even is it is not.

You would have write your first example with return:

def x():
    from IPython.core.display import Javascript
    return Javascript('alert("a")')
x()

It would have worked.
unless you added statement after the x() call.

And by the way, Javascript support more argument like passing libs, and those libs would be loaded before actually running the JS. So I'm not sure about the "premature" execution argument... but I don't know how d3 works, and this is a real nice example.

I'm just trying to point the real difference there is between the "display" and the "output" of the cell. it is not something easy to get, especially because lots of example on the internet convey False truth.

IMHO, we should really tell more to people that "display" is "print" and that "Javascript"/"Html" are equivalent of "String", so that returning an object from a function is different from printing it.

I would say that I'm maybe a little biased right now, because I'm working in nbconvert, and I see the problem when people use display_something() instead of returning, because then in some case you cannot totally convert the notebook to (some) other formats.

@asmeurer

This comment has been minimized.

Member

asmeurer commented Aug 6, 2013

SymPy Bot Summary: ❗️ There were merge conflicts (could not merge Krastanov/d3plots (08d7872) into master (a09f82e)); could not test the branch.
@Krastanov: Please rebase or merge your branch with master. See the report for a list of the merge conflicts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment