Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor WebAgg so it can communicate over another web server #2420

Merged
merged 3 commits into from Sep 30, 2013

Conversation

mdboom
Copy link
Member

@mdboom mdboom commented Sep 13, 2013

This is to address an immediate need we have at STScI, but probably needs some explanation.

We want to embed interactive WebAgg plots inside of a larger web application, but we don't necessarily want to serve the matplotlib websocket or http requests using tornado. We'd like to, instead, allow people to "bring their own webserver framework" to the party and have matplotlib communicate over that. (@stsci-sienkiew). The requirement for "websockets" specifically is still there.

This refactor should also make it easier to move to IPython's future widget communication pipeline down the road (@ellisonbg).

The easiest way to see how this works is to look at the embedding_webagg.py example, which embeds a plot in an example tornado application. I'm using tornado here for the example, even though that's what the "built-in" matplotlib server uses because it's really the only viable and mature Python tool for websockets I could find. It still makes for a useful example for how to embed in a larger Tornado application which might be doing many other things. I plan to write proper "how to" documentation for this, but thought I should hammer down the details first.

As part of this, the Javascript API has all been moved under a single "mpl" namespace, so that it should be easier to import it alongside other Javascript libraries without fear of name clashes. If anyone is currently using the Javascript API to matplotlib to insert plots on their webpages this will break that. As this stuff is all still new and marked "experimental", I'd rather just break this now to get that right rather than carry around the baggage of not namespacing things correctly to begin with.

Cc: @pelson, @andreabedini (as people who have helped with and been interested in WebAgg).

@pelson: You may also enjoy that, ignoring the example, this is a "net deletion" pull request.

@@ -1,6 +1,15 @@
"""
Displays Agg images in the browser, with interactivity
"""
# The WebAgg backend is divided into two modules:
#
# - `backend_webagg_core.py` contains code necessary to embed a WebAgg
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a certain je ne sais quoi missing here, namely the backend_webagg_core code 😊

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doh! Commit has been amended.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excuse my web-programming ignorance, but does this mean that it will be
possible to use WebAgg in a flask application served up via mod_wsgi in
apache? Asking for a friend...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If such a configuration supports websockets, yes, at least that's the idea. Due to my own web-programming ignorance, I won't guarantee it, but it would be great to have your friend's assistance working out any kinks or shortcomings to this approach.

@pelson
Copy link
Member

pelson commented Sep 13, 2013

You may also enjoy that, ignoring the example, this is a "net deletion" pull request.

I'm always a fan of those, though I fear that conclusion is a result of backend_webagg_core.py not having been added 😄.

In principle I'm happy about this refactor, looks like a sane way of going about making the webagg solution better and more re-usable.

@pelson
Copy link
Member

pelson commented Sep 13, 2013

Just tried out the example - the image is a little small in Firefox 17.6 (the status update is too big) and it seems much more laggy on localhost than it used to?
Also, I think the example should probably print something to the standard out to tell you to visit the necessary URL.

@mdboom
Copy link
Member Author

mdboom commented Sep 13, 2013

@pelson: I've addressed your easy comments.

As for the lag -- the timing is handled differently here than in the built-in server. I was trying to eliminate the dependence on having a scheduler in the server. Rather than scheduling redraws on idle, it just throttles the framerate. I tweaked it until it felt approximately the same for me, but I suspect it's a little to tied to how things work for me. I might experiment with trying to the scheduling on the Javascript side, which would also eliminate needing any special features in the server.

@mdboom
Copy link
Member Author

mdboom commented Sep 13, 2013

Consider this experimental -- I have attached another approach that's completely timerless. It has the browser control when new frames should be fetched. It seems to perform much better locally. I haven't been able to tunnel into the office to simulate running over a slow internet link, but obviously we should ensure this works there, too.

@ellisonbg
Copy link

Thanks for the ping. @minrk has opened a PR that starts to implement IPython's new message spec for widgets. It is too early to start integration, but we would love to get your eyes on this soon. Overall the messaging is very similar to WebSockets - basically micro comm channels over one WebSocket.

In terms of the requirements for this approach as @mdboom mentions anyone implementing a compliant server would need to have WebSocket support. This means you need to use an asynch framework such as tornado, twisted, gevent, etc. Flask will definitely not work.

@WeatherGod
Copy link
Member

Ah, thanks for the clarification, Brian. Like I said, I am -- err, I mean
-- my friend is very inexperienced with web development and has just been
blindly working with very simple examples. He certainly would have never
noticed such an important detail for quite a while. Any suggestions for a
web framework that could be used through Apache (since my friend's
workplace would likely never approve any other web server)?

@ellisonbg
Copy link

Apache is the worst possible choice at this point. Apache does not know
how to proxy websockets as far as I know. From this post:

http://serverfault.com/questions/290121/configuring-apache2-to-proxy-websocket

is looks like there are some work arounds, but I would probably not advise
going in that direction. The servers that have good WebSocket proxy support
are nginx and HAProxy. node-http-proxy also has nice support for
WebSockets. But these will still require using something like tornado,
gevent or twisted for the Python side of things. Not good news for your
friend...

On Fri, Sep 13, 2013 at 10:54 AM, Benjamin Root notifications@github.comwrote:

Ah, thanks for the clarification, Brian. Like I said, I am -- err, I mean
-- my friend is very inexperienced with web development and has just been
blindly working with very simple examples. He certainly would have never
noticed such an important detail for quite a while. Any suggestions for a
web framework that could be used through Apache (since my friend's
workplace would likely never approve any other web server)?


Reply to this email directly or view it on GitHubhttps://github.com//pull/2420#issuecomment-24412522
.

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

@minrk
Copy link
Contributor

minrk commented Sep 13, 2013

I haven't tested it, but I think apache can proxy websockets now.

sends a "draw" message to the browser.  If the browser is not already
waiting on another frame, it sends a "draw" message back to the server
which then returns an actual frame.  The nice thing about this approach
is that it's completely timerless.
mdboom added a commit that referenced this pull request Sep 30, 2013
Refactor WebAgg so it can communicate over another web server
@mdboom mdboom merged commit 9804c5d into matplotlib:master Sep 30, 2013
@mdboom mdboom deleted the webagg-factor branch August 7, 2014 13:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants