[FEAT] microscopium user interface, export selected data

[GenevieveBuckley]

This pull request supersedes PR #86. When this is merged, that can be closed.

I believe this also closes #80, unless you also want to include the 'nearest neighbors' view like Sarah had in the bokeh prototype here (see issue #73, branch https://github.com/birdsarah/microscopium/tree/bokeh_proto).

New features include:

• Enforced a consistent aspect ratio when using the zoom tools on image plots.
• Add instructions to README on running the web app locally (ie: not using bokeh serve ...)
• Bokeh tooltips added when you hover over
• Export selected data to csv with javascript callback to download_data.js
• Prints web app display to paper or pdf with javascript callback
• If the csv input contains a 'group' column, scatterplot data will have colors assigned to each group, plus a clickable legend to hide or display each group of data.

And a discussion of possible future problems, in order of severity:

1. Some really weird stuff is happening with the bokeh 'save' toolbar button. I've removed it for now and instead included a button that will print the whole webpage to paper or pdf. Details: It works fine to the PCA plot, but is inconsistent for the selected images plot. If you open the Chrome developer Javascript console, then you can (sometimes) magically make it work. It seems that this magical workaround doesn't work so well if you're not using bokeh serve though. I'm also pretty sure it's not entirely related to the next javascript error I'll talk about, because this odd save behaviour still happens when those lines of code are commented out.

2. I get a javascript error related to updating the datatable with the selected data. It doesn't seem like a fatal error, but should still be fixed: error handling message Message 'PATCH-DOC' (revision 1): DeserializationError("Seq(Any) expected a list or None, got {'0': -0.43731416321298383, '1': -0.4933948783766794}",)

3. Javascript doesn't support objects like python tuples, so I couldn't work out how to export dataframe columns containing these to csv (like the 'neighbors' column). So I'm excluding those for now, but I bet someone will want this!

4. It seems that bokeh's datatable doesn't support index values other than the usual 0, 1, 2, ... etc. So even though the pandas dataframe has the index values set to something fancy, we don't see those index values on the webpage or in the csv download of selected data.

5. I haven't chosen to make the table resize itself when the browser is resized, I think that would probably end up looking silly. Let me know if you think I should reconsider. I do still think it's worthwhile having a table of selected datapoints on display, since it's a good visual clue if you haven't selected anything.

[coveralls]

Coverage decreased (-2.7%) to 67.893% when pulling d747c22 on GenevieveBuckley:export-data into 0860e73 on microscopium:master.

[jni]

Hey @GenevieveBuckley! Very cool! I just tested it and ended up with a wonderful little csv in my Downloads folder! 😲 👏 👏 👏 Before I do a full review, some broad overview issues:

1. We need bokeh>=0.13 now, it seems. (This branch crashed with my 0.12.15 install.) So, that needs to be added to requirements.txt and environment.yml.
2. One of my high level goals for the bokeh rewrite is to not have any javascript whatsoever. That may be impossible, but can you check whether the save button can be implemented with a Python callback?
3. When I made my selection (or when I tried to export it, not sure), the console showed this message:

new index:  [6, 3]
ERROR:bokeh.server.protocol_handler:error handling message Message 'PATCH-DOC' (revision 1): DeserializationError("Seq(Any) expected a list or None, got {'0': -0.24282498494377924, '1': -0.3254431042231408}",)

Have you experienced anything like this in your testing?

[GenevieveBuckley]

1. We need bokeh>=0.13 now, it seems. (This branch crashed with my 0.12.15 install.) So, that needs to be added to requirements.txt and environment.yml.

• Done, thanks for the heads up!

2. One of my high level goals for the bokeh rewrite is to not have any javascript whatsoever. That may be impossible, but can you check whether the save button can be implemented with a Python callback?

I'm not sure, maybe? There is pandas.to_csv() but you'd need to supply a filepath & if the code is running on a remote server... then we'd need to know more about sockets & server file transfers.

The print webpage button would not have a good python alternative (and currently the bokeh save toolbar icons aren't working like they should). How much javascript is too much javascript? Is print() ok?

3. When I made my selection (or when I tried to export it, not sure), the console showed this message:

new index:  [6, 3]
ERROR:bokeh.server.protocol_handler:error handling message Message 'PATCH-DOC' (revision 1): DeserializationError("Seq(Any) expected a list or None, got {'0': -0.24282498494377924, '1': -0.3254431042231408}",)

Have you experienced anything like this in your testing?

Yes, this is happening in the update_table() function, but I'm not sure how to debug this:

2. I get a javascript error related to updating the datatable with the selected data. It doesn't seem like a fatal error, but should still be fixed: error handling message Message 'PATCH-DOC' (revision 1): DeserializationError("Seq(Any) expected a list or None, got {'0': -0.43731416321298383, '1': -0.4933948783766794}",)

[GenevieveBuckley]

I think that error also appears first in this list here: https://rollbar.com/blog/top-10-javascript-errors/
but I'm not sure if reading that will help us debug later on.

Also maybe this will be helpful? https://javascriptweblog.wordpress.com/2010/08/16/understanding-undefined-and-preventing-referenceerrors/

[GenevieveBuckley]

And here are some links about sockets (file transfer using python only, no downloading with javascript):

• https://pythongeekstuff.wordpress.com/2015/07/29/file-transfer-server-using-socket/
• https://www.bogotobogo.com/python/python_network_programming_server_client_file_transfer.php

[jni]

@GenevieveBuckley 🎉 Awesome work here. Saving was a pain in the ass to get right but it's such a valuable feature, thanks for pushing it forward!!! And lots of other enhancements that I now take for granted. =)

[GenevieveBuckley]

Nice work! It is much cleaner now, especially the download feature :)

You know, we should update the bokeh export data example to be more like what you've done here. I know you're swamped, if I have spare time in the next few weeks I might try to do that. It'd be good to save some other poor sod all the hassle!

[GenevieveBuckley]

Turns out we could probably use legend.click_policy = 'mute' if we do some extra manual setting of muted_color and muted_alpha (there currently aren't good defaults for these kwargs set in bokeh, as discussed here bokeh/bokeh#8390 (comment))