Plkmapper #74

empet · 2018-03-25T15:01:57Z

Here are my commits on kmapper and visuals. The module plotlyviz is new. It requires igraph and plotly.
The folder plotly-examples contains jupyter notebooks that illustrate how to generate plotly plots of kmapper graphs.
The notebook I posted as a gist one week ago is no more valid, because meanwhile I changed the function plotly_graph.

In the definition of the class KeplerMapper I modified only the visualize method:

When it is called with path_html = None, then it will return the dict json_graph and the string meta;
the string meta is in this case defined within the new function format_meta_plotly.

In visuals the following changes were performed:

defined the function format_meta_plotly
the function dict_to_json has a new argument, path_html. When path_html is None this function
returns the dict json_dict, otherwise it works as before.
-_format_tooltip has also the argument path_html. When it is None the tooltip is formatted following plotly rules;

I didn't create a version for color distribution, because I associated a colorbar to the graph plot.
(the colorbar points out how the values from node['color'] are mapped to the Plotly colorscale).
Here I used a continuous colorscale, while the color_function_distribution maps values to a set of discrete colors (am I right?)

Also I haven't defined yet a plotly version for _format_cluster_statistics.

The new module plotlyviz contains three functions:

get_plotly_data() that returns the lists of x, y-coordinates of the graph nodes (Xnodes, Ynodes),
and the x, y--coordinates of the end nodes of graph edges (Xedges, Yedges);
plotly_graph() that defines a graph as an igraph.Graph() instance, associates a graph layout,
i.e. a planar position for each node, using either the Kamada-Kawai or Fruchterman Reingold layout algorithm.
To each node one associates the attributes size and color. Color is a list of numerical values computed by a custom or default color_function.

All graph features are used to define the dicts, edges_trace, nodes_trace. The list of these dicts represents the Plotly data to plot
the graph.

The function plot_layout returns the dict that defines the Plotly plot layout.

Update with info related to Plotly plot

Inserted a line

sauln

Overall, it looks good. There are a few minor things that need fixing and then we can merge the PR.

The README hasn't been changed, undo these wholesale changes.
Remove .ipynb_checkpoints from commit.
Run all the code through a linter.
Fix the failing test.

sauln · 2018-03-27T23:02:43Z

notebooks/.ipynb_checkpoints/KeplerMapper Newsgroup20 Pipeline-checkpoint.ipynb

@@ -0,0 +1,220 @@
+{


Please remove these checkpoints files.

Removed the .ipynb_checkpoints.
README.md has been changed, because I added igraph and plotly to Kepler-Mapper requirements.

sauln · 2018-03-27T23:06:22Z

kmapper/plotlyviz.py

@@ -0,0 +1,116 @@
+import igraph as ig


Could you please run this file through a Python linter. There are many PEP8 issues.

Could you please format the comments for each function using Numpy comment specs

https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt

sauln · 2018-03-27T23:18:12Z

kmapper/visuals.py

        json_dict["nodes"].append(n)
    for i, (node_id, linked_node_ids) in enumerate(graph["links"].items()):
        for linked_node_id in linked_node_ids:
            l = {"source": node_id_to_num[node_id],
                 "target": node_id_to_num[linked_node_id],
                 "width": _size_link_width(graph, node_id, linked_node_id)}
            json_dict["links"].append(l)
+    if path_html is None:


I'm not a fan of multiple return statements. Can we consolidate these? Maybe seomthing like

return json_dict if path_html is none else json.dumps(json_dict)

sauln · 2018-03-27T23:18:38Z

kmapper/visuals.py

@@ -119,8 +137,7 @@ def _format_cluster_statistics(member_ids, inverse_X, inverse_X_names):

        stats = sorted([(s, f, i, c, a, v) for s, f, i, c, a, v in zip(std_m,
                                                                       inverse_X_names,
-                                                                       np.mean(
-                                                                           inverse_X, axis=0),


sauln · 2018-03-27T23:22:43Z

plotly-examples/Plotly-digits-kmgraph.ipynb

@@ -0,0 +1,221 @@
+{


All of these notebooks are the same as the normal example, right?

Let's put them inside the normal examples directory and name them something parallel to what's in there, like cat.ipynb

So then the cat example directory will have files

cat_reference.csv cat.py cat.ipynb ...

…sort

pep8speaks · 2018-05-02T18:49:02Z

Hello @empet, Thank you for updating !

In the file kmapper/kmapper.py, following are the PEP8 issues :

Line 116:17: E722 do not use bare except'
Line 120:9: E722 do not use bare except'
Line 447:31: E241 multiple spaces after ','
Line 541:13: E265 block comment should start with '# '

In the file kmapper/visuals.py, following are the PEP8 issues :

Line 44:42: E272 multiple spaces before keyword
Line 74:13: E741 ambiguous variable name 'l'
Line 80:1: E302 expected 2 blank lines, found 1
Line 143:49: E712 comparison to True should be 'if cond is True:' or 'if cond:'
Line 144:49: E712 comparison to False should be 'if cond is False:' or 'if not cond:'
Line 204:11: E271 multiple spaces after keyword
Line 214:47: E261 at least two spaces before inline comment
Line 214:48: E262 inline comment should start with '# '

Comment last updated on May 02, 2018 at 20:22 Hours UTC

sauln · 2018-05-10T08:35:10Z

Thank you @empet for this work. I really like what's going on here and think having easier plotting for Jupyter would be great.

I've been overhauling some of the D3 code and I think we should put some more thought into how we integrate these features. Ideally, we could trim the interface down considerably and make it more natural to develop code for both the D3 output and the plotly output.

Maybe it would be better if the user doesn't call visualize when using plotly. Instead, they would only call pl.plotly_graph and pl.plot_layout. We could also move visualize out of KeplerMapper() all together and into it's own class. This could make it a bit more natural to switch which visualization method we use.

For plotly, is it normal for users to always run the init_notebook_mode(connected=True) boiler plate all the time? Can we wrap all of these plotly details and just supply an function to produce the plot?

empet · 2018-05-13T15:17:03Z

@sauln

We can define an independent option to plot the graph via Plotly, and avoid using visualize .
plotly.offline must be run in each notebook, because it injects the plotly.js source files into that notebook. Another option is to send the plot to Plotly cloud in the user's online account, via:

import plotly.plotly as py
py.sign_in('username', 'api_key')
py.iplot(fig, filename='my-kmapper-graph')

or his company's internal Plotly Enterprise.

When you decide how to split the two cases, let me know, and I'll perform the necessary changes.

sauln · 2018-06-19T10:01:27Z

Hi @empet, sorry for the long delay. I've been mulling over the best way to deal with multiple visualization interfaces and think I have something.

How about we employ the strategy pattern here and build separate interfaces for each plotting utility as a class. This would end up looking similar to how the Cover and Nerve work. I image there would be a few different strategies, plotly, html, htmlJupyter, and these could then be expanded to incorporate other methods.

What are your thoughts on this?

The API might look like

km.visualize(graph, meta_data, **options, visualizer=km.VizPlotly())

or

km.visualize(graph, meta_data, **options, visualizer=km.DefaultHTML())

empet · 2018-06-19T19:01:56Z

Hi @sauln, I'll work on it during the next week. Thanks for suggestion!

sauln · 2018-06-19T22:02:16Z

Great!

I’ll also have some time the next week, so please push to this PR as often as possible and I’ll try to contribute as much as I can. I expect this will be a bit of an overhaul, so us putting some thought into it will be important.

Thank you again for this! I’m excited for easy viz possibilities.

empet · 2018-07-01T16:41:55Z

Hi @sauln,

Recently a major Plotly release was announced. That's why I have to postpone working on Plotly visualize, until I experiment the new great features, in order to be able to use them in my work. Thanks for your patience and understanding.

empet · 2018-07-27T19:39:50Z

@sauln The code in plkmapper works only with Plotly version <3.0.0. Since 3.0.0 was a major release a few classes were removed or changed, and a lot of new features were introduced. Only after 2 weeks is recommended to use a much more stable version to rewrite the code as you requested. The author of these changes in plotly.py is still working on a few issues.

sauln · 2018-07-27T21:01:49Z

@empet, thanks for the update! I look forward to seeing what the new version of plotly can do and incorporating it into km.

sauln · 2018-09-11T02:00:23Z

Hi @empet, we're hoping to submit km to JoSS in the coming weeks and I would like to include your plotly visualization system in the submission. Would it be possible to get a PR in before then? How could I help? Please feel free to reach out via twitter dm or email (nat@saulgill.com) if you'd like to talk a bit about this.

Thank you!

empet · 2018-09-11T04:25:22Z

Hi @sauln,
I have only to insert details in Readme and I'll open the new PR in the next 20 hours. :)

empet added 7 commits March 25, 2018 13:28

modified for plotly visualization of kmapper graph

585f4cf

modified some function for plotly viz

8d706f0

a new created file that defines Plotly data

7b04de3

new folder

887bc26

Add files via upload

e985d36

Update with info related to Plotly plot

Add files via upload

106c3b2

Inserted a line

Add files via upload

1577c5f

sauln self-requested a review March 27, 2018 22:55

sauln requested changes Apr 27, 2018

View reviewed changes

replaced double return starting in line 78 with a single one; undone …

f8a3312

…sort

empet added 6 commits May 2, 2018 21:50

solved PEP8issues

8da5350

added .ipynb_checkpoints to .gitignore

bb07977

added ipynb files illustrating plotly viz

5725f0a

deleted folder plotly-examples

324849e

deleted .ipynb_checkpoints

6803755

removed 4 more PEP8 issues unidentified by flake8

2d725eb

empet mentioned this pull request May 2, 2018

An alternative to the actual graph plot in Jupyter Notebook #73

Closed

sauln mentioned this pull request Jun 2, 2018

How to download the plot with white background? #96

Closed

sauln approved these changes Jul 27, 2018

View reviewed changes

empet closed this Sep 12, 2018

empet deleted the plkmapper branch September 12, 2018 13:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Plkmapper #74

Plkmapper #74

empet commented Mar 25, 2018

sauln left a comment

sauln Mar 27, 2018

empet May 2, 2018

sauln Mar 27, 2018

sauln Mar 27, 2018

sauln Mar 27, 2018

sauln Mar 27, 2018

sauln Mar 27, 2018

pep8speaks commented May 2, 2018 •

edited

sauln commented May 10, 2018

empet commented May 13, 2018 •

edited

sauln commented Jun 19, 2018 •

edited

empet commented Jun 19, 2018

sauln commented Jun 19, 2018

empet commented Jul 1, 2018

empet commented Jul 27, 2018 •

edited

sauln commented Jul 27, 2018

sauln commented Sep 11, 2018

empet commented Sep 11, 2018

Plkmapper #74

Plkmapper #74

Conversation

empet commented Mar 25, 2018

sauln left a comment

Choose a reason for hiding this comment

sauln Mar 27, 2018

Choose a reason for hiding this comment

empet May 2, 2018

Choose a reason for hiding this comment

sauln Mar 27, 2018

Choose a reason for hiding this comment

sauln Mar 27, 2018

Choose a reason for hiding this comment

sauln Mar 27, 2018

Choose a reason for hiding this comment

sauln Mar 27, 2018

Choose a reason for hiding this comment

sauln Mar 27, 2018

Choose a reason for hiding this comment

pep8speaks commented May 2, 2018 • edited

Comment last updated on May 02, 2018 at 20:22 Hours UTC

sauln commented May 10, 2018

empet commented May 13, 2018 • edited

sauln commented Jun 19, 2018 • edited

empet commented Jun 19, 2018

sauln commented Jun 19, 2018

empet commented Jul 1, 2018

empet commented Jul 27, 2018 • edited

sauln commented Jul 27, 2018

sauln commented Sep 11, 2018

empet commented Sep 11, 2018

pep8speaks commented May 2, 2018 •

edited

empet commented May 13, 2018 •

edited

sauln commented Jun 19, 2018 •

edited

empet commented Jul 27, 2018 •

edited