Add force layout visualization in paragraphs for SQL queries

[davidtzoor]

What is this PR for?

Add force layout (graph) visualization to the existing set of visualizations in Zeppelin Web notp.

What type of PR is it?

[Feature]

PR description

data model used by the graph:

{
    nodes: [
        { name, group },
        ...
    ],
    links: [
        { source, target }
    ]
}

node.name is the unique identifier of a node, the group determines the color of the node.
link.source/target are the start and end points of the edges of the graph, based on the nodes names.

What is the Jira issue?

[ZEPPELIN-1011]

How should this be tested?

Run the following 2 code samples in a paragraph to generate the data set in the context:

data = """7901850786671431223   A   -9201925366945671176        B
-8214570892815727335    A   -9142458413787123325        B
9209298202739151797 A   -9142458413787123325        B
-235707035423510460 A   -9142458413787123325        B
-4432871508352121742    A   -8985197548318453090        B
-5282254351261019420    A   -8951838399863319184        B
-4766152790540222112    A   -8933665362858120936        B
-1111863962775664382    A   -8867555589789477095        B
5476637451143911605 A   -8864983663710882628        B
4361582717736404885 A   -8825261852220531319        B
-7605907640540369084    A   -8825261852220531319        B
-5837573774451340812    A   -8825261852220531319        B
7163053636126215249 A   -8825261852220531319        B
861139451162471159  A   -8825261852220531319        B
4710296859646728237 A   -8825261852220531319        B
-599615685466611445 A   -8799537192434911718        B
-8744540049061656557    A   -8780425837578325961        B
-7395255090693731764    A   -8780425837578325961        B
-8450625258438666747    A   -8776915776245574573        B
-5109485004342136017    A   -8768104402834035581        B
-6412466919921704273    A   -8719754505429102068        B
-5110717923604683987    A   -8719754505429102068        B
-3658773984531314435    A   -8719754505429102068        B
-8450625258438666747    A   -8719501119501924815        B
-6088842233080689224    A   -8719501119501924815        B
5080785082159503629 A   -8719501119501924815        B
-235707035423510460 A   -8719501119501924815        B
4710296859646728237 A   -8719501119501924815        B
574465287884870837  A   -8618660447454744071        B
2627058191174051503 A   -8618660447454744071        B
3361213877988256304 A   -8556158745209629329        B
1753480874442477887 A   -8542160690522263351        B
4710082952441870044 A   -8523724932066627876        B
3593037200969895583 A   -8523724932066627876        B
6549790973119727915 A   -8523724932066627876        B
-6391647869209764907    A   -8512098745672685655        B
6628570063743632012 A   -8511148708034410263        B
5476637451143911605 A   -8346285257963178932        B
4817374285510464702 A   -8301495021220733733        B
-443915960696260770 A   -8301495021220733733        B
8256503083397256159 A   -8297987713158283102        B
-6523542708797056082    A   -8294378768128938939        B
-8450625258438666747    A   -8221781283317332638        B
-5763079946737833038    A   -8221781283317332638        B
8437764668109048693 A   -8217668597515102322        B
-7080088993011396672    A   -8217668597515102322        B
5332146435231274697 A   -8217668597515102322        B
8424739580045126352 A   -8217668597515102322        B
-7605907640540369084    A   -8217668597515102322        B
-4004532099129726555    A   -8217668597515102322        B
4586059026815355905 A   -8217668597515102322        B
-3549137688772712918    A   -8217668597515102322        B
3775141695186132785 A   -8217668597515102322        B
1081087240374379258 A   -8217668597515102322        B
-2484166165546302047    A   -8204410554003120406        B
7196795530660323225 A   -8204410554003120406        B
8945507541332375803 A   -8161835148739852333        B
4113673676745842975 A   -8160815874720256810        B
809496795310563074  A   -8160815874720256810        B
7821859219458408878 A   -8160815874720256810        B
-8333264540320731861    A   -8160815874720256810        B
7088437698409708484 A   -8160815874720256810        B
-235707035423510460 A   -8160815874720256810        B
-4773953531103614805    A   -8160815874720256810        B
9192952707487082353 A   -8160815874720256810        B
207587472337949580  A   -8081710953330287587        B
207587472337949580  A   -8048937451483613113        B
5332146435231274697 A   -8035077502230232796        B
6218300533454972222 A   -8023106060481013958        B
-7607387850802540117    A   -8006424366939731598        B
6628570063743632012 A   -7981526580741804911        B
5549018648153862860 A   -7937163015311011155        B
3537420035516183306 A   -7928986520757371761        B
5171836565124133221 A   -7928986520757371761        B
6193634154424484163 A   -7925817622173741244        B
3361213877988256304 A   -7925817622173741244        B
-235707035423510460 A   -7898905965985334148        B
1901177400995208605 A   -7883519319216828057        B
-8563242677372011407    A   -7841902181988625360        B
1785460983681264737 A   -7839581486700963709        B
-2932784130250822001    A   -7825120470248334761        B
7735790126417584160 A   -7814285974898155285        B
-199144186590703694 A   -7814285974898155285        B
5968949691686201056 A   -7783329245620004076        B
6125353611611344475 A   -7783329245620004076        B
5518918926728439070 A   -7783329245620004076        B
-6391647869209764907    A   -7783329245620004076        B
5080785082159503629 A   -7758062101959114640        B
-2798991516883042219    A   -7758062101959114640        B
809496795310563074  A   -7758062101959114640        B
534292520350604233  A   -7758062101959114640        B
6910051452477481802 A   -7688774361473897237        B
5295618851878447136 A   -7663697366780658174        B
2660413613931574480 A   -7663697366780658174        B
-8450625258438666747    A   -7649960976820763242        B
4817374285510464702 A   -7649960976820763242        B
6621281190285958998 A   -7605823123068693377        B
-7358461975222165537    A   -7503340627119474561        B
7735790126417584160 A   -7503340627119474561        B
4738737749005310255 A   -7503340627119474561        B"""

%pyspark 

data2 = [[y  for y in x.split()] for x in data.split("\n")]
distData = sc.parallelize(data2)
#print data2

df = distData.toDF(["src","type1","dst","type2"])

df.registerTempTable("example")

now run the following to get the data set:
%sql select * from example

and finally choose the source and destination columns to generate the graph:

Screenshot

Questions:

• Does the licenses files need update?
  No.
• Is there breaking changes for older versions?
  No.
• Does this needs documentation?
  We can add some documentation explaining about the different fields that need to be selected in order to generate the force layout component.

[davidtzoor]

@AhyoungRyu I did exactly as you told me, and all of the other commits were added to my PR. I undid the push and went back to the original commit of the pull request. Can you please advise? thanks

[AhyoungRyu]

@davidtzoor Oh sorry for my late response. Did you fetch the latest version of upstream master?

git checkout master
git fetch upstream
git pull upstream master

Then check git log in your local and compare with this page.
If those two have same history, you need to rebase with this status

git checkout forcelayout
git rebase master

then try to fix the conflict.

[corneadoug]

You might also need git push -f after resolving conflicts

[davidtzoor]

@corneadoug are the build errors due to my code?

[corneadoug]

@davidtzoor No the CI is fine

[davidtzoor]

@corneadoug so is my PR ok?

[davidtzoor]

hi @corneadoug any update on this PR? I am sorry to poke you it. Just wanted to know if there something else I need to do. Thanks 👍

[corneadoug]

@davidtzoor sorry for the long wait, I was trying to finish a few things before looking closer into this PR.

There was another visualization that got merged, so you will have to rebase.

Can you also provide a few things to make it easier for me to review:

• Some code (paragraph) that create or import a data set necessary to use that graph, you can update the How should this be tested part of the PR's description with it.
• The data structure used by the graph (in PR description)

Then I should be able to test it out, and give some feedback to have it merged

[davidtzoor]

@corneadoug great! I will get it done by end of day and update you.

[corneadoug]

@davidtzoor Take your time, just ping me when its done

[davidtzoor]

@corneadoug I have rebased, added the test data, instructions and data model above. Please let me know what you think

[davidtzoor]

@corneadoug any update on my PR? looking forward to your feedback