New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New chart type : Chord Diagrams #3013

Merged
merged 1 commit into from Jun 26, 2017

Conversation

Projects
None yet
8 participants
@mistercrunch
Contributor

mistercrunch commented Jun 21, 2017

This PR is used as an example on how to add new visualizations to Superset, find comments in the code that put things into perspective.
chord

@mistercrunch mistercrunch requested a review from graceguo-supercat Jun 21, 2017

@graceguo-supercat

LGTM!!

@@ -622,6 +624,37 @@ const visTypes = {
},
},
},
chord: {

This comment has been minimized.

@mistercrunch

mistercrunch Jun 22, 2017

Contributor

This file contains configuration of the new chord viz type. It defines the panels of controls that should render on the left panel of the explore view.

controlSetRows references controls defined in controls.jsx, and can have one or two items and that defines how many items will render on that row.

@mistercrunch

mistercrunch Jun 22, 2017

Contributor

This file contains configuration of the new chord viz type. It defines the panels of controls that should render on the left panel of the explore view.

controlSetRows references controls defined in controls.jsx, and can have one or two items and that defines how many items will render on that row.

],
},
],
controlOverrides: {

This comment has been minimized.

@mistercrunch

mistercrunch Jun 22, 2017

Contributor

controlOverrides allows you to override the control configuration coming from controls.jsx. By reusing the same control across visualizations we allow continuity in the sense that the values are carried over when going from a viz type to the next. For instance as you go from one viz type to the next, it's nice to have metrics carried over, though you might want a slightly different label or tooltip in the different contexts, this is what the overrides are for.

@mistercrunch

mistercrunch Jun 22, 2017

Contributor

controlOverrides allows you to override the control configuration coming from controls.jsx. By reusing the same control across visualizations we allow continuity in the sense that the values are carried over when going from a viz type to the next. For instance as you go from one viz type to the next, it's nice to have metrics carried over, though you might want a slightly different label or tooltip in the different contexts, this is what the overrides are for.

default: [],
description: 'One or many controls to pivot as columns',
optionRenderer: c => <ColumnOption column={c} />,

This comment has been minimized.

@mistercrunch

mistercrunch Jun 22, 2017

Contributor

This file (controls.jsx) holds configuration for a "pool" of controls that can be used across visualizations. When creating a new visualization, you'll want to reuse controls that already exist. If the exact control that you need doesn't exist already, you'll want to create it here.

Note that type references controls React components defined in javascripts/explore/component/controls/.

Also note that controls can be dynamic with mapStateToProps, allowing to map anything from the app's state into a prop for the component. This allows in this case to get configuration elements proper to the datasource in this case, but can also be used to change a control based on another control's value.

@mistercrunch

mistercrunch Jun 22, 2017

Contributor

This file (controls.jsx) holds configuration for a "pool" of controls that can be used across visualizations. When creating a new visualization, you'll want to reuse controls that already exist. If the exact control that you need doesn't exist already, you'll want to create it here.

Note that type references controls React components defined in javascripts/explore/component/controls/.

Also note that controls can be dynamic with mapStateToProps, allowing to map anything from the app's state into a prop for the component. This allows in this case to get configuration elements proper to the datasource in this case, but can also be used to change a control based on another control's value.

@@ -0,0 +1,17 @@
.chord svg #circle circle {

This comment has been minimized.

@mistercrunch

mistercrunch Jun 22, 2017

Contributor

by convention each visualization has its own css file, and the entries need to be "namespaced" properly to make sure that one viz's css doesn't spill and apply to other viz types

@mistercrunch

mistercrunch Jun 22, 2017

Contributor

by convention each visualization has its own css file, and the entries need to be "namespaced" properly to make sure that one viz's css doesn't spill and apply to other viz types

import { category21 } from '../javascripts/modules/colors';
import './chord.css';
function chordViz(slice, json) {

This comment has been minimized.

@mistercrunch

mistercrunch Jun 22, 2017

Contributor

This function is the heart of this new visualization, the framework will pass it a slice and json object, and expects you to mutate the dom element referenced in the slice object. It will be called whenever the slice needs to be rendered or re-rendered. Knowing that this function may be called many times, you can use closures here if you want to persists certain things in the module namespace.

The json objects corresponds to the data returned from running your query and related context. To see exactly what you should expect in here you can click the json button in the explore view.

The slice object contains, amongst other things, a reference to the dom element that you should mutate.

@mistercrunch

mistercrunch Jun 22, 2017

Contributor

This function is the heart of this new visualization, the framework will pass it a slice and json object, and expects you to mutate the dom element referenced in the slice object. It will be called whenever the slice needs to be rendered or re-rendered. Knowing that this function may be called many times, you can use closures here if you want to persists certain things in the module namespace.

The json objects corresponds to the data returned from running your query and related context. To see exactly what you should expect in here you can click the json button in the explore view.

The slice object contains, amongst other things, a reference to the dom element that you should mutate.

const div = d3.select(slice.selector);
const nodes = json.data.nodes;
const fd = slice.formData;

This comment has been minimized.

@mistercrunch

mistercrunch Jun 22, 2017

Contributor

On of the things that the slice object exposes isf the information provided by the user controls. The keys of this slice.formData object are the control's "id" defined in controls.jsx and their values are the ones provided by the user.

@mistercrunch

mistercrunch Jun 22, 2017

Contributor

On of the things that the slice object exposes isf the information provided by the user controls. The keys of this slice.formData object are the control's "id" defined in controls.jsx and their values are the ones provided by the user.

const fd = slice.formData;
const f = d3.format(fd.y_axis_format);
const width = slice.width();

This comment has been minimized.

@mistercrunch

mistercrunch Jun 22, 2017

Contributor

The slice object exposes width() and height() methods that can be useful while rendering your chart.

@mistercrunch

mistercrunch Jun 22, 2017

Contributor

The slice object exposes width() and height() methods that can be useful while rendering your chart.

});
}
module.exports = chordViz;

This comment has been minimized.

@mistercrunch

mistercrunch Jun 22, 2017

Contributor

that function should be exported as the default export for the visualization module

@mistercrunch

mistercrunch Jun 22, 2017

Contributor

that function should be exported as the default export for the visualization module

@@ -10,6 +10,7 @@ const vizMap = {
cal_heatmap: require('./cal_heatmap.js'),
compare: require('./nvd3_vis.js'),
directed_force: require('./directed_force.js'),
chord: require('./chord.jsx'),

This comment has been minimized.

@mistercrunch

mistercrunch Jun 22, 2017

Contributor

simply adding the mapping from the chord visualization type to the appropriate javascript modules. It's possible here to have multiple visualization types target the same module (look at nvd3.js!)

@mistercrunch

mistercrunch Jun 22, 2017

Contributor

simply adding the mapping from the chord visualization type to the appropriate javascript modules. It's possible here to have multiple visualization types target the same module (look at nvd3.js!)

@@ -1231,6 +1232,39 @@ def get_data(self, df):
return df.to_dict(orient='records')
class ChordViz(BaseViz):

This comment has been minimized.

@mistercrunch

mistercrunch Jun 22, 2017

Contributor

This is the backend part of this component, you can derive BaseViz or other BaseViz derivatives

@mistercrunch

mistercrunch Jun 22, 2017

Contributor

This is the backend part of this component, you can derive BaseViz or other BaseViz derivatives

credits = '<a href="https://github.com/d3/d3-chord">Bostock</a>'
is_timeseries = False
def query_obj(self):

This comment has been minimized.

@mistercrunch

mistercrunch Jun 22, 2017

Contributor

You need to override query_obj and it has to return a query object, which is essentially a Python dict with key/values that are expected by the query method of the connectors. Until this is better documented, you can look at the connector's Datasource query method to get a sense for it.

@mistercrunch

mistercrunch Jun 22, 2017

Contributor

You need to override query_obj and it has to return a query object, which is essentially a Python dict with key/values that are expected by the query method of the connectors. Until this is better documented, you can look at the connector's Datasource query method to get a sense for it.

qry['metrics'] = [fd.get('metric')]
return qry
def get_data(self, df):

This comment has been minimized.

@mistercrunch

mistercrunch Jun 22, 2017

Contributor

get_data receives a pandas dataframe (df) that correspond to what is returned when applying your query object defined above. The shape of that dataframe is related to what you ask for in that query_obj.

Now whatever get_data returns needs to be json-serializable, and will be made available to the javascript visualization function as payload.data.

@mistercrunch

mistercrunch Jun 22, 2017

Contributor

get_data receives a pandas dataframe (df) that correspond to what is returned when applying your query object defined above. The shape of that dataframe is related to what you ask for in that query_obj.

Now whatever get_data returns needs to be json-serializable, and will be made available to the javascript visualization function as payload.data.

slice.container.html('');
const div = d3.select(slice.selector);
const nodes = json.data.nodes;

This comment has been minimized.

@mistercrunch

mistercrunch Jun 22, 2017

Contributor

json.data corresponds exactly to what is returned in the backend ChordViz.get_data lower in this PR.

@mistercrunch

mistercrunch Jun 22, 2017

Contributor

json.data corresponds exactly to what is returned in the backend ChordViz.get_data lower in this PR.

@mistercrunch mistercrunch merged commit 7045018 into apache:master Jun 26, 2017

3 checks passed

codeclimate no new or fixed issues
Details
codeclimate/coverage 55.38%
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@mistercrunch mistercrunch deleted the mistercrunch:chords branch Jun 26, 2017

@Visitante

This comment has been minimized.

Show comment
Hide comment
@Visitante

Visitante Jul 10, 2017

Mistercrunch, I'm new to Superset. We are doing a POC here at work, and we want to know how easy it would be to add a new visualization to it. So I just updated all the files from your example to my superset local server, but the new visualization is not showing on the visualization tab.
Do I need to change anything else? I´ve restarted the server as well.
edit: I´m using version 0.18.5.

Visitante commented Jul 10, 2017

Mistercrunch, I'm new to Superset. We are doing a POC here at work, and we want to know how easy it would be to add a new visualization to it. So I just updated all the files from your example to my superset local server, but the new visualization is not showing on the visualization tab.
Do I need to change anything else? I´ve restarted the server as well.
edit: I´m using version 0.18.5.

@xrmx

This comment has been minimized.

Show comment
Hide comment
@xrmx

xrmx Jul 10, 2017

Contributor

@Visitante just use master as this PR has been merged there.

Contributor

xrmx commented Jul 10, 2017

@Visitante just use master as this PR has been merged there.

@Visitante

This comment has been minimized.

Show comment
Hide comment
@Visitante

Visitante Jul 10, 2017

@xrmx thanks for the fast reply. The point of my POC is to see how hard it would be to add a new visualization to Superset.
I´ve followed Mistercrunch example and I was not able to make the new visualization appear on dashboard.
image

Visitante commented Jul 10, 2017

@xrmx thanks for the fast reply. The point of my POC is to see how hard it would be to add a new visualization to Superset.
I´ve followed Mistercrunch example and I was not able to make the new visualization appear on dashboard.
image

graceguo-supercat added a commit to graceguo-supercat/superset that referenced this pull request Aug 7, 2017

@apoku

This comment has been minimized.

Show comment
Hide comment
@apoku

apoku Aug 17, 2017

Hi! Our team followed the FAQ on the https://superset.incubator.apache.org/faq.html page to this git thread.

Is there a mechanism for adding source for a new visualization type without manually building the Superset source? Since we are using Superset by installing via pip, is there a way to drop in new javascript files in a directory etc. that Superset knows to look for?

apoku commented Aug 17, 2017

Hi! Our team followed the FAQ on the https://superset.incubator.apache.org/faq.html page to this git thread.

Is there a mechanism for adding source for a new visualization type without manually building the Superset source? Since we are using Superset by installing via pip, is there a way to drop in new javascript files in a directory etc. that Superset knows to look for?

@xrmx

This comment has been minimized.

Show comment
Hide comment
@xrmx

xrmx Aug 17, 2017

Contributor

@apoku nope

Contributor

xrmx commented Aug 17, 2017

@apoku nope

@mistercrunch

This comment has been minimized.

Show comment
Hide comment
@mistercrunch

mistercrunch Aug 17, 2017

Contributor

@apoku thought quite a bit about that and given the modern javascript building pipeline there's no easy/clean way to do this without bundling all of Superset together. We may reorganize visualizations so that it's easier to maintain a fork. The idea would be to have all the code related to one each visualization (js & py) in a specific folder.

Contributor

mistercrunch commented Aug 17, 2017

@apoku thought quite a bit about that and given the modern javascript building pipeline there's no easy/clean way to do this without bundling all of Superset together. We may reorganize visualizations so that it's easier to maintain a fork. The idea would be to have all the code related to one each visualization (js & py) in a specific folder.

@mistercrunch

This comment has been minimized.

Show comment
Hide comment
@mistercrunch

mistercrunch Aug 17, 2017

Contributor

@Visitante you're probably missing something like an entry in stores/visTypes.jsx

Contributor

mistercrunch commented Aug 17, 2017

@Visitante you're probably missing something like an entry in stores/visTypes.jsx

@ouyangal

This comment has been minimized.

Show comment
Hide comment
@ouyangal

ouyangal Aug 18, 2017

I get an error when trying to make a custom build so I can use the chord visualization: Error Message: Failed at superset@0.18.5 build script 'NODE_ENV=production webpack --colors --progress'. Is this referring to the missing entry in stores/visTypes.jsx?
Versions of npm and node I have:
npm v3.5.2
node v4.2.6

ouyangal commented Aug 18, 2017

I get an error when trying to make a custom build so I can use the chord visualization: Error Message: Failed at superset@0.18.5 build script 'NODE_ENV=production webpack --colors --progress'. Is this referring to the missing entry in stores/visTypes.jsx?
Versions of npm and node I have:
npm v3.5.2
node v4.2.6

@xrmx

This comment has been minimized.

Show comment
Hide comment
@xrmx

xrmx Aug 18, 2017

Contributor

@ouyangal the actual error would be helpful but please try to reproduce with a modern version of node (6+) and if you cannot upgrade to a node with npm 5+ try to build with yarn instead.

Contributor

xrmx commented Aug 18, 2017

@ouyangal the actual error would be helpful but please try to reproduce with a modern version of node (6+) and if you cannot upgrade to a node with npm 5+ try to build with yarn instead.

dimaslv added a commit to dimaslv/superset that referenced this pull request Feb 2, 2018

@MTShannon

This comment has been minimized.

Show comment
Hide comment
@MTShannon

MTShannon Jun 12, 2018

Have the file locations changed? I can't find "controls.jsx" in the stores folder.

MTShannon commented Jun 12, 2018

Have the file locations changed? I can't find "controls.jsx" in the stores folder.

@HaiNguyenHuynh

This comment has been minimized.

Show comment
Hide comment
@HaiNguyenHuynh

HaiNguyenHuynh Jun 21, 2018

The new path of "controls.jsx" is superset/assets/src/explore/controls.jsx

HaiNguyenHuynh commented Jun 21, 2018

The new path of "controls.jsx" is superset/assets/src/explore/controls.jsx

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment