Dash Clientside Transformations

[chriddyp]

This PR enables outputs to be updated clientside with user-defined JavaScript code. It enables users to replace certain Python callbacks with JavaScript callbacks for faster updates, lighter server load, and re-usable components+logic groups.

Unlike the Python callbacks, the JS callback signatures are embedded in the app layout. In this way, it's similar to the approach discussed in https://community.plot.ly/t/could-output-be-moved-inside-the-components-to-improve-readability/17178/2, except done clientside in JavaScript.

Embedding the function signatures in the layout has a couple of main advantages:

1. It more easily enables re-usable code logic+component blocks.
2. It enables slightly easier callbacks with dynamic components (e.g. a TODO list), as the IDs or indices of the dynamic components can be embedded directly in the static argument list.
3. It's relatively easy to read.

This PR introduces two new serializations: the function signatures and the clientside version of Input/State. These objects are serialized within the initial layout. Here's an example of what this looks like:

{
    "type": "Div",
    "namespace": "dash_html_components",
    "props": {
        "children": [

            {
                "type": "Div",
                "namespace": "dash_html_components",
                "props": {
                    "id": "my-output",
                    "children": {
                        # A simple check for "_dash-type" will indicate to
                        # dash-renderer that this is something "special"
                        # and not user defined.
                        "_dash_type": "function",
                        "function": "my_function",
                        "namespace": "my_library",
                        "positional_arguments": [
                            {
                                "_dash_type": "input",  # or state
                                "id": "my-input",
                                "property": "value"
                            },
                            3  # also allow constants
                        ]
                    }
                }
            },

            {
                "type": "Input",
                "namespace": "dash_core_components",
                "props": {
                    "id": "my-input",
                    "value": "my value"
                }
            }
        ]
    }
}

This implementation differs from previous attempts in a few ways:

1. We are no longer prescriptive about what clientside libraries or functions are available to you. You are responsible for writing the functions. Of course, you may find yourself very productive with Ramda 😉
2. We do not allow updating arbitrarily nested properties of components (e.g. figure.layout.title) - only top level properties can be updated. In this way, it mirrors the Python callbacks.
3. The Python interface is light - it just serializes a lookup to the window[namespace][function] and descriptions the arguments of that function (which can be a combination of constants & deferred-evaluated Input and State objects)

This feels to me like the appropriate level of abstraction: very little "magic", pretty similar to Python callbacks, and still quite fast and powerful. It also requires very little backend integration, so all Dash backends (Python & R) can easily have the same interface and community members will be able to re-use the same clientside snippets.

Here are four examples of apps built with clientside that demonstrate different features.

Serverside initial data + clientside filtering

import dash
from dash.dependencies import Input, Output
import dash_core_components as dcc
import dash_html_components as html

from dash_renderer.clientside import ClientsideFunction, ClientsideInput, ClientsideState

import pandas as pd

df = pd.read_csv('https://raw.githubusercontent.com/plotly/datasets/master/gapminder_unfiltered.csv')

app = dash.Dash(
    __name__,
    external_scripts=['https://cdnjs.cloudflare.com/ajax/libs/ramda/0.25.0/ramda.min.js']
)
app.css.config.serve_locally = True
app.scripts.config.serve_locally = True



app.layout = html.Div([
    dcc.Store(
        id='df',
        data=df.to_dict('records')
    ),

    dcc.Dropdown(
        id='country-search',
        options=[
            {'value': i, 'label': i}
            for i in df.country.unique()
        ],
        value='Canada'
    ),

    dcc.Dropdown(
        id='year',
        options=[
            {'value': i, 'label': i}
            for i in df.year.unique()
        ],
        multi=True,
        value=df.year.unique()
    ),

    dcc.RadioItems(
        id='mode',
        options=[
            {'label': 'Lines', 'value': 'lines'},
            {'label': 'Markers', 'value': 'markers'},
        ],
        value='lines'
    ),

    dcc.Graph(
        id='my-fig',
        figure=ClientsideFunction(
            'clientside',
            'updateFig',
            [ClientsideInput('country-search', 'value'),
             ClientsideInput('year', 'value'),
             ClientsideInput('mode', 'value'),
             ClientsideState('df', 'data')]
        )
    ),

])


if __name__ == '__main__':
    app.run_server(debug=True, dev_tools_hot_reload=False)

with the following clientside function in assets/my_functions.js:

window.clientside = {
    updateFig: function(search, years, mode, rows) {
        var filtered_rows = R.filter(
          R.allPass([
            R.compose(
              R.contains(search),
              R.prop('country')
            ),
            R.compose(
              R.flip(R.contains)(years),
              R.prop('year')
          ),
        ]), rows);

        return {
            'data': [{
                'x': R.pluck('gdpPercap', filtered_rows),
                'y': R.pluck('lifeExp', filtered_rows),
                'text': R.map(
                    R.join(' - '),
                    R.zip(
                        R.pluck('year', filtered_rows),
                        R.pluck('country', filtered_rows)
                    )
                ),
                'type': 'scatter',
                'mode': mode,
                'marker': {
                    'opacity': 0.7
                }
            }],
            'layout': {
                'hovermode': 'closest',
                'xaxis': {'type': 'log'}
            }
        }
    },
}

---

Example 2 - Serverside "Refresh" button + clientside graphing & filtering

Same as above, but with dynamic data, refreshed via a serverside function

import dash
from dash.dependencies import Input, Output
import dash_core_components as dcc
import dash_html_components as html

from dash_renderer.clientside import ClientsideFunction, ClientsideInput, ClientsideState

import numpy as np
import pandas as pd

df = pd.read_csv('https://raw.githubusercontent.com/plotly/datasets/master/gapminder_unfiltered.csv')

app = dash.Dash(
    __name__,
    external_scripts=['https://cdnjs.cloudflare.com/ajax/libs/ramda/0.25.0/ramda.min.js']
)
app.css.config.serve_locally = True
app.scripts.config.serve_locally = True

df = pd.read_csv('https://raw.githubusercontent.com/plotly/datasets/master/gapminder_unfiltered.csv')

app.layout = html.Div([
    html.Button('Refresh Data', id='refresh', n_clicks=0),
    dcc.Store(
        id='df',
        # data=df.to_dict('records')
    ),
    html.Pre(id='head'),

    dcc.Dropdown(
        id='country-search',
        options=[
            {'value': i, 'label': i}
            for i in df.country.unique()
        ],
        value='Canada'
    ),

    dcc.Dropdown(
        id='year',
        options=[
            {'value': i, 'label': i}
            for i in df.year.unique()
        ],
        multi=True,
        value=df.year.unique()
    ),

    dcc.RadioItems(
        id='mode',
        options=[
            {'label': 'Lines', 'value': 'lines'},
            {'label': 'Markers', 'value': 'markers'},
        ],
        value='markers'
    ),

    dcc.Graph(
        id='my-fig',
        figure=ClientsideFunction(
            'clientside',
            'updateFig',
            [ClientsideInput('country-search', 'value'),
             ClientsideInput('year', 'value'),
             ClientsideInput('mode', 'value'),
             ClientsideInput('df', 'data')]
        )
    ),

])


@app.callback(Output('df', 'data'), [Input('refresh', 'n_clicks')])
def update_data(n_clicks):
    df = pd.read_csv('https://raw.githubusercontent.com/plotly/datasets/master/gapminder_unfiltered.csv')
    df['lifeExp'] = np.random.randn(len(df))
    df['gdpPercap'] = np.random.randn(len(df))
    return df.to_dict('records')


@app.callback(Output('head', 'children'), [Input('df', 'data')])
def display_head(data):
    return str(pd.DataFrame(data).head())



if __name__ == '__main__':
    app.run_server(debug=True, dev_tools_hot_reload=False)

Example 3 - Chaining clientside + serverside callbacks
Mix and match! The DAG remains alive and well - callbacks, no matter where they are executed, will wait their turn.

import dash
from dash.dependencies import Input, Output
import dash_core_components as dcc
import dash_html_components as html

from dash_renderer.clientside import ClientsideFunction, ClientsideInput, ClientsideState

import pandas as pd

df = pd.read_csv('https://raw.githubusercontent.com/plotly/datasets/master/gapminder_unfiltered.csv')

app = dash.Dash(
    __name__,
    external_scripts=['https://cdnjs.cloudflare.com/ajax/libs/ramda/0.25.0/ramda.min.js']
)
app.css.config.serve_locally = True
app.scripts.config.serve_locally = True



app.layout = html.Div([

    html.Label('x'),
    dcc.Input(id='x', value=3),

    html.Label('y'),
    dcc.Input(id='y', value=6),

    # clientside
    html.Label('x + y (clientside)'),
    dcc.Input(
        id='x+y',
        value=ClientsideFunction(
            'R',
            'add',
            [ClientsideInput('x', 'value'),
             ClientsideInput('y', 'value')]
        )
    ),

    # server-side
    html.Label('x+y / 2 (serverside - takes 5 seconds)'),
    dcc.Input(id='x+y / 2'),

    # server-side
    html.Div([
        html.Label('Display x, y, x+y/2 (serverside) - takes 5 seconds'),
        html.Pre(id='display-all-of-the-values'),
    ]),

    # clientside
    html.Label('Mean(x, y, x+y, x+y/2) (clientside)'),
    html.Div(
        id='mean-of-all-values',
        children=ClientsideFunction(
            'clientside',
            'mean',
            [
                ClientsideInput('x', 'value'),
                ClientsideInput('y', 'value'),
                ClientsideInput('x+y', 'value'),
                ClientsideInput('x+y / 2', 'value'),
            ]
        )
    ),


])


@app.callback(Output('x+y / 2', 'value'),
              [Input('x+y', 'value')])
def divide_by_two(value):
    import time; time.sleep(4)
    return float(value) / 2.0


@app.callback(Output('display-all-of-the-values', 'children'),
              [Input('x', 'value'),
               Input('y', 'value'),
               Input('x+y', 'value'),
               Input('x+y / 2', 'value')])
def display_all(*args):
    import time; time.sleep(4)
    return '\n'.join([str(a) for a in args])


if __name__ == '__main__':
    app.run_server(debug=True, dev_tools_hot_reload=False)

with the following assets/my_functions.js (my_functions.js could be named anything) file:

window.clientside = {

    mean: function(...args) {
        console.warn('mean.args: ', args);
        const meanValues = R.mean(args);
        console.warn('meanValues: ', meanValues);
        return meanValues;
    }
}

Example 4 - Adding rows & columns to a table via clientside functions

This is where clientside could shine - the simple operations between components: adding rows to tables or updating dropdowns.

import dash
from dash.dependencies import Input, Output
import dash_core_components as dcc
import dash_html_components as html
import dash_table
import json

from dash_renderer.clientside import ClientsideFunction, ClientsideInput, ClientsideState

import pandas as pd

app = dash.Dash(
    __name__,
    external_scripts=['https://cdnjs.cloudflare.com/ajax/libs/ramda/0.25.0/ramda.min.js']
)
app.css.config.serve_locally = True
app.scripts.config.serve_locally = True


app.layout = html.Div([
    html.Label('New Column'),
    dcc.Input(id='new-column-name', placeholder='name'),
    html.Button('Add Column', id='add-column', n_clicks=0),
    html.Button('Add Row', id='add-row', n_clicks=1),
    dash_table.DataTable(
        id='table',
        editable=True,
        columns=ClientsideFunction(
            'clientside',
            'tableColumns',
            [ClientsideInput('add-column', 'n_clicks'),
             ClientsideState('new-column-name', 'value'),
             ClientsideState('table', 'columns'),
             [{'id': 'column-1', 'name': 'Column 1'}]],
        ),
        data=ClientsideFunction(
            'clientside',
            'tableData',
            [ClientsideInput('table', 'columns'),
             ClientsideInput('add-row', 'n_clicks'),
             ClientsideState('table', 'data'),
             [{'column-1': 9}]]
        )
    ),

    html.Div(html.B('Clientside')),
    dcc.Graph(
        id='graph',
        figure=ClientsideFunction(
            'clientside',
            'graphTable',
            [ClientsideInput('table', 'data')]
        )
    ),

    html.B('Server Side'),
    html.Pre(id='display')
])


@app.callback(Output('display', 'children'),
              [Input('table', 'columns'),
               Input('table', 'data')])
def display_data(columns, data):
    return html.Div([
        html.Div(html.B('Columns')),
        html.Pre(json.dumps(columns, indent=2)),
        html.Div(html.B('Data')),
        html.Pre(json.dumps(data, indent=2)),
    ])


if __name__ == '__main__':
    app.run_server(debug=True, dev_tools_hot_reload=False)

with the following assets/you_name_it.js functions:

window.clientside = {

    tableColumns: function(
        addColumnNClicks, newColumnName, existingColumns, defaultColumns
    ) {
        if (addColumnNClicks === 0) {
            return defaultColumns;
        } 
            return R.concat(
                existingColumns,
                [{'name': newColumnName, 'id': newColumnName}]
            );
        
    },

    tableData: function(
        columns, n_clicks, data, initial_data
    ) {
        if (n_clicks === 0 && columns.length === 1) {
            return initial_data;
        } else if (R.isNil(data)) {
            return initial_data;
        } else if (columns.length > R.values(data[0]).length) {
            return data.map(row => {
                const newCell = {};
                newCell[columns[columns.length - 1].id] = 9;
                return R.merge(row, newCell)
            });
        } else if(n_clicks > data.length) {
            const newRow = {};
            columns.forEach(col => newRow[col.id] = 9);
            return R.concat(
                data,
                [newRow]
            );
        }
    },

    graphTable(data) {
        return {
            'data': [{
                'z': R.map(R.values, data),
                'type': 'heatmap'
            }]
        }
    }
}

[chriddyp]

This was close but not quite there. Consensus discussing this with folks today is that the "functions embedded in layout" is too far away from the way traditional serverside callbacks work, even if there are some advantages in functionality. There are a few advantages to keeping the syntax closer to @app.callback:

1. Easier story around this being a "escape hatch" - if one of your callbacks is slow, rewrite it in JS. Fundamentally, the architecture of your app won't change.
2. Similarly, it'll be easier to always start in Python and then "optimize later"
3. Easier to have parity with other features in the future like wildcards
4. This doesn't prevent us from adding embedded functions in the layout later. And when we do so, we could do it for both serverside and clientside.

This would be the new syntax:

app.client_callback(
    Output('head', 'children'),
    [Input('df', 'data')],
    ClientFunction(namespace='clientside', function_name='updateFig')
)

An update is forthcoming this weekend.

[chriddyp]

Closing in preference for #143