Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Callback Graph #1179

Merged
merged 62 commits into from
Sep 3, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
6d9cdfd
Move callback graph to cytoscape and add clientside coloring
Apr 4, 2020
18539a2
Update styling and add live component and callback introspection
Apr 4, 2020
45c3b98
Add State to the callback graph with dashed lines
Apr 4, 2020
5d99381
Remove redundant value fields in introspection data
Apr 4, 2020
42c7664
Change to BFS layout for better State support
Apr 4, 2020
d4652ca
Add store and reducers for callback profiling and change notifications
Apr 5, 2020
5a4b996
Connect CallbackGraph to profiling and change notifications
Apr 5, 2020
221d4ba
Add basic profiling and execution highlighting
Apr 5, 2020
a89994b
Cleanup effects and highlight the sub-callback graph on select.
Apr 10, 2020
1fbd48c
Add support for Server-Timing headers and resource timing API
Apr 11, 2020
147d3be
Fix error when callback selected before first run.
Apr 11, 2020
2e2215c
Switch to react-json-tree because of bugs
Apr 11, 2020
6390f01
Update JSON styling
Apr 20, 2020
8553fb6
Merge remote-tracking branch 'upstream/dev' into callback_graph. Brea…
Apr 20, 2020
0b24da2
Partial implementation of wildcards. Introspection is broken.
May 15, 2020
5fa1850
Merge branch 'dev' into callback_graph
alexcjohnson Jul 24, 2020
2ad16f3
Merge branch 'dev' into callback_graph
alexcjohnson Jul 24, 2020
06f051b
callback graph merge fixes, and restrict profiling to debug mode
alexcjohnson Jul 27, 2020
c480342
incorporate callback status in profiling
alexcjohnson Jul 28, 2020
2026f10
error boundary for callback graph
alexcjohnson Jul 28, 2020
c67bd64
basic callback graph + pattern-matching fix
alexcjohnson Jul 28, 2020
c24a8e1
callback graph compatibility with hot reloading
alexcjohnson Jul 28, 2020
c89bd92
avoid errors on callback graph info for pattern-matching callbacks
alexcjohnson Jul 28, 2020
0f4924e
Merge branch 'dev' into callback_graph
alexcjohnson Jul 28, 2020
39ee385
lint js
alexcjohnson Jul 29, 2020
19de6ec
prettier + tslint via tslint-config-prettier
alexcjohnson Jul 29, 2020
8be31bf
fix profiler with unresponsive server
alexcjohnson Jul 29, 2020
d6325a6
lint profile reducer again??
alexcjohnson Jul 29, 2020
a2c6db4
black for callback profiling
alexcjohnson Jul 29, 2020
5ba3378
callback graph - respect effect cleanup
alexcjohnson Jul 29, 2020
97f4bee
pylint callback profiling
alexcjohnson Jul 29, 2020
244c6a4
only add callback profiling in debug mode with ui
alexcjohnson Aug 3, 2020
712b8df
Merge branch 'dev' into callback_graph
alexcjohnson Aug 3, 2020
cd54525
js/py->client/server & fix displayed count/time
alexcjohnson Aug 3, 2020
fd749c8
clean up callback graph styles & profile report
alexcjohnson Aug 4, 2020
5c927f0
fix callback graph info for missing IDs
alexcjohnson Aug 4, 2020
fc967a4
use dagre, cache layout, and save it when users pan/zoom/move nodes
alexcjohnson Aug 4, 2020
21d1e70
prettier
alexcjohnson Aug 4, 2020
3cbd990
get callback inputs/outputs from request/response, not layout
alexcjohnson Aug 6, 2020
bd8a752
fix history -> undo/redo
alexcjohnson Aug 6, 2020
0cec93f
display average times and bytes, not totals, in callback profiles
alexcjohnson Aug 6, 2020
5bda9d8
Merge branch 'dev' into callback_graph
alexcjohnson Aug 6, 2020
fdde1f1
changelog for callback graph improvements
alexcjohnson Aug 6, 2020
e9977d6
lint setting tweaks - and include tsx
alexcjohnson Aug 14, 2020
da76ae0
map w/ no return -> forEach
alexcjohnson Aug 14, 2020
df041e1
don't move the graph when selecting a node
alexcjohnson Aug 14, 2020
fe759e8
Merge branch 'dev' into callback_graph
alexcjohnson Aug 14, 2020
f6ac62b
black
alexcjohnson Aug 17, 2020
ef7b60c
fiddling with cb graph layout
alexcjohnson Aug 19, 2020
75160ce
Merge branch 'dev' into callback_graph
alexcjohnson Aug 20, 2020
65e14c5
user-selectable callback graph layout algos
alexcjohnson Aug 20, 2020
6ee83d5
callback graph tests
alexcjohnson Aug 30, 2020
18035a7
Merge branch 'dev' into callback_graph
alexcjohnson Aug 30, 2020
4b10e65
py2 test fix?
alexcjohnson Aug 30, 2020
02bbe5f
Merge branch 'dev' into callback_graph
alexcjohnson Sep 3, 2020
fd28d05
fix changelog
alexcjohnson Sep 3, 2020
ec345d3
Update CHANGELOG.md
Marc-Andre-Rivet Sep 3, 2020
8af36d5
fix bad merge on requestedCallbacks.ts
Sep 3, 2020
fdc662e
trigger build
Sep 3, 2020
a710e5f
update dash-test-components lock file
Sep 3, 2020
607773d
trigger build
Sep 3, 2020
c065fd2
trigger build
Sep 3, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 44 additions & 3 deletions dash/testing/browser.py
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,7 @@ def visit_and_snapshot(
resource_path,
hook_id,
wait_for_callbacks=True,
convert_canvases=False,
assert_check=True,
stay_on_page=False,
):
Expand All @@ -114,7 +115,11 @@ def visit_and_snapshot(

# wait for the hook_id to present and all callbacks get fired
self.wait_for_element_by_id(hook_id)
self.percy_snapshot(path, wait_for_callbacks=wait_for_callbacks)
self.percy_snapshot(
path,
wait_for_callbacks=wait_for_callbacks,
convert_canvases=convert_canvases,
)
if assert_check:
assert not self.driver.find_elements_by_css_selector(
"div.dash-debug-alert"
Expand All @@ -125,7 +130,7 @@ def visit_and_snapshot(
logger.exception("snapshot at resource %s error", path)
raise e

def percy_snapshot(self, name="", wait_for_callbacks=False):
def percy_snapshot(self, name="", wait_for_callbacks=False, convert_canvases=False):
"""percy_snapshot - visual test api shortcut to `percy_runner.snapshot`.
It also combines the snapshot `name` with the Python version.
"""
Expand All @@ -148,7 +153,43 @@ def percy_snapshot(self, name="", wait_for_callbacks=False):
self.redux_state_rqs,
)

self.percy_runner.snapshot(name=snapshot_name)
if convert_canvases:
self.driver.execute_script(
"""
const stash = window._canvasStash = [];
Array.from(document.querySelectorAll('canvas')).forEach(c => {
const i = document.createElement('img');
i.src = c.toDataURL();
i.width = c.width;
i.height = c.height;
i.setAttribute('style', c.getAttribute('style'));
i.className = c.className;
i.setAttribute('data-canvasnum', stash.length);
stash.push(c);
c.parentElement.insertBefore(i, c);
c.parentElement.removeChild(c);
});
"""
)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New feature for dash.testing - percy_snapshot adds a kwarg convert_canvases that turns every <canvas> into an <img> with matching contents, then puts the canvases back afterward. This lets us capture Cytoscape, among other things (cc @xhlulu)
Screen Shot 2020-08-29 at 11 39 51 PM
Sizing isn't quite as expected, but I assume that's just Percy rendering as a different screen size from what's used in circleci.


self.percy_runner.snapshot(name=snapshot_name)

self.driver.execute_script(
"""
const stash = window._canvasStash;
Array.from(
document.querySelectorAll('img[data-canvasnum]')
).forEach(i => {
const c = stash[+i.getAttribute('data-canvasnum')];
i.parentElement.insertBefore(c, i);
i.parentElement.removeChild(i);
});
delete window._canvasStash;
"""
)

else:
self.percy_runner.snapshot(name=snapshot_name)

def take_snapshot(self, name):
"""Hook method to take snapshot when a selenium test fails. The
Expand Down
Empty file added tests/__init__.py
Empty file.
Empty file added tests/assets/__init__.py
Empty file.
123 changes: 123 additions & 0 deletions tests/assets/todo_app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
from multiprocessing import Value

import dash
from dash.dependencies import Input, Output, State, MATCH, ALL, ALLSMALLER
import dash_html_components as html
import dash_core_components as dcc


def todo_app(content_callback=False):
app = dash.Dash(__name__)

content = html.Div(
[
html.Div("Dash To-Do list"),
dcc.Input(id="new-item"),
html.Button("Add", id="add"),
html.Button("Clear Done", id="clear-done"),
html.Div(id="list-container"),
html.Hr(),
html.Div(id="totals"),
]
)

if content_callback:
app.layout = html.Div([html.Div(id="content"), dcc.Location(id="url")])

@app.callback(Output("content", "children"), [Input("url", "pathname")])
def display_content(_):
return content

else:
app.layout = content

style_todo = {"display": "inline", "margin": "10px"}
style_done = {"textDecoration": "line-through", "color": "#888"}
style_done.update(style_todo)

app.list_calls = Value("i", 0)
app.style_calls = Value("i", 0)
app.preceding_calls = Value("i", 0)
app.total_calls = Value("i", 0)

@app.callback(
Output("list-container", "children"),
Output("new-item", "value"),
Input("add", "n_clicks"),
Input("new-item", "n_submit"),
Input("clear-done", "n_clicks"),
State("new-item", "value"),
State({"item": ALL}, "children"),
State({"item": ALL, "action": "done"}, "value"),
)
def edit_list(add, add2, clear, new_item, items, items_done):
app.list_calls.value += 1
triggered = [t["prop_id"] for t in dash.callback_context.triggered]
adding = len(
[1 for i in triggered if i in ("add.n_clicks", "new-item.n_submit")]
)
clearing = len([1 for i in triggered if i == "clear-done.n_clicks"])
new_spec = [
(text, done)
for text, done in zip(items, items_done)
if not (clearing and done)
]
if adding:
new_spec.append((new_item, []))
new_list = [
html.Div(
[
dcc.Checklist(
id={"item": i, "action": "done"},
options=[{"label": "", "value": "done"}],
value=done,
style={"display": "inline"},
),
html.Div(
text, id={"item": i}, style=style_done if done else style_todo
),
html.Div(id={"item": i, "preceding": True}, style=style_todo),
],
style={"clear": "both"},
)
for i, (text, done) in enumerate(new_spec)
]
return [new_list, "" if adding else new_item]

@app.callback(
Output({"item": MATCH}, "style"),
Input({"item": MATCH, "action": "done"}, "value"),
)
def mark_done(done):
app.style_calls.value += 1
return style_done if done else style_todo

@app.callback(
Output({"item": MATCH, "preceding": True}, "children"),
Input({"item": ALLSMALLER, "action": "done"}, "value"),
Input({"item": MATCH, "action": "done"}, "value"),
)
def show_preceding(done_before, this_done):
app.preceding_calls.value += 1
if this_done:
return ""
all_before = len(done_before)
done_before = len([1 for d in done_before if d])
out = "{} of {} preceding items are done".format(done_before, all_before)
if all_before == done_before:
out += " DO THIS NEXT!"
return out

@app.callback(
Output("totals", "children"), Input({"item": ALL, "action": "done"}, "value")
)
def show_totals(done):
app.total_calls.value += 1
count_all = len(done)
count_done = len([d for d in done if d])
result = "{} of {} items completed".format(count_done, count_all)
if count_all:
result += " - {}%".format(int(100 * count_done / count_all))
return result

return app
Empty file.
120 changes: 2 additions & 118 deletions tests/integration/callbacks/test_wildcards.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
from multiprocessing import Value
import pytest
import re
from selenium.webdriver.common.keys import Keys
Expand All @@ -9,130 +8,15 @@
from dash.testing import wait
from dash.dependencies import Input, Output, State, ALL, ALLSMALLER, MATCH

from ...assets.todo_app import todo_app


def css_escape(s):
sel = re.sub("[\\{\\}\\\"\\'.:,]", lambda m: "\\" + m.group(0), s)
print(sel)
return sel


def todo_app(content_callback):
app = dash.Dash(__name__)

content = html.Div(
[
html.Div("Dash To-Do list"),
dcc.Input(id="new-item"),
html.Button("Add", id="add"),
html.Button("Clear Done", id="clear-done"),
html.Div(id="list-container"),
html.Hr(),
html.Div(id="totals"),
]
)

if content_callback:
app.layout = html.Div([html.Div(id="content"), dcc.Location(id="url")])

@app.callback(Output("content", "children"), [Input("url", "pathname")])
def display_content(_):
return content

else:
app.layout = content

style_todo = {"display": "inline", "margin": "10px"}
style_done = {"textDecoration": "line-through", "color": "#888"}
style_done.update(style_todo)

app.list_calls = Value("i", 0)
app.style_calls = Value("i", 0)
app.preceding_calls = Value("i", 0)
app.total_calls = Value("i", 0)

@app.callback(
Output("list-container", "children"),
Output("new-item", "value"),
Input("add", "n_clicks"),
Input("new-item", "n_submit"),
Input("clear-done", "n_clicks"),
State("new-item", "value"),
State({"item": ALL}, "children"),
State({"item": ALL, "action": "done"}, "value"),
)
def edit_list(add, add2, clear, new_item, items, items_done):
app.list_calls.value += 1
triggered = [t["prop_id"] for t in dash.callback_context.triggered]
adding = len(
[1 for i in triggered if i in ("add.n_clicks", "new-item.n_submit")]
)
clearing = len([1 for i in triggered if i == "clear-done.n_clicks"])
new_spec = [
(text, done)
for text, done in zip(items, items_done)
if not (clearing and done)
]
if adding:
new_spec.append((new_item, []))
new_list = [
html.Div(
[
dcc.Checklist(
id={"item": i, "action": "done"},
options=[{"label": "", "value": "done"}],
value=done,
style={"display": "inline"},
),
html.Div(
text, id={"item": i}, style=style_done if done else style_todo
),
html.Div(id={"item": i, "preceding": True}, style=style_todo),
],
style={"clear": "both"},
)
for i, (text, done) in enumerate(new_spec)
]
return [new_list, "" if adding else new_item]

@app.callback(
Output({"item": MATCH}, "style"),
Input({"item": MATCH, "action": "done"}, "value"),
)
def mark_done(done):
app.style_calls.value += 1
return style_done if done else style_todo

@app.callback(
Output({"item": MATCH, "preceding": True}, "children"),
Input({"item": ALLSMALLER, "action": "done"}, "value"),
Input({"item": MATCH, "action": "done"}, "value"),
)
def show_preceding(done_before, this_done):
app.preceding_calls.value += 1
if this_done:
return ""
all_before = len(done_before)
done_before = len([1 for d in done_before if d])
out = "{} of {} preceding items are done".format(done_before, all_before)
if all_before == done_before:
out += " DO THIS NEXT!"
return out

@app.callback(
Output("totals", "children"), Input({"item": ALL, "action": "done"}, "value")
)
def show_totals(done):
app.total_calls.value += 1
count_all = len(done)
count_done = len([d for d in done if d])
result = "{} of {} items completed".format(count_done, count_all)
if count_all:
result += " - {}%".format(int(100 * count_done / count_all))
return result

return app


@pytest.mark.parametrize("content_callback", (False, True))
def test_cbwc001_todo_app(content_callback, dash_duo):
app = todo_app(content_callback)
Expand Down
Empty file.
38 changes: 38 additions & 0 deletions tests/integration/devtools/test_callback_timing.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
from time import sleep
import requests

import dash_html_components as html
import dash
from dash.dependencies import Output, Input


def test_dvct001_callback_timing(dash_thread_server):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rpkyle this test should cover parity between Python and R for the Server-Timing header, including record_timing.

app = dash.Dash(__name__)
app.layout = html.Div()

@app.callback(Output("x", "p"), Input("y", "q"))
def x(y):
dash.callback_context.record_timing("pancakes", 1.23)
sleep(0.5)
return y

dash_thread_server(app, debug=True, use_reloader=False, use_debugger=True)

response = requests.post(
dash_thread_server.url + "/_dash-update-component",
json={
"output": "x.p",
"outputs": {"id": "x", "property": "p"},
"inputs": [{"id": "y", "property": "q", "value": 9}],
"changedPropIds": ["y.q"],
},
)

# eg 'Server-Timing': '__dash_server;dur=505, pancakes;dur=1230'
assert "Server-Timing" in response.headers
st = response.headers["Server-Timing"]
times = {k: int(v) for k, v in [p.split(";dur=") for p in st.split(", ")]}
assert "__dash_server" in times
assert times["__dash_server"] >= 500 # 0.5 sec wait
assert "pancakes" in times
assert times["pancakes"] == 1230
Loading