Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename long callbacks to background. #2116

Merged
merged 7 commits into from Jul 7, 2022
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
11 changes: 9 additions & 2 deletions CHANGELOG.md
Expand Up @@ -18,8 +18,15 @@ This project adheres to [Semantic Versioning](https://semver.org/).
### Added

- [#2039](https://github.com/plotly/dash/pull/2039) Long callback changes:
- Add `long=False` to `dash.callback` to use instead of `app.long_callback`.
- Add previous `app.long_callback` arguments to `dash.callback` prefixed with `long_` (`interval`, `running`, `cancel`, `progress`, `progress_default`, `cache_args_to_ignore`, `manager`)
- Add `background=False` to `dash.callback` to use instead of `app.long_callback`.
- Add previous `app.long_callback` arguments to `dash.callback` (`interval`, `running`, `cancel`, `progress`, `progress_default`, `cache_args_to_ignore`, `manager`)

## Changed

- [#2116](https://github.com/plotly/dash/pull/2116) Rename long callbacks to background callbacks
- Deprecated `dash.long_callback.managers.CeleryLongCallbackManager`, use `dash.CeleryManager` instead.
- Deprecated `dash.long_callback.managers.DiskcacheLongCallbackManager`, use `dash.DiskcacheManager` instead.
- Deprecated dash constructor argument `long_callback_manager` in favor of `background_callback_manager`.

## [2.5.1] - 2022-06-13

Expand Down
5 changes: 5 additions & 0 deletions dash/__init__.py
Expand Up @@ -26,6 +26,10 @@
get_relative_path,
strip_relative_path,
)
from .long_callback import ( # noqa: F401,E402
CeleryBackgroundExecutor,
DiskcacheBackgroundExecutor,
)


from ._pages import register_page, PAGE_REGISTRY as page_registry # noqa: F401,E402
Expand All @@ -35,4 +39,5 @@
page_container,
)


ctx = callback_context
76 changes: 39 additions & 37 deletions dash/_callback.py
Expand Up @@ -53,14 +53,14 @@ def is_no_update(obj):
# pylint: disable=too-many-locals
def callback(
*_args,
long=False,
long_interval=1000,
long_progress=None,
long_progress_default=None,
long_running=None,
long_cancel=None,
long_manager=None,
long_cache_args_to_ignore=None,
background=False,
interval=1000,
progress=None,
progress_default=None,
running=None,
cancel=None,
manager=None,
cache_args_to_ignore=None,
**_kwargs,
):
"""
Expand All @@ -81,35 +81,35 @@ def callback(
`False` and unlike `app.callback` is not configurable at the app level.

:Keyword Arguments:
:param long:
:param background:
Mark the callback as a long callback to execute in a manager for
callbacks that take a long time without locking up the Dash app
or timing out.
:param long_manager:
A long callback manager instance. Currently an instance of one of
`DiskcacheLongCallbackManager` or `CeleryLongCallbackManager`.
Defaults to the `long_callback_manager` instance provided to the
:param manager:
A long callback manager instance. Currently, an instance of one of
`DiskcacheManager` or `CeleryManager`.
Defaults to the `background_callback_manager` instance provided to the
`dash.Dash constructor`.
- A diskcache manager (`DiskcacheLongCallbackManager`) that runs callback
- A diskcache manager (`DiskcacheManager`) that runs callback
logic in a separate process and stores the results to disk using the
diskcache library. This is the easiest backend to use for local
development.
- A Celery manager (`CeleryLongCallbackManager`) that runs callback logic
- A Celery manager (`CeleryManager`) that runs callback logic
in a celery worker and returns results to the Dash app through a Celery
broker like RabbitMQ or Redis.
:param long_running:
:param running:
A list of 3-element tuples. The first element of each tuple should be
an `Output` dependency object referencing a property of a component in
the app layout. The second element is the value that the property
should be set to while the callback is running, and the third element
is the value the property should be set to when the callback completes.
:param long_cancel:
:param cancel:
A list of `Input` dependency objects that reference a property of a
component in the app's layout. When the value of this property changes
while a callback is running, the callback is canceled.
Note that the value of the property is not significant, any change in
value will result in the cancellation of the running job (if any).
:param long_progress:
:param progress:
An `Output` dependency grouping that references properties of
components in the app's layout. When provided, the decorated function
will be called with an extra argument as the first argument to the
Expand All @@ -118,18 +118,18 @@ def callback(
current progress. This function accepts a single argument, which
correspond to the grouping of properties specified in the provided
`Output` dependency grouping
:param long_progress_default:
:param progress_default:
A grouping of values that should be assigned to the components
specified by the `progress` argument when the callback is not in
progress. If `progress_default` is not provided, all the dependency
properties specified in `progress` will be set to `None` when the
callback is not running.
:param long_cache_args_to_ignore:
:param cache_args_to_ignore:
Arguments to ignore when caching is enabled. If callback is configured
with keyword arguments (Input/State provided in a dict),
this should be a list of argument names as strings. Otherwise,
this should be a list of argument indices as integers.
:param long_interval:
:param interval:
Time to wait between the long callback update requests.
"""

Expand All @@ -141,32 +141,32 @@ def callback(
callback_map = _kwargs.pop("callback_map", GLOBAL_CALLBACK_MAP)
callback_list = _kwargs.pop("callback_list", GLOBAL_CALLBACK_LIST)

if long:
if background:
long_spec = {
"interval": long_interval,
"interval": interval,
}

if long_manager:
long_spec["manager"] = long_manager
if manager:
long_spec["manager"] = manager

if long_progress:
long_spec["progress"] = coerce_to_list(long_progress)
if progress:
long_spec["progress"] = coerce_to_list(progress)
validate_long_inputs(long_spec["progress"])

if long_progress_default:
long_spec["progressDefault"] = coerce_to_list(long_progress_default)
if progress_default:
long_spec["progressDefault"] = coerce_to_list(progress_default)

if not len(long_spec["progress"]) == len(long_spec["progressDefault"]):
raise Exception(
"Progress and progress default needs to be of same length"
)

if long_running:
long_spec["running"] = coerce_to_list(long_running)
if running:
long_spec["running"] = coerce_to_list(running)
validate_long_inputs(x[0] for x in long_spec["running"])

if long_cancel:
cancel_inputs = coerce_to_list(long_cancel)
if cancel:
cancel_inputs = coerce_to_list(cancel)
validate_long_inputs(cancel_inputs)

cancels_output = [Output(c.component_id, "id") for c in cancel_inputs]
Expand All @@ -176,19 +176,21 @@ def callback(
@callback(cancels_output, cancel_inputs, prevent_initial_call=True)
def cancel_call(*_):
job_ids = flask.request.args.getlist("cancelJob")
manager = long_manager or context_value.get().long_callback_manager
executor = (
manager or context_value.get().background_callback_manager
)
if job_ids:
for job_id in job_ids:
manager.terminate_job(job_id)
executor.terminate_job(job_id)
return NoUpdate()

except DuplicateCallback:
pass # Already a callback to cancel, will get the proper jobs from the store.

long_spec["cancel"] = [c.to_dict() for c in cancel_inputs]

if long_cache_args_to_ignore:
long_spec["cache_args_to_ignore"] = long_cache_args_to_ignore
if cache_args_to_ignore:
long_spec["cache_args_to_ignore"] = cache_args_to_ignore

return register_callback(
callback_list,
Expand Down
34 changes: 20 additions & 14 deletions dash/dash.py
Expand Up @@ -322,9 +322,12 @@ class Dash:
want to control the document.title through a separate component or
clientside callback.

:param long_callback_manager: Long callback manager instance to support the
``@app.long_callback`` decorator. Currently an instance of one of
``DiskcacheLongCallbackManager`` or ``CeleryLongCallbackManager``
:param long_callback_manager: Deprecated, use ``background_callback_manager``
instead.

:param background_callback_manager: Background callback manager instance
to support the ``@callback(..., background=True)`` decorator.
One of ``DiskcacheManager`` or ``CeleryManager`` currently supported.
"""

def __init__( # pylint: disable=too-many-statements
Expand Down Expand Up @@ -356,6 +359,7 @@ def __init__( # pylint: disable=too-many-statements
title="Dash",
update_title="Updating...",
long_callback_manager=None,
background_callback_manager=None,
**obsolete,
):
_validate.check_obsolete(obsolete)
Expand Down Expand Up @@ -484,7 +488,7 @@ def __init__( # pylint: disable=too-many-statements

self._assets_files = []
self._long_callback_count = 0
self._long_callback_manager = long_callback_manager
self._background_manager = background_callback_manager or long_callback_manager

self.logger = logging.getLogger(name)
self.logger.addHandler(logging.StreamHandler(stream=sys.stdout))
Expand Down Expand Up @@ -1153,14 +1157,14 @@ def long_callback(
"""
return _callback.callback(
*_args,
long=True,
long_manager=manager,
long_interval=interval,
long_progress=progress,
long_progress_default=progress_default,
long_running=running,
long_cancel=cancel,
long_cache_args_to_ignore=cache_args_to_ignore,
background=True,
manager=manager,
interval=interval,
progress=progress,
progress_default=progress_default,
running=running,
cancel=cancel,
cache_args_to_ignore=cache_args_to_ignore,
callback_map=self.callback_map,
callback_list=self._callback_list,
config_prevent_initial_callbacks=self.config.prevent_initial_callbacks,
Expand All @@ -1186,7 +1190,9 @@ def dispatch(self):
input_values
) = inputs_to_dict(inputs)
g.state_values = inputs_to_dict(state) # pylint: disable=assigning-non-slot
g.long_callback_manager = self._long_callback_manager # pylint: disable=E0237
g.background_callback_manager = (
self._background_manager
) # pylint: disable=E0237
changed_props = body.get("changedPropIds", [])
g.triggered_inputs = [ # pylint: disable=assigning-non-slot
{"prop_id": x, "value": input_values.get(x)} for x in changed_props
Expand Down Expand Up @@ -1255,7 +1261,7 @@ def dispatch(self):
func,
*args,
outputs_list=outputs_list,
long_callback_manager=self._long_callback_manager,
long_callback_manager=self._background_manager,
callback_context=g,
)
)
Expand Down
11 changes: 11 additions & 0 deletions dash/long_callback/__init__.py
@@ -1,2 +1,13 @@
from .managers.celery_manager import CeleryLongCallbackManager # noqa: F401,E402
from .managers.diskcache_manager import DiskcacheLongCallbackManager # noqa: F401,E402


# Renamed without `LongCallback`


class DiskcacheBackgroundExecutor(DiskcacheLongCallbackManager):
"""Manage the background execution of callbacks with subprocesses and a diskcache result backend."""


class CeleryBackgroundExecutor(CeleryLongCallbackManager):
"""Manage background execution of callbacks with a celery queue."""
2 changes: 2 additions & 0 deletions dash/long_callback/managers/celery_manager.py
Expand Up @@ -13,6 +13,8 @@


class CeleryLongCallbackManager(BaseLongCallbackManager):
"""Deprecated: use `import CeleryManager from dash` instead."""
alexcjohnson marked this conversation as resolved.
Show resolved Hide resolved

def __init__(self, celery_app, cache_by=None, expire=None):
"""
Long callback manager that runs callback logic on a celery task queue,
Expand Down
2 changes: 2 additions & 0 deletions dash/long_callback/managers/diskcache_manager.py
Expand Up @@ -7,6 +7,8 @@


class DiskcacheLongCallbackManager(BaseLongCallbackManager):
"""Deprecated: use `import DiskcacheManager from dash` instead."""

def __init__(self, cache=None, cache_by=None, expire=None):
"""
Long callback manager that runs callback logic in a subprocess and stores
Expand Down
4 changes: 2 additions & 2 deletions tests/integration/long_callback/app_callback_ctx.py
Expand Up @@ -23,9 +23,9 @@
@callback(
Output("result", "children"),
[Input({"type": "run-button", "index": ALL}, "n_clicks")],
long=True,
background=True,
prevent_initial_call=True,
long_running=[(Output("running", "children"), "on", "off")],
running=[(Output("running", "children"), "on", "off")],
)
def update_output(n_clicks):
triggered = json.loads(ctx.triggered[0]["prop_id"].split(".")[0])
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/long_callback/app_pattern_matching.py
Expand Up @@ -20,7 +20,7 @@
@callback(
Output("result", "children"),
[Input({"type": "run-button", "index": ALL}, "n_clicks")],
long=True,
background=True,
prevent_initial_call=True,
)
def update_output(n_clicks):
Expand Down
10 changes: 5 additions & 5 deletions tests/integration/long_callback/app_short_interval.py
Expand Up @@ -21,11 +21,11 @@
@callback(
Output("result", "children"),
[Input("run-button", "n_clicks")],
long=True,
long_progress=Output("status", "children"),
long_progress_default="Finished",
long_cancel=[Input("cancel-button", "n_clicks")],
long_interval=0,
background=True,
progress=Output("status", "children"),
progress_default="Finished",
cancel=[Input("cancel-button", "n_clicks")],
interval=0,
prevent_initial_call=True,
)
def update_output(set_progress, n_clicks):
Expand Down
10 changes: 5 additions & 5 deletions tests/integration/long_callback/app_side_update.py
Expand Up @@ -22,11 +22,11 @@
@callback(
Output("result", "children"),
[Input("run-button", "n_clicks")],
long=True,
long_progress=Output("status", "children"),
long_progress_default="Finished",
long_cancel=[Input("cancel-button", "n_clicks")],
long_interval=0,
background=True,
progress=Output("status", "children"),
progress_default="Finished",
cancel=[Input("cancel-button", "n_clicks")],
interval=0,
prevent_initial_call=True,
)
def update_output(set_progress, n_clicks):
Expand Down