Skip to content

Commit

Permalink
Remove references to ConcurrentTaskRunner from docs and ensure exam…
Browse files Browse the repository at this point in the history
…ples use `.wait()` (#14144)

Co-authored-by: Bill Palombi <bill@prefect.io>
  • Loading branch information
zangell44 and billpalombi committed Jun 20, 2024
1 parent a0bb447 commit 339c794
Show file tree
Hide file tree
Showing 5 changed files with 7 additions and 16 deletions.
2 changes: 1 addition & 1 deletion docs/2.19.x/concepts/task-runners.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: task Runners
title: Task runners
description: Task runners enable you to engage specific executors for Prefect tasks, such as for concurrent, parallel, or distributed execution of tasks.
---

Expand Down
9 changes: 1 addition & 8 deletions docs/3.0rc/develop/write-flows/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -409,7 +409,7 @@ Flows allow a great deal of configuration by passing arguments to the decorator.
| `retries` | An optional number of times to retry on flow run failure. |
| <span class="no-wrap">`retry_delay_seconds`</span> | An optional number of seconds to wait before retrying the flow after failure. This is only applicable if `retries` is nonzero. |
| `flow_run_name` | An optional name to distinguish runs of this flow; this name can be provided as a string template with the flow's parameters as variables; you can also provide this name as a function that returns a string. |
| `task_runner` | An optional [task runner](/3.0rc/develop/write-tasks/use-task-runners/) to use for task execution within the flow when you `.submit()` tasks. If not provided and you `.submit()` tasks, the `ConcurrentTaskRunner` is used. |
| `task_runner` | An optional [task runner](/3.0rc/develop/write-tasks/use-task-runners/) to use for task execution within the flow when you `.submit()` tasks. If not provided and you `.submit()` tasks, the `ThreadPoolTaskRunner` is used. |
| `timeout_seconds` | An optional number of seconds indicating a maximum runtime for the flow. If the flow exceeds this runtime, it is marked as failed. Flow execution may continue until the next task is called. |
| `validate_parameters` | Boolean indicating whether parameters passed to flows are validated by Pydantic. Default is `True`. |
| `version` | An optional version string for the flow. If not provided, we will attempt to create a version string as a hash of the file containing the wrapped function. If the file cannot be located, the version will be null. |
Expand Down Expand Up @@ -600,7 +600,6 @@ Running the `hello_world()` flow (in this example from the file `hello.py`) crea
```bash
$ python hello.py
15:19:21.651 | INFO | prefect.engine - Created flow run 'daft-cougar' for flow 'Hello Flow'
15:19:21.651 | INFO | Flow run 'daft-cougar' - Using task runner 'ConcurrentTaskRunner'
15:19:21.945 | INFO | Flow run 'daft-cougar' - Created task run 'Print Hello-84f0fe0e-0' for task 'Print Hello'
Hello Marvin!
15:19:22.055 | INFO | Task run 'Print Hello-84f0fe0e-0' - Finished in state Completed()
Expand Down Expand Up @@ -676,7 +675,6 @@ Running this flow produces the following result:
<div class="terminal">
```bash
22:22:36.864 | INFO | prefect.engine - Created flow run 'acrid-tuatara' for flow 'always-fails-flow'
22:22:36.864 | INFO | Flow run 'acrid-tuatara' - Starting 'ConcurrentTaskRunner'; submitted tasks will be run concurrently...
22:22:37.060 | ERROR | Flow run 'acrid-tuatara' - Encountered exception during execution:
Traceback (most recent call last):...
ValueError: This flow immediately fails
Expand Down Expand Up @@ -713,7 +711,6 @@ Running this flow produces the following result:
<div class="terminal">
```bash
18:32:05.345 | INFO | prefect.engine - Created flow run 'auburn-lionfish' for flow 'always-fails-flow'
18:32:05.346 | INFO | Flow run 'auburn-lionfish' - Starting 'ConcurrentTaskRunner'; submitted tasks will be run concurrently...
18:32:05.582 | INFO | Flow run 'auburn-lionfish' - Created task run 'always_fails_task-96e4be14-0' for task 'always_fails_task'
18:32:05.582 | INFO | Flow run 'auburn-lionfish' - Submitted task run 'always_fails_task-96e4be14-0' for execution.
18:32:05.610 | ERROR | Task run 'always_fails_task-96e4be14-0' - Encountered exception during execution:
Expand Down Expand Up @@ -763,7 +760,6 @@ Running this flow produces the following result&mdash;it succeeds because it ret
<div class="terminal">
```bash
18:35:24.965 | INFO | prefect.engine - Created flow run 'whispering-guan' for flow 'always-succeeds-flow'
18:35:24.965 | INFO | Flow run 'whispering-guan' - Starting 'ConcurrentTaskRunner'; submitted tasks will be run concurrently...
18:35:25.204 | INFO | Flow run 'whispering-guan' - Created task run 'always_fails_task-96e4be14-0' for task 'always_fails_task'
18:35:25.205 | INFO | Flow run 'whispering-guan' - Submitted task run 'always_fails_task-96e4be14-0' for execution.
18:35:25.232 | ERROR | Task run 'always_fails_task-96e4be14-0' - Encountered exception during execution:
Expand Down Expand Up @@ -814,7 +810,6 @@ Note that the final state is `Failed`, but the states of each of the returned fu
<div class="terminal">
```bash
20:57:51.547 | INFO | prefect.engine - Created flow run 'impartial-gorilla' for flow 'always-fails-flow'
20:57:51.548 | INFO | Flow run 'impartial-gorilla' - Using task runner 'ConcurrentTaskRunner'
20:57:51.645 | INFO | Flow run 'impartial-gorilla' - Created task run 'always_fails_task-58ea43a6-0' for task 'always_fails_task'
20:57:51.686 | INFO | Flow run 'impartial-gorilla' - Created task run 'always_succeeds_task-c9014725-0' for task 'always_succeeds_task'
20:57:51.727 | ERROR | Task run 'always_fails_task-58ea43a6-0' - Encountered exception during execution:
Expand Down Expand Up @@ -870,7 +865,6 @@ Running this flow produces the following result.
<div class="terminal">
```bash
18:37:42.844 | INFO | prefect.engine - Created flow run 'lavender-elk' for flow 'always-succeeds-flow'
18:37:42.845 | INFO | Flow run 'lavender-elk' - Starting 'ConcurrentTaskRunner'; submitted tasks will be run concurrently...
18:37:43.125 | INFO | Flow run 'lavender-elk' - Created task run 'always_fails_task-96e4be14-0' for task 'always_fails_task'
18:37:43.126 | INFO | Flow run 'lavender-elk' - Submitted task run 'always_fails_task-96e4be14-0' for execution.
18:37:43.162 | INFO | Flow run 'lavender-elk' - Created task run 'always_succeeds_task-9c27db32-0' for task 'always_succeeds_task'
Expand Down Expand Up @@ -911,7 +905,6 @@ Running this flow produces the following result.
<div class="terminal">
```bash
21:02:45.715 | INFO | prefect.engine - Created flow run 'sparkling-pony' for flow 'always-succeeds-flow'
21:02:45.715 | INFO | Flow run 'sparkling-pony' - Using task runner 'ConcurrentTaskRunner'
21:02:45.816 | INFO | Flow run 'sparkling-pony' - Created task run 'always_fails_task-58ea43a6-0' for task 'always_fails_task'
21:02:45.853 | ERROR | Task run 'always_fails_task-58ea43a6-0' - Encountered exception during execution:
Traceback (most recent call last):...
Expand Down
2 changes: 1 addition & 1 deletion docs/3.0rc/develop/write-tasks/dask-and-ray.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -318,7 +318,7 @@ Many workflows include a variety of tasks, and not all of them benefit from para
Because task runners are specified on flows, you can assign different task runners to tasks by using
[subflows](/3.0rc/develop/write-flows/#composing-flows) to organize those tasks.

This example uses the default `ConcurrentTaskRunner`. Then you call a `ray_greetings()` subflow that
This example uses the default `ThreadPoolTaskRunner`. Then you call a `ray_greetings()` subflow that
uses the `RayTaskRunner` to execute the same tasks in a Ray instance.

```python
Expand Down
8 changes: 3 additions & 5 deletions docs/3.0rc/develop/write-tasks/use-task-runners.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,8 @@ which runs tasks concurrently within a thread pool.
The following Prefect-developed task runners for parallel or distributed task
execution are available as [integrations](/integrations/catalog/):

- [`DaskTaskRunner`](https://prefecthq.github.io/prefect-dask/) can run tasks using
[`dask.distributed`](http://distributed.dask.org/).
- [`RayTaskRunner`](https://prefecthq.github.io/prefect-ray/) can run tasks using
[Ray](https://www.ray.io/).
- [`DaskTaskRunner`](https://github.com/PrefectHQ/prefect/tree/main/src/integrations/prefect-dask) can run tasks using [`dask.distributed`](http://distributed.dask.org/).
- [`RayTaskRunner`](https://github.com/PrefectHQ/prefect/tree/main/src/integrations/prefect-ray) can run tasks using [Ray](https://www.ray.io/).

<Note>
**Concurrency vs. parallelism**
Expand Down Expand Up @@ -208,7 +206,7 @@ def my_task():
def my_flow():
future = my_task.submit()
result = future.result(raise_on_failure=False)
if future.get_state().is_failed():
if future.state.is_failed():
# `result` is an exception! handle accordingly
...
else:
Expand Down
2 changes: 1 addition & 1 deletion src/prefect/flows.py
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,7 @@ class Flow(Generic[P, R]):
be provided as a string template with the flow's parameters as variables,
or a function that returns a string.
task_runner: An optional task runner to use for task execution within the flow;
if not provided, a `ConcurrentTaskRunner` will be used.
if not provided, a `ThreadPoolTaskRunner` will be used.
description: An optional string description for the flow; if not provided, the
description will be pulled from the docstring for the decorated function.
timeout_seconds: An optional number of seconds indicating a maximum runtime for
Expand Down

0 comments on commit 339c794

Please sign in to comment.