Skip to content

Commit

Permalink
Merge branch 'main' into feature/cli-wait
Browse files Browse the repository at this point in the history
  • Loading branch information
zzstoatzz committed Feb 9, 2024
2 parents e8a9d76 + 1da5190 commit baf2332
Show file tree
Hide file tree
Showing 54 changed files with 1,732 additions and 481 deletions.
343 changes: 320 additions & 23 deletions .github/workflows/python-tests.yaml

Large diffs are not rendered by default.

47 changes: 47 additions & 0 deletions RELEASE-NOTES.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,52 @@
# Prefect Release Notes

## Release 2.14.21

### Introducing work queue status

We're excited to unveil the new status indicators for work queues in Prefect's UI, enhancing your ability to oversee and control flow run execution within our hybrid work pools.

Work queues will now display one of three distinct statuses:

- `Ready` - one or more online workers are actively polling the work queue
- `Not Ready` - no online workers are polling the work queue, signaling a need for intervention
- `Paused` - the work queue is intentionally paused, preventing execution

<p align="center">
<img width="1109" alt="Prefect dashboard snapshot" src="https://github.com/PrefectHQ/prefect/assets/42048900/e5bb0a33-1ae2-44a7-a64e-ef0d308fce7a">
</p>
<img width="1109" alt="work pools page work queues table here with work queues of all statuses" src="https://github.com/PrefectHQ/prefect/assets/42048900/834f0f66-79e9-420b-9d11-d771a5b8cf02">

With the introduction of work queue status, you'll notice the absence of deprecated work queue health indicators in the UI.

See the documentation on [work queue status](https://docs.prefect.io/latest/concepts/work-pools/#work-queues) for more information.


For now, this is an experimental feature, and can be enabled by running:
```console
prefect config set PREFECT_EXPERIMENTAL_ENABLE_WORK_QUEUE_STATUS=True
```

See the following pull request for implementation details:
- https://github.com/PrefectHQ/prefect/pull/11829

### Fixes
- Remove unnecessary `WARNING` level log indicating a task run completed successfully — https://github.com/PrefectHQ/prefect/pull/11810
- Fix a bug where block placeholders declared in pull steps of the `deployments` section of a `prefect.yaml` file were not resolved correctly — https://github.com/PrefectHQ/prefect/pull/11740
- Use `pool_pre_ping` to improve stability for long-lived PostgreSQL connections — https://github.com/PrefectHQ/prefect/pull/11911

### Documentation
- Clarify Docker tutorial code snippet to ensure commands are run from the correct directory — https://github.com/PrefectHQ/prefect/pull/11833
- Remove beta tag from incident documentation and screenshots — https://github.com/PrefectHQ/prefect/pull/11921
- Update Prefect Cloud account roles docs to reflect renaming of previous "Admin" role to "Owner" and creation of new "Admin" role that cannot bypass SSO — https://github.com/PrefectHQ/prefect/pull/11925

### Experimental
- Ensure task subscribers can only pick up task runs they are able to execute — https://github.com/PrefectHQ/prefect/pull/11805
- Allow a task server to reuse the same task runner to speed up execution — https://github.com/PrefectHQ/prefect/pull/11806
- Allow configuration of maximum backlog queue size and maximum retry queue size for autonomous task runs — https://github.com/PrefectHQ/prefect/pull/11825

**All changes**: https://github.com/PrefectHQ/prefect/compare/2.14.20...2.14.21

## Release 2.14.20

### Fixes
Expand Down
6 changes: 5 additions & 1 deletion docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
version: "3.8"
services:
test-db:
image: postgres:13
image: postgres:14
ports:
- 15432:5432
environment:
Expand All @@ -14,3 +14,7 @@ services:
LC_COLLATE: 'C.UTF-8'
LC_CTYPE: 'C.UTF-8'
tmpfs: /var/lib/postgresql/data
command:
- postgres
- -c
- max_connections=250
2 changes: 1 addition & 1 deletion docs/cloud/incidents.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ search:
boost: 2
---

# Incidents <span class="badge cloud"></span> <span class="badge pro"></span> <span class="badge enterprise"></span> <span class="badge beta"/>
# Incidents <span class="badge cloud"></span> <span class="badge pro"></span> <span class="badge enterprise"></span>

## Overview

Expand Down
3 changes: 2 additions & 1 deletion docs/cloud/users/roles.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,8 @@ The following built-in roles have permissions across an account in Prefect Cloud

| Role | Abilities |
| ------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Admin | &bull; Set/change all account profile settings allowed to be set/changed by a Prefect user. <br> &bull; Add and remove account members, and their account roles. <br> &bull; Create and delete service accounts in the account. <br> &bull; Create workspaces in the account. <br> &bull; Implicit workspace owner access on all workspaces in the account. |
| Owner | &bull; Set/change all account profile settings allowed to be set/changed by a Prefect user. <br> &bull; Add and remove account members, and their account roles. <br> &bull; Create and delete service accounts in the account. <br> &bull; Create workspaces in the account. <br> &bull; Implicit workspace owner access on all workspaces in the account. <br> &bull; Bypass SSO. |
| Admin | &bull; Set/change all account profile settings allowed to be set/changed by a Prefect user. <br> &bull; Add and remove account members, and their account roles. <br> &bull; Create and delete service accounts in the account. <br> &bull; Create workspaces in the account. <br> &bull; Implicit workspace owner access on all workspaces in the account. <br> &bull; Cannot bypass SSO. |
| Member | &bull; View account profile settings. <br> &bull; View workspaces I have access to in the account. <br> &bull; View account members and their roles. <br> &bull; View service accounts in the account. |

### Workspace-level roles
Expand Down
3 changes: 3 additions & 0 deletions docs/concepts/work-pools.md
Original file line number Diff line number Diff line change
Expand Up @@ -430,6 +430,9 @@ Priority for flow run submission proceeds from the highest priority to the lowes

If new flow runs are received on the "critical" queue while flow runs are still in scheduled on the "high" and "low" queues, flow run submission goes back to ensuring all scheduled work is first satisfied from the highest priority queue, until it is empty, in waterfall fashion.

!!! tip "Work queue status"
A work queue has a `READY` status when it has been polled by a worker in the last 60 seconds. Pausing a work queue will give it a `PAUSED` status and mean that it will accept no new work until it is unpaused. A user can control the work queue's paused status in the UI. Unpausing a work queue will give the work queue a `NOT_READY` status unless a worker has polled it in the last 60 seconds.

### Local debugging

As long as your deployment's infrastructure block supports it, you can use work pools to temporarily send runs to a worker running on your local machine for debugging by running `prefect worker start -p my-local-machine` and updating the deployment's work pool to `my-local-machine`.
Expand Down
Binary file modified docs/img/ui/incidents-active.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/img/ui/incidents-dashboard.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit baf2332

Please sign in to comment.