Skip to content

Commit

Permalink
Reorganizes the develop section to fewer, more straightforward groups (
Browse files Browse the repository at this point in the history
  • Loading branch information
billpalombi committed Jun 20, 2024
1 parent 5baa7cc commit 9e05a16
Show file tree
Hide file tree
Showing 34 changed files with 72 additions and 72 deletions.
10 changes: 5 additions & 5 deletions docs/3.0rc/deploy/work-pools/prefect-deploy.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -269,7 +269,7 @@ if __name__ == "__main__":
)
```

Note that you used a [Prefect Secret block](/3.0rc/develop/connect-third-party/) to load the URL configuration for
Note that you used a [Prefect Secret block](/3.0rc/develop/blocks/) to load the URL configuration for
the artifact registry above.

See all the optional keyword arguments for the [DockerImage class](https://docker-py.readthedocs.io/en/stable/images.html#docker.models.images.ImageCollection.build).
Expand Down Expand Up @@ -393,7 +393,7 @@ In the example above, your credentials are auto-discovered from your deployment
must be available in your runtime environment.

If you need additional configuration for your cloud-based storage (for example, with a private S3 Bucket), we recommend using a
[storage block](/3.0rc/develop/connect-third-party/).
[storage block](/3.0rc/develop/blocks/).
A storage block also ensures your credentials are available in both your deployment creation environment and your execution
environment.

Expand Down Expand Up @@ -790,7 +790,7 @@ The bucket is populated with the provided value (which also could have been prov
`folder` property of the `push` step is a template—the `pull_from_s3` step outputs both a `bucket` value as well as a `folder`
value for the template downstream steps. This helps you keep your steps consistent across edits.

As discussed above, if you use [blocks](/3.0rc/develop/connect-third-party/), you can template the credentials section with
As discussed above, if you use [blocks](/3.0rc/develop/blocks/), you can template the credentials section with
a block reference for secure and dynamic credentials access:

```yaml
Expand Down Expand Up @@ -932,10 +932,10 @@ Values that you place within your `prefect.yaml` file can reference dynamic valu
- **step outputs**: every step of both `build` and `push` produce named fields such as `image_name`; you can reference these
fields within `prefect.yaml` and `prefect deploy` will populate them with each call. References must be enclosed in double
brackets and in `"{{ field_name }}"` format
- **blocks**: you can reference [Prefect blocks](/3.0rc/develop/connect-third-party) with the
- **blocks**: you can reference [Prefect blocks](/3.0rc/develop/blocks) with the
`{{ prefect.blocks.block_type.block_slug }}` syntax. It is highly recommended that you use block references for any sensitive
information (such as a GitHub access token or any credentials) to avoid hardcoding these values in plaintext
- **variables**: you can reference [Prefect variables](/3.0rc/develop/write-flows/variables) with the
- **variables**: you can reference [Prefect variables](/3.0rc/develop/variables) with the
`{{ prefect.variables.variable_name }}` syntax. Use variables to reference non-sensitive, reusable pieces of information
such as a default image name or a default work pool name.
- **environment variables**: you can also reference environment variables with the special syntax `{{ $MY_ENV_VAR }}`.
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ You can create and use these block types through the UI and without installing a
The `S3`, `Azure`, `GCS`, and `GitHub` blocks are deprecated in favor of the the corresponding `S3Bucket`,
`AzureBlobStorageCredentials`, `GCSBucket`, and `GitHubRepository` blocks found in the
[Prefect integration libraries](/integrations/).
The JSON, DateTime, and String blocks are deprecated in favor of [Variables](/3.0rc/develop/write-flows/variables/).
The JSON, DateTime, and String blocks are deprecated in favor of [Variables](/3.0rc/develop/variables/).
</Warning>

## Blocks in Prefect integration libraries
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -484,7 +484,7 @@ async def sender():

An iterator helps keep track of the inputs your flow has already received. If you want your flow to suspend and then resume later,
save the keys of the inputs you've seen so the flow can read them back out when it resumes.
Consider using a [block](/3.0rc/develop/connect-third-party/), such as a `JSONBlock`.
Consider using a [block](/3.0rc/develop/blocks/), such as a `JSONBlock`.

The following flow receives input for 30 seconds then suspends itself, which exits the flow and tears down infrastructure:

Expand Down
2 changes: 1 addition & 1 deletion docs/3.0rc/develop/interact-with-data.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ If the bucket is private, there are several options to authenticate:

If you saved credential details in a block, you can use a credentials block specific to the cloud provider
or use a more generic secret block.
You can create [blocks](/3.0rc/develop/connect-third-party/) through the UI or Python code.
You can create [blocks](/3.0rc/develop/blocks/) through the UI or Python code.

The example below uses Python code to create a credentials block for your cloud provider.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ for flow and task runs.

## Logging configuration

Prefect provides several settings to configure [logging level and loggers](/3.0rc/develop/observe-workflows/logging/).
Prefect provides several settings to configure [logging level and loggers](/3.0rc/develop/logging/).

By default, Prefect displays `INFO`-level and above logging records. Change this level to `DEBUG` and see `DEBUG`-level
logs created by Prefect as well. You may need to change the log level used by loggers from other libraries to see their log records.
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ If the concurrency limit is set to 0 for a tag, any attempt to run a task with t

### Execution behavior

Task tag limits are checked whenever a task run attempts to enter a [`Running` state](/3.0rc/develop/observe-workflows/manage-states/).
Task tag limits are checked whenever a task run attempts to enter a [`Running` state](/3.0rc/develop/manage-states/).

If there are no concurrency slots available for any one of your task's tags, it delays the transition to a `Running` state
and instructs the client to try entering a `Running` state again in 30 seconds
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ configuration information to Prefect configuration files, such as deployment ste
**Variables are not Encrypted**

We do not recommend using variables to store sensitive information. Instead, use
[Secret blocks](https://docs.prefect.io/develop/connect-third-party/#prefect-built-in-blocks) to store and access
[Secret blocks](https://docs.prefect.io/develop/blocks/#prefect-built-in-blocks) to store and access
sensitive information.
</Warning>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ They can take inputs, perform work, and return an output.
You can turn any function into a Prefect flow by adding the `@flow` decorator.
When a function becomes a flow, its behavior changes, giving it the following advantages:

- All runs of the flow have persistent [state](/3.0rc/develop/observe-workflows/manage-states/). Transitions between states are recorded,
- All runs of the flow have persistent [state](/3.0rc/develop/manage-states/). Transitions between states are recorded,
allowing you to observe and act on flow execution.
- Input arguments can be type validated as workflow parameters.
- Retries can be performed on failure.
Expand Down Expand Up @@ -299,7 +299,7 @@ However you run the flow, the Prefect API monitors the flow run, capturing flow

Prefect enables you to log a variety of useful information about your flow and task runs.
You can capture information about your workflows for purposes such as monitoring, troubleshooting, and auditing.
Check out [Logging](/3.0rc/develop/observe-workflows/logging) for more information.
Check out [Logging](/3.0rc/develop/logging) for more information.
</Tip>

When you run a flow that contains tasks or additional flows, Prefect tracks the relationship of each child run to the parent flow run.
Expand Down Expand Up @@ -409,7 +409,7 @@ Flows allow a great deal of configuration by passing arguments to the decorator.
| `retries` | An optional number of times to retry on flow run failure. |
| <span class="no-wrap">`retry_delay_seconds`</span> | An optional number of seconds to wait before retrying the flow after failure. This is only applicable if `retries` is nonzero. |
| `flow_run_name` | An optional name to distinguish runs of this flow; this name can be provided as a string template with the flow's parameters as variables; you can also provide this name as a function that returns a string. |
| `task_runner` | An optional [task runner](/3.0rc/develop/write-tasks/task-runners/) to use for task execution within the flow when you `.submit()` tasks. If not provided and you `.submit()` tasks, the `ThreadPoolTaskRunner` is used. |
| `task_runner` | An optional [task runner](/3.0rc/develop/task-runners/) to use for task execution within the flow when you `.submit()` tasks. If not provided and you `.submit()` tasks, the `ThreadPoolTaskRunner` is used. |
| `timeout_seconds` | An optional number of seconds indicating a maximum runtime for the flow. If the flow exceeds this runtime, it is marked as failed. Flow execution may continue until the next task is called. |
| `validate_parameters` | Boolean indicating whether parameters passed to flows are validated by Pydantic. Default is `True`. |
| `version` | An optional version string for the flow. If not provided, we will attempt to create a version string as a hash of the file containing the wrapped function. If the file cannot be located, the version will be null. |
Expand Down Expand Up @@ -515,7 +515,7 @@ The primary flow is the "parent" flow. The flow created within the parent is the

Subflow runs behave like normal flow runs.
There is a full representation of the flow run in the backend as if it had been called separately.
When a subflow starts, it creates a new [task runner](/3.0rc/develop/write-tasks/task-runners/) for tasks within the subflow.
When a subflow starts, it creates a new [task runner](/3.0rc/develop/task-runners/) for tasks within the subflow.
When the subflow completes, the task runner shuts down.

Subflows block execution of the parent flow until completion.
Expand Down Expand Up @@ -641,7 +641,7 @@ task runner for each subflow.
## Final state determination

<Note>
Read the documentation about [states](/3.0rc/develop/observe-workflows/manage-states) before proceeding with this section.
Read the documentation about [states](/3.0rc/develop/manage-states) before proceeding with this section.
</Note>

The final state of the flow is determined by its return value. The following rules apply:
Expand Down Expand Up @@ -917,7 +917,7 @@ Completed(message=None, type=COMPLETED, result='foo', flow_run_id=7240e6f5-f0a8-
## See also
- Store and reuse non-sensitive bits of data, such as configuration information, by using [variables](/3.0rc/develop/write-flows/variables).
- Store and reuse non-sensitive bits of data, such as configuration information, by using [variables](/3.0rc/develop/variables).
- Supercharge this flow with [tasks](/3.0rc/develop/write-tasks/) to break down the workflow's complexity and make it more performant and observable.
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Prefect workflow. Tasks can:
- Cache their execution across invocations
- Encapsulate workflow logic into reusable units across flows and subflows
- Receive metadata about upstream task dependencies and their state before running
- Use automatic [logging](/3.0rc/develop/observe-workflows/logging/) to capture runtime details, tags,
- Use automatic [logging](/3.0rc/develop/logging/) to capture runtime details, tags,
and final state
- Execute concurrently
- Be defined in the same file as the flow or imported from modules
Expand Down Expand Up @@ -73,7 +73,7 @@ This task run is tracked in the UI as well.
Almost any standard Python function can be turned into a Prefect task by adding the `@task` decorator.

<Tip>
Tasks are always executed in the main thread by default, unless a specific [task runner](/3.0rc/develop/write-tasks/task-runners) is used to execute them on different threads, processes, or infrastructure. This facilitates native Python debugging and profiling.
Tasks are always executed in the main thread by default, unless a specific [task runner](/3.0rc/develop/task-runners) is used to execute them on different threads, processes, or infrastructure. This facilitates native Python debugging and profiling.
</Tip>

### Synchronous functions
Expand Down Expand Up @@ -496,7 +496,7 @@ By default Prefect's caching logic is based on the following attributes of a tas
All of these are hashed to compute the task's _cache key_.
This implies that, by default, calling the same task with the same inputs more than once within a flow
will result in cached behavior for all calls after the first.
This behavior can be configured - see [customizing the cache](/3.0rc/develop/write-tasks/index#customizing-the-cache) below.
This behavior can be configured - see [customizing the cache](/3.0rc/develop/write-tasks#customizing-the-cache) below.

### Cache keys

Expand Down Expand Up @@ -760,23 +760,23 @@ def show_timeouts():
## Task results

Depending on how you call tasks, they can return different types of results and optionally engage the use of
a [task runner](/3.0rc/develop/write-tasks/task-runners/).
a [task runner](/3.0rc/develop/task-runners/).

Any task can return:

- Data , such as `int`, `str`, `dict`, `list`. This is the default behavior any time you
call `your_task()`.
- [`PrefectFuture`](/3.0rc/api-ref/prefect/futures/#prefect.futures.PrefectFuture). This is achieved
by calling [`your_task.submit()`](/3.0rc/develop/write-tasks/task-runners/#using-a-task-runner).
by calling [`your_task.submit()`](/3.0rc/develop/task-runners/#using-a-task-runner).
A `PrefectFuture` contains both _data_ and _State_.
- Prefect [`State`](/3.0rc/api-ref/server/schemas/states/). Anytime you call your task or flow with
the argument `return_state=True`, it directly returns a state to build custom behavior based
on a state change you care about, such as task or flow failing or retrying.

To run your task with a [task runner](/3.0rc/develop/write-tasks/task-runners/), you must call the task
To run your task with a [task runner](/3.0rc/develop/task-runners/), you must call the task
with `.submit()`.

See [state returned values](/3.0rc/develop/write-tasks/task-runners/#using-results-from-submitted-tasks)
See [state returned values](/3.0rc/develop/task-runners/#using-results-from-submitted-tasks)
for examples.

<Tip>
Expand Down
10 changes: 5 additions & 5 deletions docs/3.0rc/get-started/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,12 @@ It's the easiest way to transform any Python function into a unit of work that c
With Prefect you gain:

* [Scheduling](/3.0rc/automate/add-schedules)
* [Retries](/3.0rc/develop/write-tasks/index#task-arguments)
* [Logging](/3.0rc/develop/observe-workflows/logging/)
* [Convenient async functionality](/3.0rc/develop/write-tasks/task-runners/#task-runners)
* [Caching](/3.0rc/develop/write-tasks/index#caching)
* [Retries](/3.0rc/develop/write-tasks#task-arguments)
* [Logging](/3.0rc/develop/logging/)
* [Convenient async functionality](/3.0rc/develop/task-runners/#task-runners)
* [Caching](/3.0rc/develop/write-tasks#caching)
* [Notifications](/3.0rc/automate/events/automations-triggers/)
* [Observability](/3.0rc/develop/observe-workflows/index/)
* [Observability](/3.0rc/develop/runtime-context/)
* [Event-based orchestration](/3.0rc/automate/events/webhook-triggers)

![screenshot of Prefect Cloud dashboard](/3.0rc/img/ui/cloud-dashboard.png)
Expand Down
4 changes: 2 additions & 2 deletions docs/3.0rc/get-started/quickstart.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -251,8 +251,8 @@ Deployments elevate flows to remotely configurable entities that have their own
You've seen how to move from a Python script to a scheduled, observable, remotely orchestrated workflow with Prefect.
Now considering reading:

* [Write flows](/3.0rc/develop/write-flows/index)
* [Write tasks](/3.0rc/develop/write-tasks/index)
* [Write flows](/3.0rc/develop/write-flows)
* [Write tasks](/3.0rc/develop/write-tasks)
* [Self-host](/3.0rc/manage/self-host) your Prefect server

<Tip>
Expand Down
2 changes: 1 addition & 1 deletion docs/3.0rc/manage/cloud/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ A workspace is an isolated environment within Prefect Cloud for all Prefect acti
- [Flows](/3.0rc/develop/write-flows/) associated with flow runs and deployments observed by the Prefect Cloud API
- [Deployments](/3.0rc/deploy/serve-flows/)
- [Work pools](/3.0rc/deploy/work-pools/control-runs/)
- [Blocks](/3.0rc/develop/connect-third-party/)
- [Blocks](/3.0rc/develop/blocks/)
- [Events](/3.0rc/automate/events/)
- [Automations](/3.0rc/automate/events/automations-triggers/)
- [Incidents](/3.0rc/automate/events/events/)
Expand Down
2 changes: 1 addition & 1 deletion docs/3.0rc/manage/cloud/rate-limits.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ The retention period applies to all workspaces that belong to the account.

The retention period is the number of days that metadata is available after it is created.
For flow and task runs, it is calculated from the time the run reaches a
[terminal state](/3.0rc/develop/observe-workflows/manage-states/#state-types).
[terminal state](/3.0rc/develop/manage-states/#state-types).
Subflow runs are retained independently from their parent flow runs. They are removed based on
the time each subflow run reaches a terminal state.

Expand Down
2 changes: 1 addition & 1 deletion docs/3.0rc/manage/cloud/troubleshoot-cloud.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ They should be upgraded simultaneously with the core library, using the same met

## Logs

In many cases, there is an informative stack trace in Prefect's [logs](/3.0rc/develop/observe-workflows/logging/).
In many cases, there is an informative stack trace in Prefect's [logs](/3.0rc/develop/logging/).
**Read it carefully**, locate the source of the error, and try to identify the cause.

There are two types of logs:
Expand Down
2 changes: 1 addition & 1 deletion docs/3.0rc/manage/configure-client.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@ If running a self-hosted Prefect server instance, there are several [database co

### Logging settings

Prefect provides several [logging configuration settings](/3.0rc/develop/observe-workflows/logging/).
Prefect provides several [logging configuration settings](/3.0rc/develop/logging/).

## Configure settings

Expand Down
2 changes: 1 addition & 1 deletion docs/3.0rc/manage/self-host.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,7 @@ Prefect Cloud has an option for automation notifications. The more limited Notif
self-hosted Prefect server.

Notifications enable you to set up alerts that are sent when a flow enters any state you specify.
When your flow and task runs changes [state](/3.0rc/develop/observe-workflows/manage-states/),
When your flow and task runs changes [state](/3.0rc/develop/manage-states/),
Prefect notes the state change and checks whether the new state matches any notification policies. If it does, a new notification is queued.

Prefect supports sending notifications through:
Expand Down

0 comments on commit 9e05a16

Please sign in to comment.