Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1.0 release candidate #5267

Merged
merged 31 commits into from
Jan 25, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
8abd447
Remove support for `Environment`s (#5072)
zanieb Oct 28, 2021
898975e
Remove general deprecated items for 1.0.0rc (#5088)
zanieb Nov 1, 2021
211b088
Add breaking changes list
zanieb Nov 4, 2021
fc7cb49
Remove experimental warnings (#5115)
zanieb Nov 8, 2021
7bfc702
[1.0.0] Drop support for Python 3.6 (#5136)
zanieb Nov 11, 2021
2c2293c
[1.0.0] Drop client handling of authentication tokens (#5140)
zanieb Nov 29, 2021
ea93e68
Move changes into the changelog
zanieb Nov 30, 2021
673961c
Merge branch 'master' into 1.0.0-rc1
zanieb Nov 30, 2021
948a674
Fix black formatting
zanieb Dec 17, 2021
edffa27
Docs rc1 tutorial cleanup (#5241)
tpdorsey Dec 17, 2021
c520117
Docs changes to address 1.0.0rc deprecations (revised) (#5238)
tpdorsey Dec 20, 2021
15cf321
Merge branch 'master' into 1.0.0-rc1
zanieb Dec 20, 2021
ec05415
Fix duplicate key from merge conflict
zanieb Dec 20, 2021
16c6373
Fixup test fixture from merge
zanieb Dec 20, 2021
c705027
Remove token from new k8s image secret tests
zanieb Dec 20, 2021
b2f6247
Add build of release candidate docker image (#5253)
zanieb Dec 21, 2021
f82c945
Update branch and tag
zanieb Dec 22, 2021
9674044
Merge pull request #5113 from PrefectHQ/1.0.0-rc1
zanieb Dec 22, 2021
3d9c517
Fix tags in CI
zanieb Dec 22, 2021
442d65c
Fix tag again
zanieb Dec 22, 2021
300b466
Merge branch 'master' into 1.0rc
zanieb Dec 22, 2021
9b935d0
Empty commit to trigger build
zanieb Dec 22, 2021
e442c35
Docs: Fix broken links in 1.0rc (#5272)
tpdorsey Dec 22, 2021
2fe8867
fix minor doc typos
mashun4ek Dec 27, 2021
d31622c
Add release candidate support to `Docker` storage (#5271)
zanieb Dec 29, 2021
f01de94
fix: declare prefect's public api (#5293)
tekumara Jan 4, 2022
1fc9346
Issue 5046 futures implementation (#5279)
jacques- Jan 11, 2022
8f5a74c
Merge branch 'master' into 1.0rc
zanieb Jan 25, 2022
bbb6f00
Drop rc image builds since they will be built from `master`
zanieb Jan 25, 2022
de37477
Include rc image builds without `latest` tag
zanieb Jan 25, 2022
377a401
Merge branch 'master' into 1.0rc
zanieb Jan 25, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
134 changes: 43 additions & 91 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ jobs:
# to ensure our requirements.txt file is accurate
test_lower_prefect:
docker:
- image: python:3.6
- image: python:3.7
auth:
username: $DOCKER_HUB_USER
password: $DOCKER_HUB_PW
Expand Down Expand Up @@ -195,39 +195,9 @@ jobs:
command: pytest tests/tasks -vvrfEsx --numprocesses 4 --dist=loadfile

# ----------------------------------
# Run unit tests in Python 3.6-3.9
# Run unit tests in Python 3.7-3.9
# ----------------------------------

test_36:
docker:
- image: python:3.6
auth:
username: $DOCKER_HUB_USER
password: $DOCKER_HUB_PW
steps:
- *attach_workspace
- checkout
- setup_remote_docker
- run:
name: Install zsh for tests
command: apt-get update && apt-get install -y zsh

- run:
name: Install graphviz
command: apt-get update && apt-get install -y graphviz

- run:
name: Upgrade pip
command: pip install "pip==20.2.4"

- run:
name: Install Prefect
command: pip install ".[base_library_ci]"

- run:
name: Run tests
command: pytest tests --ignore=tests/tasks -vvrfEsx --numprocesses 4 --dist=loadfile

test_37:
docker:
- image: python:3.7
Expand Down Expand Up @@ -437,6 +407,7 @@ jobs:
command: |
docker login --username $DOCKER_HUB_USER --password $DOCKER_HUB_PW
docker push prefecthq/prefect:master

build_core_docker_image:
docker:
- image: docker
Expand Down Expand Up @@ -493,49 +464,6 @@ jobs:
docker login --username $DOCKER_HUB_USER --password $DOCKER_HUB_PW
docker push prefecthq/prefect:core

build_release_candidate_docker_image:
docker:
- image: docker
auth:
username: $DOCKER_HUB_USER
password: $DOCKER_HUB_PW
parameters:
python_version:
type: string
extras:
type: string
environment:
PYTHON_VERSION: << parameters.python_version >>
EXTRAS: << parameters.extras >>
steps:
- checkout
- run:
name: 1.0rc branch check
command: |
apk add git
if [[ $(git branch --contains $CIRCLE_SHA1 --points-at 1.0rc | grep 1.0rc | wc -l) -ne 1 ]]; then
echo "commit $CIRCLE_SHA1 is not a member of the 1.0rc branch"
exit 1
fi
- setup_remote_docker:
docker_layer_caching: true
- run:
name: Build image
command: |
set -u
docker build \
--build-arg GIT_SHA=$CIRCLE_SHA1 \
--build-arg BUILD_DATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ') \
--build-arg PREFECT_VERSION=$CIRCLE_SHA1 \
--build-arg PYTHON_VERSION=$PYTHON_VERSION \
--build-arg EXTRAS=$EXTRAS \
-t prefecthq/prefect:1.0rc0 \
.
- run:
name: Push 1.0rc0 tag
command: |
docker login --username $DOCKER_HUB_USER --password $DOCKER_HUB_PW
docker push prefecthq/prefect:1.0rc0
promote_server_artifacts:
docker:
- image: docker
Expand Down Expand Up @@ -600,7 +528,6 @@ workflows:

'Run tests':
jobs:
- test_36
- test_37
- test_38
- test_39
Expand All @@ -622,23 +549,8 @@ workflows:
branches:
only: master

- build_release_candidate_docker_image:
python_version: '3.7'
extras: 'all_orchestration_extras'
filters:
branches:
only: 1.0rc

'Build and publish release artifacts':
jobs:
- build_docker_image:
python_version: '3.6'
extras: 'all_orchestration_extras'
filters:
branches:
ignore: /.*/
tags:
only: /^[0-9]+\.[0-9]+\.[0-9]+$/
- build_docker_image:
python_version: '3.7'
extras: 'all_orchestration_extras'
Expand Down Expand Up @@ -684,3 +596,43 @@ workflows:
ignore: /.*/
tags:
only: /^[0-9]+\.[0-9]+\.[0-9]+$/

# RC -----

- build_docker_image:
python_version: '3.7'
extras: 'all_orchestration_extras'
filters:
branches:
ignore: /.*/
tags:
only: /^1.0rc[0-9]$/
- build_docker_image:
python_version: '3.8'
extras: 'all_orchestration_extras'
filters:
branches:
ignore: /.*/
tags:
only: /^1.0rc[0-9]$/
- build_docker_image:
python_version: '3.9'
extras: 'all_orchestration_extras'
filters:
branches:
ignore: /.*/
tags:
only: /^1.0rc[0-9]$/
- release_to_pypi:
filters:
branches:
ignore: /.*/
tags:
only: /^1.0rc[0-9]$/
- build_core_docker_image:
python_version: '3.9'
filters:
branches:
ignore: /.*/
tags:
only: /^1.0rc[0-9]$/
28 changes: 28 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,33 @@
# Changelog

## 1.0.0

### Highlights

- Authentication with tokens has been removed; use API keys instead. - [#4643](https://github.com/PrefectHQ/prefect/pull/4643)
- Python 3.6 is no longer supported. Use Python 3.7+ instead. - [#5136](https://github.com/PrefectHQ/prefect/pull/5136)
- Flow `Environment`s have been removed; use `RunConfig`s instead. - [#5072](https://github.com/PrefectHQ/prefect/pull/5072), [docs](https://docs.prefect.io/orchestration/flow_config/upgrade.html)

### Breaking Changes

<!-- agent changes -->
- The AWS Fargate agent has been removed; use the ECS agent instead. - [#3812](https://github.com/PrefectHQ/prefect/pull/3812)
- `DockerAgent(docker_interface=...)` will now raise an exception if passed. - [#4446](https://github.com/PrefectHQ/prefect/pull/4446)
- Agents will no longer check for authentication at the `prefect.cloud.agent.auth_token` config key. - [#5140](https://github.com/PrefectHQ/prefect/pull/5140)
<!-- name/import changes -->
- Executors can no longer be imported from `prefect.engine.executors`; use `prefect.executors` instead. - [#3798](https://github.com/PrefectHQ/prefect/pull/3798)
- `Parameter` is not importable from `prefect.core.tasks` anymore; use `prefect.Parameter` instead.
- Exceptions are no longer importable from `prefect.utilities.exceptions`; use `prefect.exceptions` instead. - [#4664](https://github.com/PrefectHQ/prefect/pull/4664)
- `Client.login_to_tenant` has been renamed to `Client.switch_tenant`
<!-- cli changes -->
- The `prefect register flow` command has been removed; use `prefect register` instead. - [#4256](https://github.com/PrefectHQ/prefect/pull/4256)
- The `prefect run flow` command has been removed; use `prefect run` instead. - [#4463](https://github.com/PrefectHQ/prefect/pull/4463)
- Authentication token CLI commands `create-token`, `revoke-token`, `list-tokens` have been removed; use API keys instead. - [#4643](https://github.com/PrefectHQ/prefect/pull/4643)
- `prefect auth login` no longer accepts authentication tokens. - [#5140](https://github.com/PrefectHQ/prefect/pull/5140)
- `prefect auth purge-tokens` has been added to delete the Prefect-managed tokens directory. - [#5140](https://github.com/PrefectHQ/prefect/pull/5140)
<!-- config changes -->
- The `log_to_cloud` setting is now ignored; use `send_flow_run_logs` instead. - [#4487](https://github.com/PrefectHQ/prefect/pull/4487)]

## 0.15.13 <Badge text="beta" type="success" />

Released on January 25, 2022.
Expand Down
1 change: 0 additions & 1 deletion MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,5 @@ include test-requirements.txt
include src/prefect/_version.py
include src/prefect/_siginfo.py
include src/prefect/_sig29/*.txt
graft src/prefect/environments
graft src/prefect/agent
graft src/prefect/tasks/aws/waiters
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,7 @@ Read about Prefect's [community](https://docs.prefect.io/core/community.html) or

### Requirements

Prefect requires Python 3.6+. If you're new to Python, we recommend installing the [Anaconda distribution](https://www.anaconda.com/distribution/).
Prefect requires Python 3.7+. If you're new to Python, we recommend installing the [Anaconda distribution](https://www.anaconda.com/distribution/).

### Latest Release

Expand Down
2 changes: 2 additions & 0 deletions changes/pr5279.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
enhancement:
- "Switched to a futures based implementation for LocalDaskExecutor - [#5046](https://github.com/PrefectHQ/prefect/issues/5046)"
5 changes: 5 additions & 0 deletions changes/pr5293.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
fix:
- "Declare Prefect's public api - [#5293](https://github.com/PrefectHQ/prefect/pull/5293)"

contributor:
- "[Oliver Mannion](https://github.com/tekumara)"
6 changes: 0 additions & 6 deletions docs/.vuepress/config.js
Original file line number Diff line number Diff line change
Expand Up @@ -129,11 +129,6 @@ module.exports = {
collapsable: true,
children: getChildren('docs/api/latest', 'engine')
},
{
title: 'prefect.environments',
collapsable: true,
children: getChildren('docs/api/latest', 'environments')
},
{
title: 'prefect.executors',
collapsable: true,
Expand Down Expand Up @@ -238,7 +233,6 @@ module.exports = {
'agents/kubernetes',
'agents/vertex',
'agents/ecs',
'agents/fargate'
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion docs/core/advanced_tutorials/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ sidebarDepth: 0
# Contents

These tutorials are intended to help the reader get acquainted with the many features of Prefect and its vocabulary. All code examples
are locally executable in any Python version supported by Prefect (3.6+). Note that all features presented here are run without
are locally executable in any Python version supported by Prefect (3.7+). Note that all features presented here are run without
the Prefect server.

## [ETL](etl.md)
Expand Down
4 changes: 2 additions & 2 deletions docs/core/advanced_tutorials/local-debugging.md
Original file line number Diff line number Diff line change
Expand Up @@ -300,7 +300,7 @@ def whoami():
return reddit.user.me()


storage = Docker(base_image="python:3.6", registry_url="http://my.personal.registry")
storage = Docker(base_image="python:3.7", registry_url="http://my.personal.registry")
flow = Flow("reddit-flow", storage=storage, tasks=[whoami])
```

Expand Down Expand Up @@ -347,7 +347,7 @@ Which will result in a very explicit traceback!
```
Traceback (most recent call last):
flow = cloudpickle.loads(decrypted_pickle)
File "/usr/local/lib/python3.6/site-packages/cloudpickle/cloudpickle.py", line 944, in subimport
File "/usr/local/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 944, in subimport
__import__(name)
ModuleNotFoundError: No module named 'praw'
```
Expand Down
2 changes: 1 addition & 1 deletion docs/core/concepts/execution.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ from prefect import task, Flow
@task(name="Task A")
def task_a():
if random.random() > 0.5:
raise ValueError("Non-deterministic error has occured.")
raise ValueError("Non-deterministic error has occurred.")

@task(name="Task B", trigger=all_successful)
def task_b():
Expand Down
2 changes: 1 addition & 1 deletion docs/core/concepts/persistence.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
Prefect provides a few ways to work with cached data between tasks or flows. In-memory caching of task **inputs** is automatically applied by the Prefect pipeline to optimize retries or other times when Prefect can anticipate rerunning the same task in the future. Users can also configure to cache the **output** of a prior run of a task and use it as the output of a future run of that task or even as the output of a run of a different task.

Out of the box, Prefect Core does not persist cached data in a permanent fashion. All data, results, _and_ cached states are only stored in memory within the
Python process running the flow. However, Prefect Core provides all of the necessary hooks for configuring your data to be persisted and retrieved from external locations. When combined with a compatible state persistence layer, such as Prefect Core's server or [Prefect Cloud](../../orchestration/faq.html#what-is-the-difference-between-prefect-core-and-prefect-cloud), this means flows can pick up exactly where they left off if the in-memory cache is lost.
Python process running the flow. However, Prefect Core provides all of the necessary hooks for configuring your data to be persisted and retrieved from external locations. When combined with a compatible state persistence layer, such as Prefect Core's server or [Prefect Cloud](/orchestration/getting-started/set-up.html), this means flows can pick up exactly where they left off if the in-memory cache is lost.

[[toc]]

Expand Down
2 changes: 0 additions & 2 deletions docs/core/development/style.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,6 @@ cd prefect
black .
```

Please note that black requires Python 3.6+ (though Prefect does not).

Formatting can be easy to forget when developing, so you may choose to install a pre-push hook for black, as follows:

```
Expand Down
2 changes: 1 addition & 1 deletion docs/core/development/tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ The `--sw` flag will exit `pytest` the first time it encounters an error; subseq

CI will run automatically against any PR you open. Please run your tests locally first to avoid "debugging in CI", as this takes up resources that could be used by other contributors.

In CI, Prefect's unit tests are run against Python 3.6, 3.7, and 3.8. A separate "formatting" CI job is also run. Since formatting errors are common in PRs, we have found this to be a useful early-warning during development.
In CI, Prefect's unit tests are run against Python 3.7, 3.8, and 3.9. A separate "formatting" CI job is also run. Since formatting errors are common in PRs, we have found this to be a useful early-warning during development.

## Documentation

Expand Down
16 changes: 8 additions & 8 deletions docs/core/examples/overview.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
# Overview
# Prefect Tutorial Examples

Prefect includes a number of examples covering different features. These can be
viewed live in the docs, or accessed from the GitHub repo
Prefect includes a number of examples covering different features. Some are
covered in the tutorials, and all can be accessed from the GitHub repo
[here](https://github.com/PrefectHQ/prefect/tree/master/examples).

## Running with Prefect Cloud or Server
## Running examples with Prefect Cloud or Server

When running with Prefect Cloud or Prefect Server, you can register the
examples in a new project with the Prefect CLI. You can either register all the
examples at once, or select specific examples by name.

```
```bash
# Create a new project named "Prefect Examples"
$ prefect create project "Prefect Examples"

Expand All @@ -32,9 +32,9 @@ make sure the Github extra is installed so the agent can pull the flows by doing

Then to start a local agent for running the examples:

```
```bash
$ prefect agent local start -l prefect-examples
```

If you haven't already, we recommend going through the [Orchestration
Tutorial](/orchestration/tutorial/overview.md) beforehand.
If you haven't already, we recommend going through the Getting Started
[Orchestration Layer](/orchestration/getting-started/set-up.html) topics beforehand.
2 changes: 1 addition & 1 deletion docs/core/getting_started/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Basic installation

Prefect requires Python 3.6+. If you're new to Python, we recommend installing the [Anaconda distribution](https://www.anaconda.com/distribution/).
Prefect requires Python 3.7+. If you're new to Python, we recommend installing the [Anaconda distribution](https://www.anaconda.com/distribution/).

To install Prefect, run:

Expand Down
2 changes: 1 addition & 1 deletion docs/core/tutorial/07-next-steps.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,4 +83,4 @@ with Flow("Simple Pipeline") as flow:
flow.run()
```

The Task library includes integrations with Kubernetes, GitHub, Slack, Docker, AWS, GCP, [and more](/core/task_library/)!
The Task library includes integrations with Kubernetes, GitHub, Slack, Docker, AWS, GCP, [and more](/core/task_library/overview.html)!
2 changes: 1 addition & 1 deletion docs/orchestration/agents/ecs.md
Original file line number Diff line number Diff line change
Expand Up @@ -378,7 +378,7 @@ When starting an ECS agent from the command line, you can configure retry behavi
the ECS agent by setting [AWS CLI retry modes](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-retries.html).

For example, the following example specifies the AWS Adaptive retry mode and up to 10
retry attemps, then starts the ECS agent:
retry attempts, then starts the ECS agent:

For example:

Expand Down