Agent Environment Hybrid Execution Model First Pass

[joshmeek]

Thanks for contributing to Prefect!

Please describe your work and make sure your PR:

• adds new tests (if appropriate)
• updates CHANGELOG.md (if appropriate)
• updates docstrings for any new functions or function arguments, including docs/outline.toml for API reference docs (if appropriate)

What does this PR change?

This PR introduces a whole overhaul of how environments are handled in Prefect. Mainly following the outline in PIN-3.

Introduced:

• A Prefect docker image (python:3.6 base)
• DockerOnKubernetes environment

Does not introduce the DaskOnKubernetes environment discussed in PIN-3. Will be implemented in a later PR.

Why is this PR important?

Environments now have a core Environment class which two base environments inherit from—LocalEnvironment and DockerEnvironment. This now allows us to make platform specific environments which abstract out infrastructure requirements to the environments themselves. For example, the Kubernetes environments subclass the DockerEnvironment because they use Docker in their deployments. Their setup functions may be used to create any other infrastructure requirements for the flow run. (e.g. creating a Dask scheduler and workers)

Closes #599, closes #600, closes #601
Starts working towards #602

Note: base environments do not implement setup and execute because they are not platform dependent.

Note: I welcome input on naming conventions for these environments

The Kubernetes environments are meant to be used in tandem with the Prefect k8s agent (https://github.com/PrefectHQ/k8s-agent). This means that all Kubernetes environments' execute function should send out the running of the flow in a Kubernetes job. This is important because the resource collection aspect of the agent looks for completed/failed jobs and then cleans up any resources tied to that job that were created during the setup phase.

[cicdw]

Could you also update the tutorial on Local Flow debugging? There's a section at the end that discusses building a container locally and using from_file to detect any serialization errors. Maybe just say "Let's use the example of a DockerEnvironment because it's easiest and representative of the types of errors you may encounter in other environments" or something.

[cicdw]

Request for MORE DOCS but otherwise 👌