Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove version top-level element from docker compose files #33831

Merged
merged 2 commits into from
Aug 30, 2023
Merged

Remove version top-level element from docker compose files #33831

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e30adbd
Remove `version` top-level element from docker compose files
Taragolis Aug 22, 2023
8fe08ad
Bump recommended version of docker compose and advice to upgrade to n…
Taragolis Aug 30, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 0 additions & 1 deletion docs/apache-airflow/howto/docker-compose/docker-compose.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,6 @@
#
# Feel free to modify this file to suit your needs.
---
version: '3.8'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One potential exception it is keep only for docker-compose from documentation

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From https://docs.docker.com/compose/history/

From July 2023 Compose V1 stopped receiving updates. It's also no longer available in new releases of Docker Desktop.

I think it's time we drop compose v1 compatibility also from the documentation one. I think we should just add information in https://airflow.apache.org/docs/apache-airflow/stable/howto/docker-compose/index.html that Only docker compose v2 is supported.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

People always can download old compose from previous Airflow versions.

x-airflow-common:
&airflow-common
# In order to add custom dependencies or upgrade provider packages you can use your extended image.
Expand Down
6 changes: 5 additions & 1 deletion docs/apache-airflow/howto/docker-compose/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ This procedure assumes familiarity with Docker and Docker Compose. If you haven'
Follow these steps to install the necessary tools, if you have not already done so.

1. Install `Docker Community Edition (CE) <https://docs.docker.com/engine/installation/>`__ on your workstation. Depending on your OS, you may need to configure Docker to use at least 4.00 GB of memory for the Airflow containers to run properly. Please refer to the Resources section in the `Docker for Windows <https://docs.docker.com/docker-for-windows/#resources>`__ or `Docker for Mac <https://docs.docker.com/docker-for-mac/#resources>`__ documentation for more information.
2. Install `Docker Compose <https://docs.docker.com/compose/install/>`__ v1.29.1 or newer on your workstation.
2. Install `Docker Compose <https://docs.docker.com/compose/install/>`__ v2.14.0 or newer on your workstation.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't come up with anything better here rather than pick up current MIN_DOCKER_COMPOSE_VERSION for CI/Breeze 🤣 At least we run compose tests by use 2.14

Copy link
Member

@potiuk potiuk Aug 30, 2023 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a very good one. I even explained that very minimum is coming in this PR today to a user in Slack


Older versions of ``docker-compose`` do not support all the features required by the Airflow ``docker-compose.yaml`` file, so double check that your version meets the minimum version requirements.

Expand Down Expand Up @@ -80,6 +80,10 @@ Fetching ``docker-compose.yaml``

curl -LfO '{{ doc_root_url }}docker-compose.yaml'

.. important::
From July 2023 Compose V1 stopped receiving updates.
We strongly advise upgrading to a newer version of Docker Compose, supplied ``docker-compose.yaml`` may not function accurately within Compose V1.

This file contains several service definitions:

- ``airflow-scheduler`` - The :doc:`scheduler </administration-and-deployment/scheduler>` monitors all tasks and DAGs, then triggers the
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/backend-mssql-docker-volume.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
mssql:
volumes:
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/backend-mssql-port.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
mssql:
ports:
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/backend-mssql-tmpfs-volume.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
# In case of tmpfs backend for docker, mssql fails because TMPFS does not support
# O_DIRECT parameter for direct writing to the filesystem
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/backend-mssql.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
airflow:
environment:
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/backend-mysql-port.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
mysql:
ports:
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/backend-mysql.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
airflow:
environment:
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/backend-postgres-port.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
postgres:
ports:
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/backend-postgres.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
airflow:
environment:
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/backend-sqlite.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
airflow:
environment:
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/base-ports.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
airflow:
ports:
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/base.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
airflow:
image: ${AIRFLOW_CI_IMAGE_WITH_TAG}
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/devcontainer-mysql.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
mysql:
image: mysql:8.0
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/devcontainer-postgres.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
postgres:
image: postgres:10
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/devcontainer.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
airflow:
stdin_open: true # docker run -i
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/files.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
airflow:
volumes:
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/forward-credentials.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
airflow:
# Forwards local credentials to docker image
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/ga.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
airflow:
environment:
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/integration-cassandra.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
cassandra:
image: cassandra:3.0
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/integration-celery.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
rabbitmq:
image: rabbitmq:3.7
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/integration-kafka.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
broker:
image: confluentinc/cp-kafka:7.3.0
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/integration-kerberos.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
kdc-server-example-com:
image: ghcr.io/apache/airflow-krb5-kdc-server:2021.07.04
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/integration-mongo.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
mongo:
image: mongo:3
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/integration-otel.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
otel-collector:
image: otel/opentelemetry-collector-contrib:0.70.0
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/integration-pinot.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
pinot:
image: apachepinot/pinot:0.8.0
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/integration-statsd.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
statsd-exporter:
image: apache/airflow:airflow-statsd-exporter-2020.09.05-v0.17.0
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/integration-trino.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
trino:
image: ghcr.io/apache/airflow-trino:359-2021.07.04
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/local-all-sources.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
airflow:
stdin_open: true # docker run -i
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/local.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
airflow:
stdin_open: true # docker run -i
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/mypy.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
airflow:
volumes:
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/docker-compose/remove-sources.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
---
version: "3.7"
services:
airflow:
# Removes airflow sources from container
Expand Down