[AIRFLOW-XXX] Group executors in one section (#5834)

docs/howto/executor/use-celery.rst → docs/executor/celery.rst

@@ -15,8 +15,8 @@
     specific language governing permissions and limitations
     under the License.
 
-Scaling Out with Celery
-=======================
+Celery Executor
+===============
 
 ``CeleryExecutor`` is one of the ways you can scale out the number of workers. For this
 to work, you need to setup a Celery backend (**RabbitMQ**, **Redis**, ...) and

docs/howto/executor/use-dask.rst → docs/executor/dask.rst

@@ -15,8 +15,8 @@
     specific language governing permissions and limitations
     under the License.
 
-Scaling Out with Dask
-=====================
+Dask Executor
+=============
 
 :class:`airflow.executors.dask_executor.DaskExecutor` allows you to run Airflow tasks in a Dask Distributed cluster.
 

docs/executor/index.rst

@@ -0,0 +1,30 @@
+..  Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+..    http://www.apache.org/licenses/LICENSE-2.0
+
+..  Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+Executor
+========
+
+Executors are the mechanism by which task instances get run.
+
+Airflow has support for various executors. Current used is determined by the ``executor`` option in the ``core``
+section of the configuration file.
+
+.. toctree::
+    :maxdepth: 1
+    :glob:
+
+    *

docs/executor/kubernetes.rst

@@ -0,0 +1,36 @@
+..  Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+..    http://www.apache.org/licenses/LICENSE-2.0
+
+..  Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+Kubernetes Executor
+===================
+
+The kubernetes executor is introduced in Apache Airflow 1.10.0. The Kubernetes executor will create a new pod for every task instance.
+
+Example helm charts are available at ``scripts/ci/kubernetes/kube/{airflow,volumes,postgres}.yaml`` in the source distribution.
+The volumes are optional and depend on your configuration. There are two volumes available:
+
+- **Dags**:
+
+  - By storing dags onto persistent disk, it will be made available to all workers
+
+  - Another option is to use ``git-sync``. Before starting the container, a git pull of the dags repository will be performed and used throughout the lifecycle of the pod
+
+- **Logs**:
+
+  - By storing logs onto a persistent disk, the files are accessible by workers and the webserver. If you don't configure this, the logs will be lost after the worker pods shuts down
+
+  - Another option is to use S3/GCS/etc to store logs

docs/howto/index.rst

@@ -34,8 +34,6 @@ configuring an Airflow environment.
     connection/index
     secure-connections
     write-logs
-    executor/use-celery
-    executor/use-dask
     run-behind-proxy
     run-with-systemd
     run-with-upstart

docs/index.rst

@@ -81,6 +81,7 @@ Content
     ui
     concepts
     scheduler
+    executor/index
     plugins
     security
     timezone

docs/kubernetes.rst

@@ -21,27 +21,14 @@ Kubernetes
 Kubernetes Executor
 ^^^^^^^^^^^^^^^^^^^
 
-The kubernetes executor is introduced in Apache Airflow 1.10.0. The Kubernetes executor will create a new pod for every task instance.
-
-Example helm charts are available at ``scripts/ci/kubernetes/kube/{airflow,volumes,postgres}.yaml`` in the source distribution.
-The volumes are optional and depend on your configuration. There are two volumes available:
-
-- **Dags**:
-
-  - By storing dags onto persistent disk, it will be made available to all workers
-
-  - Another option is to use ``git-sync``. Before starting the container, a git pull of the dags repository will be performed and used throughout the lifecycle of the pod
-
-- **Logs**:
-
-  - By storing logs onto a persistent disk, the files are accessible by workers and the webserver. If you don't configure this, the logs will be lost after the worker pods shuts down
-
-  - Another option is to use S3/GCS/etc to store logs
-
+The :doc:`Kubernetes Executor <executor/kubernetes>` allows you to run tasks on Kubernetes as Pods.
 
 Kubernetes Operator
 ^^^^^^^^^^^^^^^^^^^
 
+The :doc:`KubernetesPodOperator <executor/kubernetes>` allows you to create Pods on Kubernetes. It works with
+any type of executor.
+
 .. code:: python
 
     from airflow.contrib.operators import KubernetesOperator
@@ -146,8 +133,6 @@ Kubernetes Operator
                               )
 
 
-See :class:`airflow.contrib.operators.kubernetes_pod_operator.KubernetesPodOperator`
-
 
 Pod Mutation Hook
 ^^^^^^^^^^^^^^^^^