Update security model of Airflow (#32098)

* Update security model of Airflow

This change updates the security model of Airflow to better explain
what are the capabilities of various kinds of users in Airflow
deployments and give both users and security researchers a way to
understand what security measures they can take and whether they
can qualify potential security issues in Airflow properly - taking
into account that various users of Airflow have various capabilities
and behaviours considered by some of the users as security
vulnerabilities, are standard capabilities of the users.

It also splits the security information of ours in two separate
pages:

* .github/SECURITY.md where we explain how to report the issues
  to Apache Airflow security team by the researchers

* documentation security/index.html which is available via
  Airflow Website where we explain what our security model is
  and the different kinds of users we have.

Both serve slightly different purpose and both contain cross-reference
links to each other in order to be able to redirect people who read
about the security model to find out how they can report the issues
but also to guide security researchers who want to assess whether
their findings are real vulnerabilities, or rather normal behaviours
following the Airflow Security model.

Security has been also moved to be a top level topic, so that it
is easier to find and navigate to. Old links have been redirected
to the new locations.

Also chapters were added explaining Airflow vs. Providers security
releases, what is the relation between Airflow and Providers
security issues and how users should treat security announcements
in providers.

* Update .github/SECURITY.md

Co-authored-by: Pankaj Koti <pankajkoti699@gmail.com>

* Apply Niko's suggestions from code review

Co-authored-by: Niko Oliveira <onikolas@amazon.com>

* fixup! Apply Niko's suggestions from code review

---------

Co-authored-by: Pankaj Koti <pankajkoti699@gmail.com>
Co-authored-by: Niko Oliveira <onikolas@amazon.com>

.github/SECURITY.rst → .github/SECURITY.md

@@ -1,61 +1,69 @@
- .. Licensed to the Apache Software Foundation (ASF) under one
-    or more contributor license agreements.  See the NOTICE file
-    distributed with this work for additional information
-    regarding copyright ownership.  The ASF licenses this file
-    to you under the Apache License, Version 2.0 (the
-    "License"); you may not use this file except in compliance
-    with the License.  You may obtain a copy of the License at
-
- ..   http://www.apache.org/licenses/LICENSE-2.0
-
- .. Unless required by applicable law or agreed to in writing,
-    software distributed under the License is distributed on an
-    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-    KIND, either express or implied.  See the License for the
-    specific language governing permissions and limitations
-    under the License.
-
-Security Model
---------------
-
-In the Airflow security model, the system administrators are fully trusted.
-They are the only ones who can upload new DAGs, which gives them the ability
-to execute any code on the server.
-
-Authenticated web interface and API users with Admin/Op permissions are trusted,
-but to a lesser extent: they can configure the DAGs which gives them some control,
-but not arbitrary code execution.
-
-Authenticated Web interface and API users with 'regular' permissions are trusted
-to the point where they can impact resource consumption and pause/unpause configured DAGs,
-but not otherwise influence their functionality.
-
-Reporting Vulnerabilities
--------------------------
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements.  See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership.  The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied.  See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+This document contains information on how to report security vulnerabilities in Apache Airflow and
+how the security issues reported to Apache Airflow security team are handled. If you would like
+to learn about the security model of Airflow head to
+[Airflow Security](https://airflow.apache.org/docs/apache-airflow/stable/security/)
+
+## Reporting Vulnerabilities
 
 **⚠️ Please do not file GitHub issues for security vulnerabilities as they are public! ⚠️**
 
 The Apache Software Foundation takes security issues very seriously. Apache
 Airflow specifically offers security features and is responsive to issues
 around its features. If you have any concern around Airflow Security or believe
 you have uncovered a vulnerability, we suggest that you get in touch via the
-e-mail address security@airflow.apache.org. In the message, try to provide a
-description of the issue and ideally a way of reproducing it. The security team
-will get back to you after assessing the description.
+e-mail address [security@airflow.apache.org](mailto:security@airflow.apache.org).
+In the message, try to provide a description of the issue and ideally a way of
+reproducing it. The security team will get back to you after assessing the report.
 
 Note that this security address should be used only for undisclosed
 vulnerabilities. Dealing with fixed issues or general questions on how to use
 the security features should be handled regularly via the user and the dev
 lists. Please report any security problems to the project security address
 before disclosing it publicly.
 
-The `ASF Security team's page <https://www.apache.org/security/>`_ describes
-how vulnerability reports are handled, and includes PGP keys if you wish to use
-that.
+Before reporting vulnerabilities, please make sure to read and understand the
+[security model](https://airflow.apache.org/docs/apache-airflow/stable/security/) of Airflow, because
+some of the potential security vulnerabilities that are valid for projects that are publicly accessible
+from the Internet, are not valid for Airflow. Airflow is not designed to be used by untrusted users, and some
+trusted users are trusted enough to do a variety of operations that could be considered as vulnerabilities
+in other products/circumstances. Therefore, some potential security vulnerabilities do not
+apply to Airflow, or have a different severity than some generic scoring systems (for example `CVSS`)
+calculation suggests.
 
+The [ASF Security team's page](https://www.apache.org/security/) describes
+how vulnerability reports are handled in general by all ASF projects, and includes PGP keys if
+you wish to use them when you report the issues.
 
-Handling security issues in Airflow
------------------------------------
+## Security vulnerabilities in Airflow and Airflow community managed providers
+
+Airflow core package is released separately from provider packages. While Airflow comes with ``constraints``
+which describe which version of providers have been tested when the version of Airflow was released, the
+users of Airflow are advised to install providers independently from Airflow core when they want to apply
+security fixes found and released in providers. Therefore, the issues found and fixed in providers do
+not apply to the Airflow core package. There are also Airflow providers released by 3rd-parties, but the
+Airflow community is not responsible for releasing and announcing security vulnerabilities in them, this
+is handled entirely by the 3rd-parties that release their own providers.
+
+## Handling security issues in Airflow
 
 The security issues in Airflow are handled by the Airflow Security Team. The team consists
 of selected PMC members that are interested in looking at, discussing about and fixing the
@@ -80,7 +88,7 @@ There are certain expectations from the members of the security team:
   experts that are available through Airflow stakeholders. The intent about involving 3rd parties has
   to be discussed and agreed up at security@airflow.apache.org.
 
-* They have to have an `ICLA <https://www.apache.org/licenses/contributor-agreements.html>`_ signed with
+* They have to have an [ICLA](https://www.apache.org/licenses/contributor-agreements.html) signed with
   Apache Software Foundation.
 
 * The security team members might inform 3rd parties about fixes, for example in order to assess if the fix
@@ -92,7 +100,7 @@ There are certain expectations from the members of the security team:
   with the intent of minimizing the time between the fix being available and the fix being released. In this
   case the PR might be sent to review and comment to the PMC members on private list, in order to request
   an expedited voting on the release. The voting for such release might be done on the
-  ``private@airflow.apache.org`` mailing list and should be made public at the ``dev@apache.airflow.org``
+  `private@airflow.apache.org` mailing list and should be made public at the `dev@apache.airflow.org`
   mailing list as soon as the release is ready to be announced.
 
 * The security team members working on the fix might be mentioned as remediation developers in the CVE
@@ -104,22 +112,5 @@ There are certain expectations from the members of the security team:
   release process. This is facilitated by the security tool provided by the Apache Software Foundation.
 
 * Severity of the issue is determined based on the criteria described in the
-  `Severity Rating blog post <https://security.apache.org/blog/severityrating/>`_  by the Apache Software
+  [Severity Rating blog post](https://security.apache.org/blog/severityrating/)  by the Apache Software
   Foundation Security team
-
-Releasing Airflow with security patches
----------------------------------------
-
-Apache Airflow uses strict `SemVer <https://semver.org>`_ versioning policy, which means that we strive for
-any release of a given ``MAJOR`` Version (version "2" currently) to be backwards compatible. When we
-release ``MINOR`` version, the development continues in the ``main`` branch where we prepare the next
-``MINOR`` version, but we release ``PATCHLEVEL`` releases with selected bugfixes (including security
-bugfixes) cherry-picked to the latest released ``MINOR`` line of Apache Airflow. At the moment, when we
-release a new ``MINOR`` version, we stop releasing ``PATCHLEVEL`` releases for the previous ``MINOR`` version.
-
-For example, when we released  ``2.6.0`` version on April 30, 2023, until we release ``2.7.0`` version,
-all the security patches will be cherry-picked and released in ``2.6.*`` versions only. There will be no
-``2.5.*`` versions  released after ``2.6.0`` has been released.
-
-This means that in order to apply security fixes with Apache Airflow software released by us, you
-MUST upgrade to the latest ``MINOR`` version of Airflow.

.github/boring-cyborg.yml

@@ -270,7 +270,7 @@ labelPRBasedOnFilePath:
     - airflow/providers/**/secrets/*
     - tests/secrets/**/*
     - tests/providers/**/secrets/*
-    - docs/apache-airflow/administration-and-deployment/security/secrets/**/*
+    - docs/apache-airflow/security/secrets/**/*
 
   area:Triggerer:
     - airflow/cli/commands/triggerer_command.py

.pre-commit-config.yaml

@@ -495,7 +495,7 @@ repos:
           ^airflow/providers/microsoft/winrm/hooks/winrm.py$|
           ^airflow/www/fab_security/manager.py$|
           ^docs/.*commits.rst$|
-          ^docs/apache-airflow/administration-and-deployment/security/webserver.rst$|
+          ^docs/apache-airflow/security/webserver.rst$|
           ^docs/apache-airflow-providers-apache-cassandra/connections/cassandra.rst$|
           ^airflow/providers/microsoft/winrm/operators/winrm.py$|
           ^airflow/providers/opsgenie/hooks/opsgenie.py$|

docs/apache-airflow-providers/core-extensions/auth-backends.rst

@@ -26,7 +26,7 @@ capabilities. You can read more about those in
 `FAB security docs <https://flask-appbuilder.readthedocs.io/en/latest/security.html>`_.
 
 You can also
-take a look at Auth backends available in the core Airflow in :doc:`apache-airflow:administration-and-deployment/security/webserver`
+take a look at Auth backends available in the core Airflow in :doc:`apache-airflow:security/webserver`
 or see those provided by the community-managed providers:
 
 .. airflow-auth-backends::

docs/apache-airflow-providers/core-extensions/secrets-backends.rst

@@ -28,7 +28,7 @@ via providers that implement secrets backends for services Airflow integrates wi
 
 You can also take a
 look at Secret backends available in the core Airflow in
-:doc:`apache-airflow:administration-and-deployment/security/secrets/secrets-backend/index` and here you can see the ones
+:doc:`apache-airflow:security/secrets/secrets-backend/index` and here you can see the ones
 provided by the community-managed providers:
 
 .. airflow-secrets-backends::

docs/apache-airflow/administration-and-deployment/index.rst

@@ -24,7 +24,6 @@ This section contains information about deploying DAGs into production and the a
     :maxdepth: 2
 
     production-deployment
-    security/index
     logging-monitoring/index
     kubernetes
     lineage

docs/apache-airflow/howto/connection.rst

@@ -27,7 +27,7 @@ Airflow's :class:`~airflow.models.connection.Connection` object is used for stor
 Connections may be defined in the following ways:
 
   - in :ref:`environment variables <environment_variables_connections>`
-  - in an external :doc:`/administration-and-deployment/security/secrets/secrets-backend/index`
+  - in an external :doc:`/security/secrets/secrets-backend/index`
   - in the :ref:`Airflow metadata database <connections-in-database>`
     (using the :ref:`CLI <connection/cli>` or :ref:`web UI <creating_connection_ui>`)
 
@@ -86,15 +86,15 @@ See :ref:`Connection URI format <connection-uri-format>` for more details on how
 Storing connections in a Secrets Backend
 ----------------------------------------
 
-You can store Airflow connections in external secrets backends like HashiCorp Vault, AWS SSM Parameter Store, and other such services. For more details see :doc:`/administration-and-deployment/security/secrets/secrets-backend/index`.
+You can store Airflow connections in external secrets backends like HashiCorp Vault, AWS SSM Parameter Store, and other such services. For more details see :doc:`/security/secrets/secrets-backend/index`.
 
 .. _connections-in-database:
 
 Storing connections in the database
 -----------------------------------
 .. seealso::
 
-    Connections can alternatively be stored in :ref:`environment variables <environment_variables_connections>` or an :doc:`external secrets backend </administration-and-deployment/security/secrets/secrets-backend/index>` such as HashiCorp Vault, AWS SSM Parameter Store, etc.
+    Connections can alternatively be stored in :ref:`environment variables <environment_variables_connections>` or an :doc:`external secrets backend </security/secrets/secrets-backend/index>` such as HashiCorp Vault, AWS SSM Parameter Store, etc.
 
 When storing connections in the database, you may manage them using either the web UI or the Airflow CLI.
 

docs/apache-airflow/howto/variable.rst

@@ -73,4 +73,4 @@ It guarantees that without the encryption password, content cannot be manipulate
 without the key. For information on configuring Fernet, look at :ref:`security/fernet`.
 
 In addition to retrieving variables from environment variables or the metastore database, you can enable
-a secrets backend to retrieve variables. For more details see :doc:`/administration-and-deployment/security/secrets/secrets-backend/index`.
+a secrets backend to retrieve variables. For more details see :doc:`/security/secrets/secrets-backend/index`.

docs/apache-airflow/index.rst

@@ -133,6 +133,7 @@ so coding will always be required.
     Overview <self>
     start
     installation/index
+    security/index
     tutorial/index
     howto/index
     ui

docs/apache-airflow/installation/installing-from-pypi.rst

@@ -272,6 +272,7 @@ released and tested together when the version of Airflow you are installing was
     CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt"
     pip install "apache-airflow[postgres,google]==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}"
 
+.. _installing-from-pypi-managing-providers-separately-from-airflow-core:
 
 Managing providers separately from Airflow core
 ===============================================

docs/apache-airflow/integration.rst

@@ -20,18 +20,18 @@ Integration
 
 Airflow has a mechanism that allows you to expand its functionality and integrate with other systems.
 
-* :doc:`API Authentication backends </administration-and-deployment/security/api>`
+* :doc:`API Authentication backends </security/api>`
 * :doc:`Email backends </howto/email-config>`
 * :doc:`Executor </core-concepts/executor/index>`
-* :doc:`Kerberos </administration-and-deployment/security/kerberos>`
+* :doc:`Kerberos </security/kerberos>`
 * :doc:`Logging </administration-and-deployment/logging-monitoring/logging-tasks>`
 * :doc:`Metrics (statsd) </administration-and-deployment/logging-monitoring/metrics>`
 * :doc:`Operators and hooks </operators-and-hooks-ref>`
 * :doc:`Plugins </authoring-and-scheduling/plugins>`
 * :doc:`Listeners </administration-and-deployment/listeners>`
-* :doc:`Secrets backends </administration-and-deployment/security/secrets/secrets-backend/index>`
+* :doc:`Secrets backends </security/secrets/secrets-backend/index>`
 * :doc:`Tracking systems </administration-and-deployment/logging-monitoring/tracking-user-activity>`
-* :doc:`Web UI Authentication backends </administration-and-deployment/security/api>`
+* :doc:`Web UI Authentication backends </security/api>`
 * :doc:`Serialization </authoring-and-scheduling/serializers>`
 
 It also has integration with :doc:`Sentry </administration-and-deployment/logging-monitoring/errors>` service for error tracking. Other applications can also integrate using

docs/apache-airflow/public-airflow-interface.rst

@@ -339,7 +339,7 @@ All Secrets Backend implementations are public. You can extend their functionali
 
   _api/airflow/secrets/index
 
-You can read more about Secret Backends in :doc:`administration-and-deployment/security/secrets/secrets-backend/index`.
+You can read more about Secret Backends in :doc:`security/secrets/secrets-backend/index`.
 You can also find all the available Secrets Backends implemented in community providers
 in :doc:`apache-airflow-providers:core-extensions/secrets-backends`.