Skip to content
Permalink
Browse files
Remove references to python 3.6 from doc (#590)
  • Loading branch information
ephraimbuddy committed May 4, 2022
1 parent e06b6c8 commit 35be74a36aa32fde856f0b340ee8639947ec567c
Showing 25 changed files with 139 additions and 131 deletions.
@@ -96,7 +96,7 @@ with a consistent set of dependencies based on constraint files provided by Airf
:substitutions:
pip install apache-airflow[google,amazon,apache.spark]==|version| \
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-|version|/constraints-3.6.txt"
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-|version|/constraints-3.7.txt"
Note, that this will install providers in the versions that were released at the time of Airflow |version| release. You can later
upgrade those providers manually if you want to use latest versions of the providers.
@@ -86,7 +86,7 @@ You can make sure which version is used by the interpreter by running this check
.. code-block:: bash
root@b8a8e73caa2c:/opt/airflow# python
Python 3.6.12 (default, Nov 25 2020, 03:59:00)
Python 3.8.10 (default, Mar 15 2022, 12:22:08)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
@@ -37,7 +37,7 @@ Typical command to install airflow from PyPI looks like below:

.. code-block::

pip install "apache-airflow[celery]==|version|" --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-|version|/constraints-3.6.txt"
pip install "apache-airflow[celery]==|version|" --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-|version|/constraints-3.7.txt"

This is an example, see further for more explanation.

@@ -142,7 +142,7 @@ in specific version.
.. code-block:: bash
PYTHON_VERSION="$(python --version | cut -d " " -f 2 | cut -d "." -f 1-2)"
# For example: 3.6
# For example: 3.7
CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-${PYTHON_VERSION}.txt"
pip install "apache-airflow-providers-google" --constraint "${CONSTRAINT_URL}"
@@ -151,7 +151,7 @@ You can also upgrade the providers to latest versions (you need to use main vers
.. code-block:: bash
PYTHON_VERSION="$(python --version | cut -d " " -f 2 | cut -d "." -f 1-2)"
# For example: 3.6
# For example: 3.7
CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-${PYTHON_VERSION}.txt"
pip install "apache-airflow-providers-google" --upgrade --constraint "${CONSTRAINT_URL}"
@@ -166,9 +166,9 @@ If you don't want to install any extra providers, initially you can use the comm
AIRFLOW_VERSION=|version|
PYTHON_VERSION="$(python --version | cut -d " " -f 2 | cut -d "." -f 1-2)"
# For example: 3.6
# For example: 3.7
CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-no-providers-${PYTHON_VERSION}.txt"
# For example: https://raw.githubusercontent.com/apache/airflow/constraints-|version|/constraints-no-providers-3.6.txt
# For example: https://raw.githubusercontent.com/apache/airflow/constraints-|version|/constraints-no-providers-3.7.txt
pip install "apache-airflow==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}"
@@ -39,7 +39,7 @@ using the latest stable version of SQLite for local development.

**Note**: Python v3.10 is not supported yet. For details, see `#19059 <https://github.com/apache/airflow/issues/19059>`__.

Starting with Airflow 2.1.2, Airflow is tested with Python 3.6, 3.7, 3.8, and 3.9.
Starting with Airflow 2.3.0, Airflow is tested with Python 3.7, 3.8, 3.9 and 3.10.

The minimum memory required we recommend Airflow to run with is 4GB, but the actual requirements depends
wildly on the deployment options you have
@@ -49,9 +49,9 @@ constraint files to enable reproducible installation, so using ``pip`` and const
# Install Airflow using the constraints file
AIRFLOW_VERSION=|version|
PYTHON_VERSION="$(python --version | cut -d " " -f 2 | cut -d "." -f 1-2)"
# For example: 3.6
# For example: 3.7
CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt"
# For example: https://raw.githubusercontent.com/apache/airflow/constraints-|version|/constraints-3.6.txt
# For example: https://raw.githubusercontent.com/apache/airflow/constraints-|version|/constraints-3.7.txt
pip install "apache-airflow==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}"
# The Standalone command will initialise the database, make a user,
@@ -34,6 +34,8 @@ Airflow 1.10 was the last release series to support Python 2. Airflow 2.0.0
requires Python 3.6+ and has been tested with Python versions 3.6, 3.7 and 3.8.
Python 3.9 support was added from Airflow 2.1.2.

Airflow 2.3.0 dropped support for Python 3.6. It's tested with Python 3.7, 3.8, 3.9 and 3.10.

If you have a specific task that still requires Python 2 then you can use the :class:`~airflow.operators.python.PythonVirtualenvOperator` or the ``KubernetesPodOperator`` for this.

For a list of breaking changes between Python 2 and Python 3, please refer to this

Large diffs are not rendered by default.

@@ -550,24 +550,24 @@ Building from PyPI packages

This is the basic way of building the custom images from sources.

The following example builds the production image in version ``3.6`` with latest PyPI-released Airflow,
with default set of Airflow extras and dependencies. The ``2.0.2`` constraints are used automatically.
The following example builds the production image in version ``3.7`` with latest PyPI-released Airflow,
with default set of Airflow extras and dependencies. The latest PyPI-released Airflow constraints are used automatically.

.. exampleinclude:: docker-examples/customizing/stable-airflow.sh
:language: bash
:start-after: [START build]
:end-before: [END build]

The following example builds the production image in version ``3.7`` with default extras from ``2.0.2`` PyPI
package. The ``2.0.2`` constraints are used automatically.
The following example builds the production image in version ``3.7`` with default extras from ``2.3.0`` Airflow
package. The ``2.3.0`` constraints are used automatically.

.. exampleinclude:: docker-examples/customizing/pypi-selected-version.sh
:language: bash
:start-after: [START build]
:end-before: [END build]

The following example builds the production image in version ``3.8`` with additional airflow extras
(``mssql,hdfs``) from ``2.0.2`` PyPI package, and additional dependency (``oauth2client``).
(``mssql,hdfs``) from ``2.3.0`` PyPI package, and additional dependency (``oauth2client``).

.. exampleinclude:: docker-examples/customizing/pypi-extras-and-deps.sh
:language: bash
@@ -593,7 +593,7 @@ have more complex dependencies to build.
Building optimized images
.........................

The following example the production image in version ``3.6`` with additional airflow extras from ``2.0.2``
The following example the production image in version ``3.7`` with additional airflow extras from ``2.0.2``
PyPI package but it includes additional apt dev and runtime dependencies.

The dev dependencies are those that require ``build-essential`` and usually need to involve recompiling
@@ -1869,14 +1869,14 @@ <h6>get-value<a class="headerlink" href="#get-value" title="Permalink to this he
<div class="section" id="list">
<h6>list<a class="headerlink" href="#list" title="Permalink to this headline"></a></h6>
<p>List options for the configuration</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">airflow</span> <span class="n">config</span> <span class="nb">list</span> <span class="p">[</span><span class="o">-</span><span class="n">h</span><span class="p">]</span> <span class="p">[</span><span class="o">--</span><span class="n">color</span> <span class="p">{</span><span class="n">auto</span><span class="p">,</span><span class="n">on</span><span class="p">,</span><span class="n">off</span><span class="p">}]</span>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">airflow</span> <span class="n">config</span> <span class="nb">list</span> <span class="p">[</span><span class="o">-</span><span class="n">h</span><span class="p">]</span> <span class="p">[</span><span class="o">--</span><span class="n">color</span> <span class="p">{</span><span class="n">off</span><span class="p">,</span><span class="n">auto</span><span class="p">,</span><span class="n">on</span><span class="p">}]</span>
</pre></div>
</div>
<div class="section" id="Named Arguments_repeat4">
<h7>Named Arguments<a class="headerlink" href="#Named Arguments_repeat4" title="Permalink to this headline"></a></h7>
<dl class="option-list">
<dt><kbd>--color</kbd></dt>
<dd><p>Possible choices: auto, on, off</p>
<dd><p>Possible choices: off, auto, on</p>
<p>Do emit colored output (default: auto)</p>
<p>Default: “auto”</p>
</dd>
@@ -1960,7 +1960,7 @@ <h6>add<a class="headerlink" href="#add" title="Permalink to this headline">¶</
<div class="section" id="delete">
<h6>delete<a class="headerlink" href="#delete" title="Permalink to this headline"></a></h6>
<p>Delete a connection</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">airflow</span> <span class="n">connections</span> <span class="n">delete</span> <span class="p">[</span><span class="o">-</span><span class="n">h</span><span class="p">]</span> <span class="p">[</span><span class="o">--</span><span class="n">color</span> <span class="p">{</span><span class="n">auto</span><span class="p">,</span><span class="n">on</span><span class="p">,</span><span class="n">off</span><span class="p">}]</span> <span class="n">conn_id</span>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">airflow</span> <span class="n">connections</span> <span class="n">delete</span> <span class="p">[</span><span class="o">-</span><span class="n">h</span><span class="p">]</span> <span class="p">[</span><span class="o">--</span><span class="n">color</span> <span class="p">{</span><span class="n">off</span><span class="p">,</span><span class="n">auto</span><span class="p">,</span><span class="n">on</span><span class="p">}]</span> <span class="n">conn_id</span>
</pre></div>
</div>
<div class="section" id="Positional Arguments_repeat6">
@@ -1975,7 +1975,7 @@ <h6>delete<a class="headerlink" href="#delete" title="Permalink to this headline
<h7>Named Arguments<a class="headerlink" href="#Named Arguments_repeat6" title="Permalink to this headline"></a></h7>
<dl class="option-list">
<dt><kbd>--color</kbd></dt>
<dd><p>Possible choices: auto, on, off</p>
<dd><p>Possible choices: off, auto, on</p>
<p>Do emit colored output (default: auto)</p>
<p>Default: “auto”</p>
</dd>
@@ -2028,7 +2028,7 @@ <h6>export<a class="headerlink" href="#export" title="Permalink to this headline
<div class="section" id="get">
<h6>get<a class="headerlink" href="#get" title="Permalink to this headline"></a></h6>
<p>Get a connection</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">airflow</span> <span class="n">connections</span> <span class="n">get</span> <span class="p">[</span><span class="o">-</span><span class="n">h</span><span class="p">]</span> <span class="p">[</span><span class="o">--</span><span class="n">color</span> <span class="p">{</span><span class="n">auto</span><span class="p">,</span><span class="n">on</span><span class="p">,</span><span class="n">off</span><span class="p">}]</span>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">airflow</span> <span class="n">connections</span> <span class="n">get</span> <span class="p">[</span><span class="o">-</span><span class="n">h</span><span class="p">]</span> <span class="p">[</span><span class="o">--</span><span class="n">color</span> <span class="p">{</span><span class="n">off</span><span class="p">,</span><span class="n">auto</span><span class="p">,</span><span class="n">on</span><span class="p">}]</span>
<span class="p">[</span><span class="o">-</span><span class="n">o</span> <span class="n">table</span><span class="p">,</span> <span class="n">json</span><span class="p">,</span> <span class="n">yaml</span><span class="p">,</span> <span class="n">plain</span><span class="p">]</span> <span class="p">[</span><span class="o">-</span><span class="n">v</span><span class="p">]</span>
<span class="n">conn_id</span>
</pre></div>
@@ -2045,7 +2045,7 @@ <h6>get<a class="headerlink" href="#get" title="Permalink to this headline">¶</
<h7>Named Arguments<a class="headerlink" href="#Named Arguments_repeat8" title="Permalink to this headline"></a></h7>
<dl class="option-list">
<dt><kbd>--color</kbd></dt>
<dd><p>Possible choices: auto, on, off</p>
<dd><p>Possible choices: off, auto, on</p>
<p>Do emit colored output (default: auto)</p>
<p>Default: “auto”</p>
</dd>
@@ -3331,7 +3331,7 @@ <h6>behaviours<a class="headerlink" href="#behaviours" title="Permalink to this
<div class="section" id="get_repeat2">
<h6>get<a class="headerlink" href="#get_repeat2" title="Permalink to this headline"></a></h6>
<p>Get detailed information about a provider</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">airflow</span> <span class="n">providers</span> <span class="n">get</span> <span class="p">[</span><span class="o">-</span><span class="n">h</span><span class="p">]</span> <span class="p">[</span><span class="o">--</span><span class="n">color</span> <span class="p">{</span><span class="n">auto</span><span class="p">,</span><span class="n">on</span><span class="p">,</span><span class="n">off</span><span class="p">}]</span> <span class="p">[</span><span class="o">-</span><span class="n">f</span><span class="p">]</span>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">airflow</span> <span class="n">providers</span> <span class="n">get</span> <span class="p">[</span><span class="o">-</span><span class="n">h</span><span class="p">]</span> <span class="p">[</span><span class="o">--</span><span class="n">color</span> <span class="p">{</span><span class="n">off</span><span class="p">,</span><span class="n">auto</span><span class="p">,</span><span class="n">on</span><span class="p">}]</span> <span class="p">[</span><span class="o">-</span><span class="n">f</span><span class="p">]</span>
<span class="p">[</span><span class="o">-</span><span class="n">o</span> <span class="n">table</span><span class="p">,</span> <span class="n">json</span><span class="p">,</span> <span class="n">yaml</span><span class="p">,</span> <span class="n">plain</span><span class="p">]</span> <span class="p">[</span><span class="o">-</span><span class="n">v</span><span class="p">]</span>
<span class="n">provider_name</span>
</pre></div>
@@ -3348,7 +3348,7 @@ <h6>get<a class="headerlink" href="#get_repeat2" title="Permalink to this headli
<h7>Named Arguments<a class="headerlink" href="#Named Arguments_repeat45" title="Permalink to this headline"></a></h7>
<dl class="option-list">
<dt><kbd>--color</kbd></dt>
<dd><p>Possible choices: auto, on, off</p>
<dd><p>Possible choices: off, auto, on</p>
<p>Do emit colored output (default: auto)</p>
<p>Default: “auto”</p>
</dd>
@@ -784,7 +784,7 @@ <h2>Providers extras<a class="headerlink" href="#providers-extras" title="Permal
</div></blockquote>
<p>with a consistent set of dependencies based on constraint files provided by Airflow Community at the time 2.3.0 version was released.</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>pip install apache-airflow<span class="o">[</span>google,amazon,apache.spark<span class="o">]==</span><span class="m">2</span>.3.0 <span class="se">\</span>
--constraint <span class="s2">&quot;https://raw.githubusercontent.com/apache/airflow/constraints-2.3.0/constraints-3.6.txt&quot;</span>
--constraint <span class="s2">&quot;https://raw.githubusercontent.com/apache/airflow/constraints-2.3.0/constraints-3.7.txt&quot;</span>
</pre></div>
</div>
<p>Note, that this will install providers in the versions that were released at the time of Airflow 2.3.0 release. You can later
@@ -756,7 +756,7 @@ <h2>Setting up a SQLite Database<a class="headerlink" href="#setting-up-a-sqlite
<code class="docutils literal notranslate"><span class="pre">LD_LIBRARY_PATH</span></code> set for the python interpreter that is used to start Airflow.</p>
<p>You can make sure which version is used by the interpreter by running this check:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>root@b8a8e73caa2c:/opt/airflow# python
Python <span class="m">3</span>.6.12 <span class="o">(</span>default, Nov <span class="m">25</span> <span class="m">2020</span>, <span class="m">03</span>:59:00<span class="o">)</span>
Python <span class="m">3</span>.8.10 <span class="o">(</span>default, Mar <span class="m">15</span> <span class="m">2022</span>, <span class="m">12</span>:22:08<span class="o">)</span>
<span class="o">[</span>GCC <span class="m">8</span>.3.0<span class="o">]</span> on linux
Type <span class="s2">&quot;help&quot;</span>, <span class="s2">&quot;copyright&quot;</span>, <span class="s2">&quot;credits&quot;</span> or <span class="s2">&quot;license&quot;</span> <span class="k">for</span> more information.
&gt;&gt;&gt; import sqlite3

0 comments on commit 35be74a

Please sign in to comment.