Skip to content

Commit

Permalink
Update documentation about bundle extras (#18828)
Browse files Browse the repository at this point in the history
  • Loading branch information
@potiuk
potiuk committed Oct 8, 2021
1 parent 1a35644 commit cf98586
Show file tree
Hide file tree
Showing 2 changed files with 21 additions and 21 deletions.
30 changes: 15 additions & 15 deletions docs/apache-airflow/extra-packages-ref.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -290,21 +290,21 @@ Bundle extras

Those are extras that install one ore more extras as a bundle.

+---------------------+-----------------------------------------------------+----------------------------------------------------------------------+
| extra | install command | enables |
+=====================+=====================================================+======================================================================+
| all | ``pip install 'apache-airflow[all]'`` | All Airflow user facing features (no devel and doc requirements) |
+---------------------+-----------------------------------------------------+----------------------------------------------------------------------+
| all_dbs | ``pip install 'apache-airflow[all_dbs]'`` | All databases integrations |
+---------------------+-----------------------------------------------------+----------------------------------------------------------------------+
| devel | ``pip install 'apache-airflow[devel]'`` | Minimum dev tools requirements (without providers) |
+---------------------+-----------------------------------------------------+----------------------------------------------------------------------+
| devel_hadoop | ``pip install 'apache-airflow[devel_hadoop]'`` | Same as ``devel`` + dependencies for developing the Hadoop stack |
+---------------------+-----------------------------------------------------+----------------------------------------------------------------------+
| devel_all | ``pip install 'apache-airflow[devel_all]'`` | Everything needed for development (``devel_hadoop`` + providers) |
+---------------------+-----------------------------------------------------+----------------------------------------------------------------------+
| devel_ci | ``pip install 'apache-airflow[devel_ci]'`` | All dependencies required for CI tests. |
+---------------------+-----------------------------------------------------+----------------------------------------------------------------------+
+---------------------+-----------------------------------------------------+------------------------------------------------------------------------+
| extra | install command | enables |
+=====================+=====================================================+========================================================================+
| all | ``pip install 'apache-airflow[all]'`` | All Airflow user facing features (no devel and doc requirements) |
+---------------------+-----------------------------------------------------+------------------------------------------------------------------------+
| all_dbs | ``pip install 'apache-airflow[all_dbs]'`` | All database integrations |
+---------------------+-----------------------------------------------------+------------------------------------------------------------------------+
| devel | ``pip install 'apache-airflow[devel]'`` | Minimum development dependencies (without Hadoop, Kerberos, providers) |
+---------------------+-----------------------------------------------------+------------------------------------------------------------------------+
| devel_hadoop | ``pip install 'apache-airflow[devel_hadoop]'`` | Adds Hadoop stack libraries to ``devel`` dependencies |
+---------------------+-----------------------------------------------------+------------------------------------------------------------------------+
| devel_all | ``pip install 'apache-airflow[devel_all]'`` | Everything needed for development including Hadoop and providers |
+---------------------+-----------------------------------------------------+------------------------------------------------------------------------+
| devel_ci | ``pip install 'apache-airflow[devel_ci]'`` | All dependencies required for CI tests (same as ``devel_all``) |
+---------------------+-----------------------------------------------------+------------------------------------------------------------------------+

Doc extras
----------
Expand Down
12 changes: 6 additions & 6 deletions setup.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -497,7 +497,7 @@ def write_version(filename: str = os.path.join(*[my_dir, "airflow", "git_version
]
# End dependencies group

devel = [
devel_only = [
'aws_xray_sdk',
'beautifulsoup4~=4.7.1',
'black',
Expand Down Expand Up @@ -541,8 +541,8 @@ def write_version(filename: str = os.path.join(*[my_dir, "airflow", "git_version
'yamllint',
]

devel_minreq = cgroups + devel + doc + kubernetes + mysql + pandas + password
devel_hadoop = devel_minreq + hdfs + hive + kerberos + presto + webhdfs
devel = cgroups + devel_only + doc + kubernetes + mysql + pandas + password
devel_hadoop = devel + hdfs + hive + kerberos + presto + webhdfs

# Dict of all providers which are part of the Apache Airflow repository together with their requirements
PROVIDERS_REQUIREMENTS: Dict[str, List[str]] = {
Expand Down Expand Up @@ -778,7 +778,7 @@ def add_all_deprecated_provider_packages() -> None:

# This can be simplified to devel_hadoop + _all_requirements due to inclusions
# but we keep it for explicit sake. We are de-duplicating it anyway.
devel_all = list(set(_all_requirements + doc + devel_minreq + devel_hadoop))
devel_all = list(set(_all_requirements + doc + devel + devel_hadoop))

# Those are packages excluded for "all" dependencies
PACKAGES_EXCLUDED_FOR_ALL = []
Expand Down Expand Up @@ -812,8 +812,8 @@ def is_package_excluded(package: str, exclusion_list: List[str]) -> bool:
# Those are extras that we have to add for development purposes
# They can be use to install some predefined set of dependencies.
EXTRAS_REQUIREMENTS["doc"] = doc
EXTRAS_REQUIREMENTS["devel"] = devel_minreq # devel_minreq already includes doc
EXTRAS_REQUIREMENTS["devel_hadoop"] = devel_hadoop # devel_hadoop already includes devel_minreq
EXTRAS_REQUIREMENTS["devel"] = devel # devel already includes doc
EXTRAS_REQUIREMENTS["devel_hadoop"] = devel_hadoop # devel_hadoop already includes devel
EXTRAS_REQUIREMENTS["devel_all"] = devel_all
EXTRAS_REQUIREMENTS["devel_ci"] = devel_ci

Expand Down

0 comments on commit cf98586

Please sign in to comment.