[AIRFLOW-3502] Add celery config option for setting "pool"

[gsilk]

Make sure you have checked all steps below.

Jira

• My PR addresses the following Airflow Jira issues and references them in the PR title. For example, "[AIRFLOW-XXX] My Airflow PR"
  • https://issues.apache.org/jira/browse/AIRFLOW-XXX
  • In case you are fixing a typo in the documentation you can prepend your commit with [AIRFLOW-XXX], code changes always need a Jira issue.

Description

• Here are some details about my PR, including screenshots of any UI changes:

Basically, this just adds a config option to the "celery" config for setting the "pool", which defaults to "prefork", but some users may want to use "solo" or other options.

Tests

• My PR adds the following unit tests OR does not need testing for this extremely good reason:

This is a fairly simple change. Also, I'm not sure how to test it.

Commits

• My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
  1. Subject is separated from body by a blank line
  2. Subject is limited to 50 characters (not including Jira issue reference)
  3. Subject does not end with a period
  4. Subject uses the imperative mood ("add", not "adding")
  5. Body wraps at 72 characters
  6. Body explains "what" and "why", not "how"

Documentation

• In case of new functionality, my PR adds documentation that describes how to use it.
  • When adding new operators/hooks/sensors, the autoclass documentation generation needs to be added.
  • All the public functions and the classes in the PR contain docstrings that explain what it does

Code Quality

• Passes flake8

[kppullin]

@gsilk - Should this be tagged with AIRFLOW-3502 ("Add config option to control celery pool used" instead of AIRFLOW-3501 ("Add config option to load dags in an image with the kubernetes executor.")?

[gsilk]

Thanks for catching that, I've updated the PR

[feng-tao]

lgtm, thanks @gsilk . will commit once ci is clear.

[codecov-io]

Codecov Report

Merging #4308 into master will decrease coverage by <.01%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master    #4308      +/-   ##
==========================================
- Coverage    78.1%   78.09%   -0.01%     
==========================================
  Files         201      201              
  Lines       16464    16466       +2     
==========================================
  Hits        12859    12859              
- Misses       3605     3607       +2

Impacted Files	Coverage Δ
airflow/bin/cli.py	64.43% <0%> (-0.16%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 57e355c...4245ce6. Read the comment docs.