[AIRFLOW-4358] Speed up test_jobs by not running tasks

[ashb]

Make sure you have checked all steps below.

Jira

• https://issues.apache.org/jira/browse/AIRFLOW-4358

Description

• We don't care about the behaviour of the tasks (we test those elsewhere) - we just care that the task is "run" and yeilds a state, which we can all do more directly.

  On my laptop this takes the time for SchedulerJobTest from 133s to 35s, and reduces the BackfillJobTests from >5mins to mere seconds!

  I think this approach still tests the behaviour well enough

Tests

• Changed tests to check events that are run via executor, but it won't run it

Commits

• My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
  1. Subject is separated from body by a blank line
  2. Subject is limited to 50 characters (not including Jira issue reference)
  3. Subject does not end with a period
  4. Subject uses the imperative mood ("add", not "adding")
  5. Body wraps at 72 characters
  6. Body explains "what" and "why", not "how"

Documentation

• In case of new functionality, my PR adds documentation that describes how to use it.
  • All the public functions and the classes in the PR contain docstrings that explain what it does
  • If you implement backwards incompatible changes, please leave a note in the Updating.md so we can assign it to a appropriate release

Code Quality

• Passes flake8

[feng-tao]

Nice! It is always good to speed up the CI!

[ashb]

Looks like I might have some ordering in the tests to sort out. Sorted.

[ashb]

Imprecise numbers on tests - it looks like this saves 4mins on Sqlite tests, and ~10mins on mysql/postgres (where we run backfill!)

Before (master build):

TOX_ENV=py35-backend_mysql-env_docker 34 min 24 sec
TOX_ENV=py35-backend_sqlite-env_docker 25 min 37 sec
TOX_ENV=py35-backend_postgres-env_docker 35 min 52 sec
TOX_ENV=py35-backend_postgres-env_kubernetes KUBERNETES_VERSION=v1.13.0 28 min 11 sec

After:

TOX_ENV=py35-backend_mysql-env_docker 24 min 51 sec
TOX_ENV=py35-backend_sqlite-env_docker 21 min 28 sec
TOX_ENV=py35-backend_postgres-env_docker PYTHON_VERSION=3 26 min 19 sec
TOX_ENV=py35-backend_postgres-env_kubernetes KUBERNETES_VERSION=v1.13.0 27 min 33 sec

[codecov-io]

Codecov Report

Merging #5162 into master will decrease coverage by <.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #5162      +/-   ##
==========================================
- Coverage   78.56%   78.56%   -0.01%     
==========================================
  Files         469      469              
  Lines       29912    29912              
==========================================
- Hits        23501    23500       -1     
- Misses       6411     6412       +1

Impacted Files	Coverage Δ
airflow/jobs.py	78.49% <0%> (-0.18%)	⬇️
airflow/utils/dag_processing.py	59.47% <0%> (+0.17%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d63b6c9...902fc34. Read the comment docs.

[feng-tao]

LGTM

[potiuk]

@ashb -> i will have more time on weekend to take a look. But at first glance it looks good.

[feluelle]

Great work @ashb

I only found one thing I think can be improved here..

[potiuk]

I like this a lot!

[mik-laj]

LGTM. :-)