Fix max_active_runs not allowing moving of queued dagruns to running

[ephraimbuddy]

Currently, if you set max_active_runs for a dag and that dag has many queued dagruns
with execution dates older than another dag's queued dagruns, airflow will not move
the newer queued dagruns to running with the effect that only one dagruns would be in running at any time

This PR fixes this by updating the DagRun.last_scheduling_decision whenever a decision of scheduling
was made

Related: #14205

---

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

[ephraimbuddy]

What happens is that the method DagRun.next_dagruns_to_examine gets the earliest dagruns without considering the dag that has the dagrun.
For example:
If you have a dag with execution_date 2020,1,1 and set catchup=True, max_active_runs=1, schedule_interval='@daily' and another dag with execution_date 2021,1,1 and also set catchup=True, schedule_interval='@daily'.
When you unpause the two dags(the one with max_active_runs first), the dagruns would be created but only one dagrun would be active because of how DagRun.next_dagruns_to_examine works.
(worried about performance on this PR. Please advice)

[ashb]

will not schedule the newer dagruns with effect that only one dagruns would be in running at any time

When you say wont schedule -- do you mean that the dagruns will take a long time to get out of queued state, but once a dag run is in running state tasks should be scheduled fine -- cos the code you have changed only affects DagRuns going from queued -> running state -- nothing there will affect task scheduling.

[ephraimbuddy]

will not schedule the newer dagruns with effect that only one dagruns would be in running at any time

When you say wont schedule -- do you mean that the dagruns will take a long time to get out of queued state, but once a dag run is in running state tasks should be scheduled fine -- cos the code you have changed only affects DagRuns going from queued -> running state -- nothing there will affect task scheduling.

You are right. Not scheduling but moving from queued to running, I will update it rightly now

[github-actions]

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

[dhuang]

FWIW can confirm this fixed my issue in #17638, so big thanks! Note I did have an issue after upgrading where the max_active_runs columns for some DAGs remained NULL even after scheduler parsed them. This seemed to prevent their tasks from getting scheduled. I eventually set it manually in the database and that fixed everything, but not sure if I did something wrong to get in that state in the first place 🤷

[ephraimbuddy]

FWIW can confirm this fixed my issue in #17638, so big thanks! Note I did have an issue after upgrading where the max_active_runs columns for some DAGs remained NULL even after scheduler parsed them. This seemed to prevent their tasks from getting scheduled. I eventually set it manually in the database and that fixed everything, but not sure if I did something wrong to get in that state in the first place 🤷

Interesting.
Did you run airflow db upgrade after upgrade?
cc: @kaxil