Experimental: Support custom weight_rule implementation to calculate the TI priority_weight

[hussein-awala]

This PR adds an experimental feature that allows users to implement custom weight rules (weight strategies) to calculate the priority weight of the task instance in runtime.

This is a lighter alternative for #36029 but without serializing the class instance parameters, which removes the need to add a new column to the task table and handles many cases to keep the new functionality b/c .

[hussein-awala]

Some of the dags/plugin examples tests will fail, I will fix them tomorrow before merging.

[eladkal]

LGTM!
Very cool

[jscheffl]

Tested and like it!

Besides the comment two things that I'd really like before merge but acknowledge this might not be blocking:

• The example is removed - because obviously w/o default plugin deployed will not work. But w/o reading the code it might be hard to make it working in a frst attempt for a user. So if example is missing, then might be good to add some example DAG code in RST docs or commented example in the plugin code.
• Small UI Glitch: As the priority weight is calculated dynamically, it might change display on UI after execution. In the decreasing example the displayed weight on UI changes after task was successfully executed. Because the try_number of the data at the task is a moving target. So we need to remember that the UI is a display of the current state and might have been different at time of execution depending on rule logic. I don't see it as problem and of course it is caused by the fast that we do not persist all values (not only priority weight but also other task execution evidence data).

Thanks again for the rework and besides small comment LGTM!

[hussein-awala]

then might be good to add some example DAG code in RST docs or commented example in the plugin code.

I agree that it is not obvious for users to use it, I will add an example to the documentation.

[hussein-awala]

So we need to remember that the UI is a display of the current state and might have been different at the time of execution depending on rule logic.

I will check if there is a way to improve this, but since it displays the priority weight used to schedule the current attempt, IMHO it should be fine. However, it would be better to add something that tells the user what we will have in the next attempt or update it when the task passes from running to any other state. We can discuss and implement this before 2.10.0