Add Ability to overwrite task settings when calling a task in Functional API

[cicdw]

Currently, a common pattern is:

t = MyTask(name="test task", tags=["A"])

with Flow() as f:
   res1 = t(1)
   res2 = t(2)
   ...

and each individual copied instance of the task will appear with the same name "test task" in the UI and any other place that information is surfaced.

To allow for more nuanced control of the copying of tasks, we should add a reserved kwarg _task_args to task call signatures which allow for overriding things (such as name) when calling / copying the task.

[cicdw]

Just throwing this out there, but another option would be some sugar like:

t = MyTask(name="test task", tags=["A"])

with Flow() as f:
   res1 = t["one"](1)
   res2 = t["two"](2)

which could be used to rename the individual tasks "test task[one]" and "test task[two]".

This would mean the current __getitem__ implementation would be gone, but I personally don't see people using tasks which are so small that they are going to be using / expecting that behavior.

The reason I'm suggesting this different idea is that I think the documentation for the special kwargs to task.__call__ are somewhat hidden (they don't naturally appear in a docstring for example), so I'm hesitant to pile on more things there.

[jlowin]

This __getitem__ sugar feels very unnatural to me; it doesn't have any Python analogue. (The existing getitem sugar is tenuous, but at least it mimics Python behavior of indexing an object.)

Also, this would only solve the issue for names, not for other attributes like triggers.

If the issue is documentation, then we need to add more.

[cicdw]

Also, this would only solve the issue for names, not for other attributes like triggers.

Right, but I suspect names will be the only thing people really use this for, which is why I was trying to brainstorm a leaner version than

new_task = old_task(a, b, c, y, _task_args={"name": "new_task"})

which feels clunky to me, and is attempting to solve a more general problem than the one we know about (names in the UI). For things like triggers, retries, etc. I think people will simply create new task instances because at that point the underlying task is probably fundamentally different anyway.

But, if we think a reserved kwarg is the way to go, then I suppose adding one layer of depth to the argument for the sake of supporting a more general pattern isn't so bad.

[jlowin]

I think this feature is going to add a lot of utility to FunctionTasks (and the @task decorator more broadly) because it will allow those tasks to be reused without having to rewrite them. Currently, to use the same @task-decorated function as a task with two different retry behaviors, you have to rewrite the decorated function just to change the retry value. With this kwarg, you could affect the same behavior much more cleanly. I'd vote that this is a very explicit form of controlling behavior and should have a very explicit implementation.

[cicdw]

Good point 👍🏻👍🏻