Proposal: populate_parameters kwarg for Task.__call__

[benfogelson]

Intro

A pattern we tend to encounter is where we have a task that we call to add to a flow, and we’d like to make Parameters for some of its call method arguments. This can lead to a lot of boilerplate:

from prefect import Flow, Parameter, task

@task
def task_with_many_args(a, b, c, d, e=None, f=5, g='foo'):
    pass
with Flow('flow') as flow:
    a = some_other_task()
    task_with_many_args(
        a=a,
        b=Parameter('task_with_many_args.b'),
        c=Parameter('task_with_many_args.c'),
        d=Parameter('task_with_many_args.d'),
        e=Parameter('task_with_many_args.e', default=None),
        f=Parameter('task_with_many_args.f', default=5),
        g=Parameter('task_with_many_args.g', default='foo'),
    )

This comes up often enough that we’ve written a helper function to populate these parameters by inspecting the task’s call signature:

with Flow('flow') as flow:
    a = some_other_task()
    call_and_populate_parameters(task_with_many_args, a=a)

These two snippets produce identical flows.

Proposal

Would there be appetite for supporting this pattern directly via a kwarg added to Task.__call__? The API I'm imagining would involve changing the signature of Task.__call__ to the following:

    def __call__(
        self,
        *args: Any,
        mapped: bool = False,
        task_args: dict = None,
        upstream_tasks: Iterable[Any] = None,
        flow: "Flow" = None,
        populate_parameters: bool = False,  # <--the new kwarg
        **kwargs: Any
    ) -> "Task":

When populate_parameters is True, __call__ would instantiate Parameters for any arguments present in the task.run signature that weren't supplied by either args and kwargs. If the signature specifies argument defaults, the instantiated Parameters will also have those defaults, otherwise they will be required. Then, the task is copied and bound to the flow as usual, with a binding that includes edges for the generated parameters. I'm imagining also that the instantiated Parameters would follow a naming convention of something like task_name.parameter_name.

I'm happy to take a crack at adapting our helper function to do this (I don't think it will be that hard), but wanted to gauge whether people would be receptive to this idea before putting in the work.