Add feature to specify task parameters in doit.cfg (issue #283)

[rbdixon]

doit.cfg options take precedence over task definition default values
but with be replaced by task parameters passed on the command line.

TODO:

• Implement [task:arg_from_cfg]
• Rename cfg_defaults to cfg_values
• Remove task config checks for cfg_values parameter
• Use .overwrite_defaults() to apply cfg_values
• Refactor control.py per comment
• Tests
  • Ensure that doit.cfg specified task parameters are applied.
• Documentation

Implements #283

Demo:

$ cat dodo.py
def task_arg_from_cfg():
    def runit(x=1, y=1):
        print(f'python x={x}, y={y}')
        # if x==1: raise ValueError()

    return {
        'actions': [runit, 'echo shell x=%(x)s y=%(y)s'],
        'params': [{'default': 1, 'name': 'x', 'long': 'x'}, {'default': 2, 'name': 'y', 'long': 'y'}],
        'verbosity': 2,
}

$ cat doit.cfg
[task:arg_from_cfg]
    x = 1000

$ doit
.  arg_from_cfg
python x=1000, y=2
shell x=1000 y=2

$ doit run arg_from_cfg
.  arg_from_cfg
python x=1000, y=2
shell x=1000 y=2

$ doit run arg_from_cfg --x=123
.  arg_from_cfg
python x=123, y=2
shell x=123 y=2

[coveralls]

Coverage decreased (-0.03%) to 99.703% when pulling 6d7b6cc on rbdixon:issue_238 into 660d3a2 on pydoit:master.

[rbdixon]

I'm working on some test cases but I wanted to get this preliminary implementation out for review. Let me know what you think, please.

I've not tried watch or auto... probably doesn't work for those commands.

[smarie]

Great PR that would bring a nice feature.
Apart from tests missing (mandatory :) ) and a name choice that is maybe not the most intuitive (see comment), otherwise looks good to me.

[schettino72]

What do you think about requiring a task: prefix in the config section name. i.e.

instead of

[arg_from_cfg]

write

[task:arg_from_cfg]

This way we also avoid potential conflicts with commands.

[schettino72]

It bothers me a bit that we might be calling config.get for every task when rarely it will contain a value...

[rbdixon]

It is roughly 30% faster to do an if task.name in self.config: .... What the actual execution time benefit would be depends on how many tasks are defined.

Benchmark:

https://gist.github.com/rbdixon/797d0478f9716c4123e401a1187552e1

Number of tasks: 1326
Speedup -30.40%
Baseline duration: 0.25ms
Alternative duration: 0.18ms

Might be hard for a user to tell the difference in execution time either way.

[rbdixon]

This is very helpful and I'll rework the patch. I really appreciate the thoughtful review.

[rbdixon]

I've implemented tests, documentation, and all changes discussed in this thread. I'm using this branch in my daily work with our own internal tools and nikola. So far, so good.

I believe this is ready to be merged.

[arichiardi]

This is a great feature, just ran into this and want to say thank you and bump it 😃

[schettino72]

merged: f0303f2

thanks. and sorry for delay.