Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add feature to specify task parameters in doit.cfg (issue #283) #322

Open
wants to merge 11 commits into
base: master
from

Conversation

@rbdixon
Copy link
Contributor

commented Jul 28, 2019

doit.cfg options take precedence over task definition default values
but with be replaced by task parameters passed on the command line.

TODO:

  • Implement [task:arg_from_cfg]
  • Rename cfg_defaults to cfg_values
  • Remove task config checks for cfg_values parameter
  • Use .overwrite_defaults() to apply cfg_values
  • Refactor control.py per comment
  • Tests
    • Ensure that doit.cfg specified task parameters are applied.
  • Documentation

Implements #283

Demo:

$ cat dodo.py
def task_arg_from_cfg():
    def runit(x=1, y=1):
        print(f'python x={x}, y={y}')
        # if x==1: raise ValueError()

    return {
        'actions': [runit, 'echo shell x=%(x)s y=%(y)s'],
        'params': [{'default': 1, 'name': 'x', 'long': 'x'}, {'default': 2, 'name': 'y', 'long': 'y'}],
        'verbosity': 2,
}
$ cat doit.cfg
[task:arg_from_cfg]
    x = 1000
$ doit
.  arg_from_cfg
python x=1000, y=2
shell x=1000 y=2

$ doit run arg_from_cfg
.  arg_from_cfg
python x=1000, y=2
shell x=1000 y=2

$ doit run arg_from_cfg --x=123
.  arg_from_cfg
python x=123, y=2
shell x=123 y=2

@rbdixon rbdixon changed the title Add feature to specify task parameters in doit.cfg (issue #238) Add feature to specify task parameters in doit.cfg (issue #283) Jul 28, 2019

@coveralls

This comment has been minimized.

Copy link

commented Jul 28, 2019

Coverage Status

Coverage decreased (-0.03%) to 99.703% when pulling 6d7b6cc on rbdixon:issue_238 into 660d3a2 on pydoit:master.

@rbdixon

This comment has been minimized.

Copy link
Contributor Author

commented Jul 28, 2019

I'm working on some test cases but I wanted to get this preliminary implementation out for review. Let me know what you think, please.

I've not tried watch or auto... probably doesn't work for those commands.

@smarie
Copy link

left a comment

Great PR that would bring a nice feature.
Apart from tests missing (mandatory :) ) and a name choice that is maybe not the most intuitive (see comment), otherwise looks good to me.

Show resolved Hide resolved doit/cmdparse.py Outdated
@schettino72
Copy link
Member

left a comment

What do you think about requiring a task: prefix in the config section name. i.e.

instead of

[arg_from_cfg]

write

[task:arg_from_cfg]

This way we also avoid potential conflicts with commands.

Show resolved Hide resolved doit/cmdparse.py Outdated
Show resolved Hide resolved doit/control.py Outdated
Show resolved Hide resolved doit/task.py Outdated
Show resolved Hide resolved doit/task.py
Show resolved Hide resolved doit/task.py Outdated
@@ -578,6 +578,10 @@ def execute(self, params, args):
if not legacy_loader:
self.task_list = self.loader.load_tasks(cmd=self, pos_args=args)

# Add task options from config, if present
for task in self.task_list:
task.cfg_defaults = self.config.get(task.name, {})

This comment has been minimized.

Copy link
@schettino72

schettino72 Jul 31, 2019

Member

It bothers me a bit that we might be calling config.get for every task when rarely it will contain a value...

This comment has been minimized.

Copy link
@rbdixon

rbdixon Aug 3, 2019

Author Contributor

It is roughly 30% faster to do an if task.name in self.config: .... What the actual execution time benefit would be depends on how many tasks are defined.

Benchmark:

https://gist.github.com/rbdixon/797d0478f9716c4123e401a1187552e1

Number of tasks: 1326
Speedup -30.40%
Baseline duration: 0.25ms
Alternative duration: 0.18ms

Might be hard for a user to tell the difference in execution time either way.

@rbdixon

This comment has been minimized.

Copy link
Contributor Author

commented Aug 1, 2019

This is very helpful and I'll rework the patch. I really appreciate the thoughtful review.

Show resolved Hide resolved doit/task.py Outdated

rbdixon added some commits Jul 28, 2019

Refactor how task command line arguments are parsed.
All command line parsing should be in Task now.
Modify test_control.py to use a fixture to create sample tasks.
Task.init_options() will only initialize options once. The tests
were written so that each test reused the same task instances and
therefore the tests failed because the expected task option defaults
were not applied. The fix is to use a fixture to redefine the tasks
each time.

The only-once semantics of Task.init_options() dates back four years
and the commit indicates this was a performance enhancement.

@rbdixon rbdixon force-pushed the rbdixon:issue_238 branch from 6d7b6cc to f99dce1 Aug 13, 2019

@rbdixon

This comment has been minimized.

Copy link
Contributor Author

commented Aug 13, 2019

I've implemented tests, documentation, and all changes discussed in this thread. I'm using this branch in my daily work with our own internal tools and nikola. So far, so good.

I believe this is ready to be merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.