Smarter check #1759
Smarter check #1759
Conversation
|
@felixfontein I'd like your review of this one |
| def _call_nikola_list(site): | ||
| files = [] | ||
| for task in generate_tasks('render_site', site.gen_tasks('render_site', "Task", '')): | ||
| files.extend(task.targets) |
felixfontein
May 27, 2015
Contributor
This won't work if any of the generated tasks has a delayed task loader, but a) probably no plugin author does that, and b) I have to go through that anyway when adjusting this for the earlytask_impl branch, so I vote for leaving this as is for the moment.
This won't work if any of the generated tasks has a delayed task loader, but a) probably no plugin author does that, and b) I have to go through that anyway when adjusting this for the earlytask_impl branch, so I vote for leaving this as is for the moment.
ralsina
May 27, 2015
Author
Member
Ack.
Ack.
|
|
||
|
|
||
| def real_scan_files(l, site): | ||
| def real_scan_files(site): |
felixfontein
May 27, 2015
Contributor
This function is also called from orphans.py and github_deploy.py.
This function is also called from orphans.py and github_deploy.py.
ralsina
May 27, 2015
Author
Member
Done
Done
| fname = task.split(':', 1)[-1] | ||
| for fname in _call_nikola_list(site): | ||
| fname = fname.strip() | ||
| if output_folder in fname: |
felixfontein
May 27, 2015
Contributor
startwith() is better than in in this context, as maybe some file cache/bla/output/bla is generated, which also contains output.
startwith() is better than in in this context, as maybe some file cache/bla/output/bla is generated, which also contains output.
ralsina
May 27, 2015
Author
Member
Done
Done
| @@ -262,7 +252,7 @@ def analyze(self, task, find_sources=False, check_remote=False): | |||
| self.logger.warn("Broken link in {0}: {1}".format(filename, target)) | |||
| if find_sources: | |||
| self.logger.warn("Possible sources:") | |||
| self.logger.warn("\n".join(_call_nikola_list(self.l, self.site, ["--deps", task]))) | |||
| self.logger.warn("\n".join(_call_nikola_list(self.site))) | |||
felixfontein
May 27, 2015
Contributor
The output of nikola list --deps <taskname> is quite different than the one from nikola list --all, so this is not a valid substitution. Actually, I think we need some support from doit here, if we don't want to repeat some of its code or use internal APIs. @schettino72: what do you think (or does that already exist?) about having a simple way in doit to get essentially the output of doit list --all --deps (including a list of targets per task) in a nice way?
The output of nikola list --deps <taskname> is quite different than the one from nikola list --all, so this is not a valid substitution. Actually, I think we need some support from doit here, if we don't want to repeat some of its code or use internal APIs. @schettino72: what do you think (or does that already exist?) about having a simple way in doit to get essentially the output of doit list --all --deps (including a list of targets per task) in a nice way?
ralsina
May 27, 2015
Author
Member
Good catch. Same thing as before, I just need to check task.file_dep, no big deal. Will fix it tomorrow.
Good catch. Same thing as before, I just need to check task.file_dep, no big deal. Will fix it tomorrow.
ralsina
May 27, 2015
Author
Member
Done.
Done.
schettino72
May 27, 2015
Member
As I said before, the doit list supports a template param to format its output (it seems I didnt document this parameter - sorry). Currently the template can only output values for: name, status and doc. I guess the best option would be to extend supported values that can appear in the template. Or maybe support a json output?
There is also the doit info command that outputs the targets and dependencies but lacks a formatting and a --all options...
As I said before, the doit list supports a template param to format its output (it seems I didnt document this parameter - sorry). Currently the template can only output values for: name, status and doc. I guess the best option would be to extend supported values that can appear in the template. Or maybe support a json output?
There is also the doit info command that outputs the targets and dependencies but lacks a formatting and a --all options...
felixfontein
May 28, 2015
Contributor
I'd prefer JSON output; that's easy to parse and one can also easily add more info without breaking existing clients.
I'd prefer JSON output; that's easy to parse and one can also easily add more info without breaking existing clients.
| # Maybe we should just examine all HTML files | ||
| output_folder = self.site.config['OUTPUT_FOLDER'] | ||
| for fname in _call_nikola_list(self.site): | ||
| if output_folder in fname and '.html' in fname: |
felixfontein
May 27, 2015
Contributor
What about using endswith() here instead of (the second) in? Just in case someone likes to use .html in directory names :)
What about using endswith() here instead of (the second) in? Just in case someone likes to use .html in directory names :)
ralsina
May 27, 2015
Author
Member
well [-5:] but same idea :-)
well [-5:] but same idea :-)
| def _call_nikola_list(site): | ||
| files = [] | ||
| deps = defaultdict(list) | ||
| for task in generate_tasks('render_site', site.gen_tasks('render_site', "Task", '')): |
Kwpolska
May 28, 2015
Member
Is this safe? Are we not doing it twice?
We should probably talk to our task loader anyways.
Is this safe? Are we not doing it twice?
We should probably talk to our task loader anyways.
ralsina
May 28, 2015
Author
Member
It works anyway. To talk to our taskloader we'd have to move it out of __main__ first.
It works anyway. To talk to our taskloader we'd have to move it out of __main__ first.
Kwpolska
May 28, 2015
Member
You can just talk to self.site.doit.task_loader. You also can save the generated tasks there, if any.
You can just talk to self.site.doit.task_loader. You also can save the generated tasks there, if any.
ralsina
May 28, 2015
Author
Member
I tried it, see no difference, and I don't understand internals well enough, maybe.
So, proposing to merge this as-is.
I tried it, see no difference, and I don't understand internals well enough, maybe.
So, proposing to merge this as-is.
|
Before merging: what is that problem with the |
| for target in task.targets: | ||
| deps[target].extend(task.file_dep) | ||
| for task in generate_tasks('post_render', site.gen_tasks('render_site', "LateTask", '')): | ||
| files.extend(task.targets) |
felixfontein
May 28, 2015
Contributor
Don't you need the lines updating deps[target] here as well?
Don't you need the lines updating deps[target] here as well?
ralsina
May 28, 2015
Author
Member
You are correct. Good catch! Fixing...
You are correct. Good catch! Fixing...
|
The output of the demo site changes because I fixed a broken link generated by the listings plugin, I think. |
|
Ah ok. Then everything's fine from my POV :) |
Fix #1758 and hopefully breaks nothing ;-)