Projects plugin: distinguish between sub-projects in logging #6414
Comments
|
One additional observation: for some reason, the logs of the subproject builds are shown on Linux but not on MacOS. That seems odd to me. Both are running the same 9.4.14+insiders.4.46.0 and MkDocs 1.5.3 |
|
Thanks for suggesting! Fixed in 
The nice thing is that plugins like the blog plugin that show the file name and the folder in which it happened should now all show the relative path from where the project is built, i.e., Log messages are still interleaved, but I'm not sure what the best way is. Since the build is done in parallel, the only way to batch messages in consecutive chunks would be to wait for each project to complete and then flush log messages to stdout. I'm not sure that's a good idea, but if somebody comes up with a better idea, I'm happy to consider.
Indeed that is very weird. I also haven't observed them on macOS. Are they at least prefixed now? Might also be a difference or implementation issue in |
I don't see how this could be done without a lot of effort and possible drawbacks. The tagging should be doing enough to address the problem of interpreting the log entries. Thanks for fixing this so quickly. I saw is was a fair bit of work. On MacOS, I still don't see the log entries from the subprojects. There might be some documented OS-dependent behavior in concurrent.future? Is multiprocessing the underlying implementation? If so, might be worth looking there as well. |
|
The screenshot above was taken on macOS – don't you see any log entries? |
|
That is odd, let me run past you what I am doing: |
|
So if you put an unresolvable link in a markdown page in a project nothing happens? Sorry, I'm unable to reproduce this from the information I have. As you can see in the screenshot, works on my machine. We need to find out what is stopping this. We're still experiencing some problems with the cache (reopened the issue), so you might use: plugins:
- projects:
cache: false |
|
Right, it seems there is something deeper and more sinister going on: The subprojects do not get built. That would explain the lack of log messages. I did not notice in my previous test because I assumed the projects were being built and just looked for the log messages. Let me try this with a brand-new venv... Same result but I do note that MkDocs is complaining about the missing RSS plugin, so it is hitting the subprojects - but then not doing anything? Caching is turned off in the examples repo btw. Hope this helps. Let me know if you want me to try and debug this, esp. if you cannot reproduce this with the examples. Otherwise I will crack on with the stuff on PRs. |
|
Okay, so first of all, let's try to fix the logging problems. I've invested two hours into getting this right. Who would've though that this would be so challenging? I think I got all cases now, and introduced two new settings to the plugin:
The defaults: plugins:
- projects:
log: true
log_level: warnMkDocs itself has two command line flags which influence logging:
Now, if I take the multilingual blog demo project that I posted earlier, I get the following outputs:
|
|
I'm going to work the rest of the day on two other problems: hoisting and caching. There are configurations where hoisting does not work as expected. I'm quite sure I know where this is coming from. I also have an idea how to mitigate the caching problems, but I'm not sure I'm able to fix this today. |
|
FYI, when I run Edit: Updated after fixing the duplicate logging bug. |
|
Okay, that is pretty frickin' cool. With the latest commit, you can change the |
|
One other odd thing: projects seem to be built multiple times. This was just uncovered by logging. I'll look into that as well.
|
|
Okay, projects have not been built multiple times, but the loggers must be unregistered after completion, which I fixed in Now Before |
|
Question: you said at some point that it seemed projects were built more than once. Could that explain why I expected to see The bit about This is looking very nice now, indeed. Let me poke it a bit. I will hop back to my Mac and see if the very latest has changed anything there. You still want verification of the behavior in the different logging cases, I assume. Edit: switching back to Linux because I still get no log output on my Mac. Mysterious but I agree we should focus on the new log functionality as this problem I see does not seem to be reproducible on other machines. Perhaps I should just reboot and see if it is one of those very odd quirks. Dunno, like I switched timezones since last reboot and this confuses the filesystem watcher or something. |
That was debunked – it was a logging issue.
Where When building the site, the projects are always built fresh in their resolved subpath in the top-level directory, so you end up with one nice deployable artifact without symlinks. So. Serving: symlinks. Building: hoisting to the top-level. Also note that symlinks are necessary, because you might have projects inside projects, and projects are currently not built in topological order, but concurrently all at the same time. In the future, we will also introduce the possibility to build projects in topological order. This might be particularly helpful for when you have intermediate index pages in level 2 projects that are generated from level 3 projects. I invite you to study the source of the projects plugin. It is extensively documented with the things that need to be considered when building multiple MkDocs projects together.
Jup. A as a rule for all concurrent plugins (social, privacy, optimize, projects) – setting
Hmm. Even not with |
I have switched shells and also created new venvs and cloned the examples repo again. Let me figure this one out - unless someone reports they have a similar issue. I can figure up the debugger but want to first check that the logging options work ok, which I can do on the Linux box across the room. |
|
I am beginning to suspect you put a feature flag in. I now get no log output on Linux either and I am testing with both the examples repo and the multilingual blog zip you posted over in #2346 (reply in thread). $ unzip ~/Downloads/multi-language-blog.zip
$ mkdocs build
INFO - Cleaning site directory
INFO - Building documentation to directory: /home/avoss/src/mkdocs-material/reproduce/logs/multiling/site
INFO - Documentation built in 0.36 seconds
$ ls site/
404.html assets de en index.html sitemap.xml sitemap.xml.gzDon't waste more of your time on this. I will eventually figure out what is going on and will send back some input that is actually usable. Just let me know if the problem is that my expectations are wrong about what the default behavior should be. Edit: ok, the assumption that was wrong was that the multiling zip file would already contain the broken links. Now we are getting somewhere... |
|
I messed with the example to get some test output. Sorry, my fault. Here's the one I'm currently using (which is a mess): |
|
Here is one observation: the log entries produced by the individual examples are at the INFO level while the broken links are WARNINGs. I get the WARNINGs from the zip file you just sent fine on both Linux and MacOS. Let me see again about the examples to see what was going on with those. |
|
Right, so I expected to see this in the output when building with projects. If I change the link target to |
|
That sounds perfect! You will only get the plugins:
- projects:
log_level: infoIf that switches them on and off on both systems, we should have it. |
|
There is then the question why the thing I was looking for is not a WARNING but that would go to MkDocs, I assume? |
Erm, could you paste the output? I don't get a double warning. Double warnings for log output should not occur. |
|
If I set No double messages. |
|
I just ran another upgrade of the Insiders version, just to make sure: The input is simply this index page for the English version with a broken page reference: then: |
|
The latest commit switches the default log level to |
|
So, it's confirmed – somehow the logger prints twice on Linux. I'll investigate. |
|
Fixed in |
|
Released as part of 9.5.1+insiders-4.47.0 |
Context
When building a project with subprojects through the Projects plugin, the log entries generated by the concurrent build processes for the subprojects are interleaved and it can be difficult to tell which subproject generated which message.
Description
The log entries for the subprojects should be tagged with the name of the subproject so the user can tell which subproject generated which message. They will still be interleaved, as shown below, but it will be possible to filter them with an appropriate log filtering tool.
Note that the README file for
blog-rsscontains none of the links that MkDocs complains about here. Also, the summary of how much time each project took to build is ambiguous.This will require some modification of the subproject settings, I assume, to instruct MkDocs to not just log to the console or to include the tag. I included a link below to some info on how to log in parallel processes. This should require a change both in the projects plugin and in MkDocs, I guess.
Related links
Use Cases
There are a couple of use cases that come to my mind:
Visuals
No response
Before submitting
The text was updated successfully, but these errors were encountered: