Projects plugin doesn't seem to like to site_url

[janvanveldhuizen]

Context

No response

Bug description

As soon as I add a site_url to the mkdocs.yml file of a project, the website gets stuck with:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/mkdocs/livereload/__init__.py", line 193, in _build_loop
    func()
  File "/usr/local/lib/python3.11/site-packages/mkdocs/commands/serve.py", line 67, in builder
    build(config, live
_server=None if is_clean else server, dirty=is_dirty)
  File "/usr/local/lib/python3.11/site-packages/mkdocs/commands/build.py", line 354, in build
    config.plugins.on_post_build(config=config)
  File "/usr/local/lib/python3.11/site-packages/mkdocs/plugins.py", line 542, in on_post_build
    return self.run_event('post_build', config=config)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/mkdocs/plugins.py", line 509, in run_event
    result = method(**kwargs)
             ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/material/plugins/projects/plugin.py", line 265, in on_post_build
    path = self._path_from_slug(slug, config)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/material/plugins/projects/plugin.py", line 587, in _path_from_slug
    at = len(posixpath.commonpath([base.path, dest.path]))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen posixpath>", line 547, in commonpath
ValueError: Can't mix absolute and relative paths
ERROR   -  [21:40:05] An error happened during the rebuild. The server will appear stuck until build errors are resolved.

Related links

site_url

Reproduction

upload.zip

Steps to reproduce

1. run the attached website
2. uncomment the site_url line in /languages/en/mkdocs.yml

Browser

No response

Before submitting

• I have read and followed the bug reporting guidelines.
• I have attached links to the documentation, and possibly related issues and discussions.
• I assure that I have removed all customizations before submitting this bug report.
• I have attached a .zip file with a minimal reproduction.

[squidfunk]

Thanks for reporting. Did you use the latest master of Insiders? I pushed many bugfixes yesterday and today, so please make sure to install master.

[janvanveldhuizen]

Fair point. The image I'm using is about 2 days old. Medieval 😂
I checked the same situation. It still crashes but with another message:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/mkdocs/livereload/__init__.py", line 193, in _build_loop
    func()
  File "/usr/local/lib/python3.11/site-packages/mkdocs/commands/serve.py", line 67, in builder
    build(config, live_server=None if is_clean else server, dirty=is_dirty)
  File "/usr/local/lib/python3.11/site-packages/mkdocs/commands/build.py", line 354, in build
    config.plugins.on_post_build(config=config)
  File "/usr/local/lib/python3.11/site-packages/mkdocs/plugins.py", line 542, in on_post_build
    return self.run_event('post_build', config=config)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/mkdocs/plugins.py", line 509, in run_event
    result = method(**kwargs)
             ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/material/plugins/projects/plugin.py", line 281, in on_post_build
    os.symlink(project.site_dir, path)
FileExistsError: [Errno 17] File exists: '/docs/languages/nl/../../site/nl' -> '/'
ERROR   -  [07:13:56] An error happened during the rebuild. The server will appear stuck until build errors are resolved.

I also checked if it is related to the site_dir setting, or a local url, but that seems not to be the case.

[squidfunk]

Thanks! So, I'm running your example with the latest master of Insiders:

(venv) squidfunk@squidfunk Downloads/upload $ mkdocs serve
INFO    -  Building documentation...
INFO    -  Cleaning site directory
INFO    -  Doc file 'index.md' contains an unrecognized relative link 'en/', it was left as is.
INFO    -  Doc file 'index.md' contains an unrecognized relative link 'nl/', it was left as is.
INFO    -  Doc file 'index.md' contains an unrecognized relative link 'eo/', it was left as is.
INFO    -  [project://.en] Cleaning site directory
INFO    -  [project://.en] Building documentation to directory: /Users/squidfunk/Downloads/upload/languages/en/site
INFO    -  [project://.en] Documentation built in 0.11 seconds
INFO    -  Documentation built in 0.25 seconds
INFO    -  [10:10:40] Watching paths for changes: 'docs', 'mkdocs.yml', ...
INFO    -  [10:10:40] Serving on http://127.0.0.1:8000/
^CINFO    -  Shutting down...
(venv) squidfunk@squidfunk Downloads/upload $ mkdocs build                                        
INFO    -  Cleaning site directory
INFO    -  Building documentation to directory: /Users/squidfunk/Downloads/upload/site
INFO    -  Doc file 'index.md' contains an unrecognized relative link 'en/', it was left as is.
INFO    -  Doc file 'index.md' contains an unrecognized relative link 'nl/', it was left as is.
INFO    -  Doc file 'index.md' contains an unrecognized relative link 'eo/', it was left as is.
INFO    -  [project://.en] Cleaning site directory
INFO    -  [project://.en] Building documentation to directory: /Users/squidfunk/Downloads/upload/site/en
INFO    -  [project://.en] Documentation built in 0.11 seconds
INFO    -  Documentation built in 0.25 seconds
(venv) squidfunk@squidfunk Downloads/upload $ 

I'm on macOS and not sure where your errors are coming from.

[squidfunk]

One more thing: you don't need to specify site_dir when using the projects plugin. it looks like you're following advise in #2346. The projects plugin aims to do all this work for you, so when you build your top-level project, all sites are automatically put into the right place.

The idea is: you can build your projects individually, but also together. For that to work, the projects plugin jumps through some hoops to make that work reliably when serving and building your site. if you change site_dir in a sub project, be sure to keep it within that project.

[squidfunk]

Okay wait, I missed the thing about commenting site_url in 🤦‍♂️ I can reproduce the problem now. Looking into it! 😅

[squidfunk]

Okay, so I know where this is coming from, but I'm not sure what the ideal experience here would be. So, you're using the projects plugin to build a multilingual blog. You set the site_url in en to the top-level domain:

site_url: https://papasmurf.nl

If you invoke mkdocs build, everything works as expected, and the plugin will put the en project under site/en. This is because you did not specify a top-level site_url, but only one for the project, and thus the project plugin assumes that you are building a project of several seemingly unrelated subprojects (think git submodules).

Now, when you use mkdocs serve, the plugin will set the site_url for all projects and try to deduce the subpath from the site_url. This logic is in place so you can detach the final URL of your project from the folder name you use. However, since the path of the site_url of the nested project is the top-level, the plugin runs into a problem where it can not determine the correct path. It's unclear to me what would be the ideal path here, and it seems that you might be mixing things configuration wise, but we should at least terminate with an error or point the user to what needs to be changed.

Let me make a few examples.

Multi-lingual blog, separate languages

All languages under prefixes, i.e., no canonical top-level language. The comment after mkdocs.yml denotes the site_urlsetting. Note that the values below are what the project plugin will deduce automatically if you do not set site_url, or only set site_url at the top-level. Setting site_url is necessary for social cards to work.

.
├─ docs/
├─ languages/
│  ├─ en/
│  │  ├─ docs/
│  │  └─ mkdocs.yml # http://example.com/en
│  └─ nl/
│     ├─ docs/
│     └─ mkdocs.yml # http://example.com/nl
└─ mkdocs.yml # http://example.com

You could change http://example.com/en to http://example.com/whatever/path/you/like without renaming the folder, and the projects plugin should put it into the correct location. You would also put an index.md in the top-level project with the following contents:

---
template: redirect.html
location: en/
---

This is everything that is contained in your top-level project.

Multi-lingual blog, alternate languages

In this setup, you would put your default language blog in the top-level project, and just add alternate languages in the languages folder as subprojects.

.
├─ docs/
├─ languages/
│  └─ nl/
│     ├─ docs/
│     └─ mkdocs.yml # http://example.com/nl
└─ mkdocs.yml # http://example.com

---

The setup you're using is kind of in between, where a subproject receives the site_url of a top-level project. My gut tells me that if a nested project contains no path different from the top-level project, we should fall back to the folder name to set it. So in your example, this would become:

.
├─ docs/
├─ languages/
│  └─ en/
│     ├─ docs/
│     └─ mkdocs.yml # http://example.com/en <- overridden
└─ mkdocs.yml # -

This is only relevant while serving the site. Does this make sense? Why did you specify the site_url in the nested project in the first place? Shouldn't that go into the top-level?

[squidfunk]

I think I found a good compromise. If the site URL is set, but clashes with the path, we just fall back to the path computed from the slug. This is fixed in 8e80334. Your example now serves without errors.

[janvanveldhuizen]

Thanks for fixing!
For me the default language is English, and I want to be able to write technical blog posts in English only, because this is the lingua franca in the technical world. So it would make sense to follow your suggestion of putting that in the top level project. I'll play around with that a bit.
However, from an end-user perspective. I'd like the website to follow the user's choice. When Dutch visitors of my website switch to Dutch, then the website should open in Dutch again when they come back.
On the current Hugo version of the website, I found a way to use localstorage. (https://papasmurf.nl/en/blog/hugo_language/).
But this relates more to the language switcher developments 😛

[squidfunk]

When Dutch visitors of my website switch to Dutch, then the website should open in Dutch again when they come back.

This is a separate feature request. I understand that you already asked for this in #4835 (comment), but as noted in #4835 (comment), please create a separate feature request for that. It is only loosely related to this issue or #4835.

[squidfunk]

Released as part of 9.5.1+insiders-4.47.0