URL resolution abiguities in Lektor

[dairiki]

The current state of URL resolution

URL resolution in Lektor generally runs through SourceObject.url_to. That method has a somewhat confusing, sometimes surprising behavior in that when resolving paths, sometimes the path argument is treated as a Lektor db path, while at other times it is treated as a URL path.

Though sometimes the db path to a record is the same as the URL path, in general that is not the case. The difficulty in specifying which interpretation url_to should take, and determining which interpretation was actually made can be quite confusing.

URL resolution in templates (and from python code)

In a template, if one does:

<img src="{{ 'image.jpg' | url_to }}">

what happens is:

• An attempt is made to resolve the path ("image.jpg") via the Lektor db.
• If the resolution is successful, the URL for the resolved db record is returned. Note that while the URL for the record is sometimes identical to the db path to the record, that is not always the case. (E.g. if a custom slug as been set for the record.)
• If the db lookup fails, the path is interpreted as a URL path (relative to the base URL implied by the currently-being-rendered page.)

The resolution through the db is useful in several cases:

• For attachments when alternatives are in use. In this case the attachments are only emitted for the primary alternative, so on non-primary alternative pages, the relative URL to the attachment is not the same as the relative db path.
• If there is a custom _slug set for the record, (or a custom children.slug_format set in the parent's data model), again the db path is not the same as the URL path.

In these cases, url_to generally does the right thing.

To disable the attempt at resolving the path via the Lektor db, one may prefix the path with a "!". That forces the path to be treated as a URL path.

URL resolution for Markdown text

For markdown links and images, current code always prefixes the path with "!", thus disabling db path resolution. This forces authors to always know and use valid URL paths rather than db paths when authoring markdown text.

This causes problems with links to attachments when alternatives are in use. (See, e.g. #950, #313, #470, and #88.) Similar issues arise when custom slugs are in use.

---

Issues

The obvious issue, as evidenced by all the bug reports having to do with URL generation for attachments when alternatives are in use, is that it would be nice to have some way to resolve links in markdown as Lektor db paths.

Another issue is the way url_to even when attempting db resolution, silently falls back to treating the path as a URL path if db resolution fails. That's just a set-up for confusion.

---

Possible Solutions

Spitballing...

1. Allow db resolution in markdown.

The simplest solution is just to do away with the "!" prefixing in our markdown renderer.

This may break a few existing sites, but I think any such breakage will be minimal. For those paths which now will be successfully resolved through the Lektor db, generally, either the URL will be the same as the db path (so no change), or db resolution was probably what the writer originally intended anyway.

This does not solve the ambiguity of url_to silently treating the path as a URL path with db path resolution fails.

2. Add a new argument to url_to to control resolution

We could add a new argument to the url_to functions to control path resolution. E.g.

def url_to(path, alt=none, absolute=None, external=None, resolve=None):
    ...

With resolve=None (the default), the behavior would be identical to what it is now: attempt db resolution, fall back to interpreting path as a URL path.

Setting resolve=True would force db path resolution. Url_to would return None (or maybe Undefined) if path resolution fails.

Setting resolve=False would be equivalent to prefixing the path with "!" (but is clearer and less goofy.)

This fixes the resolution ambiguity, allowing the author a way to force db path resolution if that's what they intended.

As there is no way to directly set the optional parameters to url_to from markdown links, this change, by itself wouldn't fix the problem that there is no way to resolve markdown links via the Lektor db.

3. Add a new custom URL scheme to trigger db resolution

We could make it so that passing something like path="lektor:image.jpg" to url_to would force interpretation of the path as a db path. (Equivalent to resolve=True from the previous idea.)

---

My current preference

At the moment, my preference would be to implement solutions #1 and #3 listed above.

So: remove the "!" prefixing from our markdown ImprovedRenderer. This has a slight potential to break existing sites, but the fix is easy: just add the ! explicitly (like you would do in a template) if that's really what you wanted.

Bare paths (with no scheme) would be treated in the current DWIM fashion: try to resolve via db, but fall back to treating the path as a URL path if that resolution fails.

As it is now, paths could be prefixed with ! to prevent the db resolution.

Additionally, they could be prefixed with lektor: to force db resolution (returning an error if resolution fails.)

---

Questions for y'all

What do you think?

Is it going to give anyone gas if we remove the "!" prefixing from the markdown URL resolution?