[WIP] Linking and embedding

[jevakallio]

This RFC is Work in Progress.

Primary author: @jojanaho

Prior art:

• Make wikilinks more flexible (v2) foam#103
• Proposals: deep links and block embeds foam#105
• Proposal: Link Definition and Disambiguation foam#122

[jevakallio]

I've done a first pass review on the linking-and-embedding-specification.md, and it's looking really strong! I'll let this stew overnight and do another pass soon.

@jojanaho I'm quite excited about my idea (typical me! excited about my own ideas!) about using #tags as an anchor syntax. Would like to hear counter-feedback on that!

[jevakallio]

Suggested change

	Implementing all the mechanics into Foam will has several drawbacks:
	Implementing all the mechanics into Foam will have several drawbacks:

[jevakallio]

Proposal: Use auto-closing anchor tags by default:

Suggested change

User can add arbitrary anchors into a note with HTML-syntax: `<a name="my-anchor"></a>`. Since these names (identifiers) are written by users, they might end up being non-unique within a note. In these cases, an error is shown to the user. See the reasoning of using the name-attribute instead of the id-attribute from https://stackoverflow.com/a/7335259. In the future, better syntax for the anchors might be developed.

User can add arbitrary anchors into a note with HTML-syntax: `<a name="my-anchor" />`. Since these names (identifiers) are written by users, they might end up being non-unique within a note. In these cases, an error is shown to the user. See the reasoning of using the name-attribute instead of the id-attribute from https://stackoverflow.com/a/7335259. In the future, better syntax for the anchors might be developed.

I know technically this is not valid HTML, but it does work:
https://gist.github.com/jevakallio/086e3b375af45a82cc46d0ddd2f4fb39

IMO using self-closing tags in documentation makes this a lot more palatable to use. Folks who prefer their HTML valid could have that option.

[jevakallio]

Conceptually, I do wonder, is there really a use case of explicitly defining EBlocks? It's a cool idea, but when do we need to refer to a fragment of text that's not already its own scope (paragraph, bullet, heading-ed section)?

I'm not certain. Let's make sure there's a case for it, since this requires advanced semantics.

[nixsee]

If someone writes in pure prose, they'll have multi-sentence paragraphs. Given what you guys are discussing about embeds and block references, I think its quite reasonable to expect the desire to link to just a sentence (or even word!). But if the difficulty of advanced semantics preclude this, that's perfectly understandable. Its also likely that I misunderstand what you guys are talking about, so...

[jevakallio]

Assuming we'll end up implementing explicit EBlocks, what about piggybacking on the fact that name attributes are not unique, and that html allows for boolean attributes that don't define a value:

<a begin name="my-block" />

Content here

<a end name="my-block />

It appears that the way browsers interpret this, upon clicking a link that references a duplicate named element, it always jumps to the first matched element. See: https://gist.github.com/jevakallio/eaa22f3038347cf2294eb9c75ef40ded#my-block

Again, I am aware that these are technically not valid HTML, but they could converted to be during build-time (or the end block could be dropped entirely, as it's only required during transclusion during edit and build time.

Given that the block end tag doesn't need to be valid (or even included, we could use more pseudo-HTML syntax:

<a name="my-block" />

Content

<a close="my-block"  />

The exact wording here is up in the air, but would be keen to hear thoughts on using invalid HTML syntax, conceptually.

[jevakallio]

Further, if we ended up implementing this as a syntax extension, such as MDX, what about the following:

<block id="my-block">
  **Arbitrary** _markdown_ ~~content~~ allowed!
</block>

The MDX element could work as the link target boundaries during edit/build time, leave an a-tag, and pass through the content to the markdown processor so that it gets treated as if it was never embedded inside a HTML block at all:

<a name="my-block">
<b>Arbitrary</b> <em>markdown</em> <strikethrough>content</strikethrough> allowed!

With this syntax, we would also not have to worry about clashing duplicate names or other arbitrary achor tags.

The drawback is that it does require a preprocessing step.

[jevakallio]

What do you think about the ability to start an EBlock at the end of a text, as well as the start e.g.

- consequat non dolore <a name="x" />
  - consequat fugiat

This would improve the readability of the list, since as a reader I can scan through the left side of the list and mentally stop reading when I hit the block marker.

Semantically, this would mean that a block cannot be started mid-paragraph, and would always include the entire parent AST node. This might be confusing. Unsure. Leaving comment here anyway.

[mathieudutour]

From a discussion on Discord:

I'm not sure I agree with the example of implicit blocks for x. IMO x should refer only to consequat non dolore.

If you want it to refer to the entire bullet point, then I think it should be:

- Nostrud incididunt
<a name="x"></a>
- consequat non dolore
  - consequat fugiat
  - eiusmod esse ipsum
    - anim nulla aute
- ad reprehenderit

Otherwise the sibling of x is the content of the bullet point, not the bullet point itself.

I’m thinking in terms of AST: the anchor refers to its next sibling (that’s how I understood the spec).

[jevakallio]

Not sure if this is mentioned elsewhere, but I'd also like to consider adding a primary "title: Title" attribute-value pair that can override the markdown heading of the page. Sometimes for presentational reasons it's inconvenient to name a document in a way that makes for a good title elsewhere, or to have a H1 tag at all.

[jevakallio]

What if tags were also anchors? This sounds a bit crazy, but hear me out.

• The hashtag syntax is already mirrored in markdown link syntax [](#tag)
• The hashtag syntax is reminiscent of headings (# Heading vs #tag), which are already anchors by their own right
• In the compilation step, all we would need to do is to compile each tag to an anchor: <a name="tag"></tag>, which makes intuitive sense
  • Alternatively, they could be rendered into the document. (<a name="tag">#tag</tag>), and we could use use it as a "backlink" that shows everywhere it's being referenced, e.g. by showing a list of referrers as a popup on hover! 🤯
• They're somewhat pleasant to look at:

- some content here
  - this block needs linking to #block
    - children are part of...
    - the block scope

As to uniqueness, we could consider the semantics to be same as with multiple identical name-anchors: It points to the first one. For embeds to tag-anchors, we could do a multi-embed where we embed all references to that tag within the document.

Again, this would require a build step to work in a published Foam, but maybe this is the "future better syntax" you referred to above?

[jevakallio]

I wonder if the additional syntax here is necessary, or whether the labeling could be done as the "Title" part of the LinkRefDef?

What is the purpose of labels in work-based wikilinks? Why not use the arbitrary word content as the label?

In path-based links I see the usefulness of it, as it avoids having to generate a LinkRefDef while still allowing for a custom display text.

I'm not opposed to this syntax, just seeking to understand the use case.

[jevakallio]

I like this, as it mirrors the markdown image syntax, which is a sort of an embed!

[mathieudutour]

I'm wondering if this can cause some conflicts in certain languages: for example spanish starts exclamation sentences with ¡ (and ends with !) and so a link right after the start of a sentence would be interpreted as an embed (if a language has a similar "feature" with ! at the beginning). (Really not sure but might be worth researching)

[nixsee]

I have nothing to add, really, other than to say that I'm extremely impressed and grateful for all of the deep thought and hard work you've put into this. It's clear to me that I just need to step away for a little while, do some writing, and let the adults keep working with fewer disruptions. I look forward to seeing what you guys come up!

[nixsee]

I think this is the right RFC to suggest this.

I am wondering if/how Foam would handle/render links that point to uncreated files. Its not necessarily a problem within Foam, since clicking it will just create the file. But what about if we're publishing our notes on an SSG? You wouldn't want to have links pointing to nowhere.

I think it is pretty common for people to create placeholder links to want-to-create pages as they type, but not actually create those files until they're ready. This would result in many broken links online.

So, perhaps there could be some sort of mechanism whereby links can be ignored or removed on preview/publish if they point to a non-existent file?