globally unique id

[amitu]

Feature: terms

Status: Done

Owner: @Heulitig Started On: 10th Aug 2022

22nd September

• Adjusted Check-ID to stuck for each section only once to avoid resolving for escaped links.
• Fixed working of escaped links functionality

21st September

• Fixed resolving for links in case of [x] and [X] and other conflicting cases
• Accommodated other cases in the url regex while checking for existing links

20th September

• Adjusted to support linking logic only for markdown components.

19th September

• Adjustments in this PR - regex adjustments for change in linking syntax fastn#423
• Used a seperate URL regex to match and avoid resolved links in these cases.
  • [<linked-text>](/doc-id/[#slug-id]?) (relative links)
  • If the user had already written his/her own links (external links) those need to be ignored while resolving.
    • [<linked-text>](https://...)
    • [<linked-text>](www....)
• Did other code refactor and minor fixes (fmt + clippy) before finalizing.

16th September

• Did some progress in this PR - regex adjustments for change in linking syntax fastn#423

15th September

• Created a PR to adjust the following changes - regex adjustments for change in linking syntax fastn#423
  • To change linking syntax 2 - {<id-here>} to [<id-here>] since it is conflicting with the markup syntax.
  • Adding escape \ functionality on both linking syntax to consider link as normal raw text.

14th September

• Adjustments in Check-ID state
• id-map adjustments which includes
  • ignored capturing id on escaped out/ commented out components
  • need to ignore id defined under component definition
  • moved all ftd parsing back to ftd and used the parsing related functions from there

13th September

• Created a PR to resolve global ids from all input text sources in sections and subsections
  PR - Adjustments in Check-ID state fastn#420

12th September

• Moved all relevant constants to seperate constants.rs file
• Created a PR to add docs for reference linking in ftd.dev
  PR - Reference Linking docs ftd.dev#35
• ft.youtube adjustment to use v as argument for video-id instead of id
  PR - ft.youtube component adjustment FifthTry/doc-site#35
• Fixes to adjust the usage of youtube components in other repos based on above change
  dotcom - https://github.com/FifthTry/dotcom/pull/93
  ftd.dev - fixed youtube components ftd.dev#36
  fpm.dev - fixed youtube components fastn.com#81

8th September

Improvements on ID map construction

• ID Map adjustments

Cases adjusted:

• Id's defined under component definition ignored.
• Id's defined under commented or escaped out section/subsection ignored.
• Ignored Id's for youtube components since user can tag the same video
  having its unique video id multiple times within the package.

1st September

• Implemented Check-Id which will use the actual Id map constructed from package
  Adjusted these changes in the previous FPM PR here

29 August 2022

• Fpm PR: Construct Id Map on package in fpm

28 August 2022

• Ftd PR: Implement Check-Id in ftd. Test this by creating a hard coded id map

25 August 2022

• Created this PR - Adding universal attributes to all components (excluding term)
  • added universal attributes to all components (id, top, align etc.)
    Doc PR
  • added check for redundant declaration of universal attribute by the user on components
  • fixed all ftd tests after these changes
  • example test file added universal-attributes.ftd
  • tested on the mentioned code

  -- ftd.text foo: $id
  
  -- foo:
  id: hello

---

We are considering introducing an attribute called term, which will have allow one to add term to any ftd component, and then allow people to link to any terms with a compact syntax.

This is a little problematic as now we have both id and term. And the only reason to ever use id is to allow linking. id is just bad at it, to link with id you will need to know both the id and the document-url, and if one moves the content with id from one document to another, all links have to be updated.

So why keep both id and term? Why not fix the problems with id and not introduce term at all.

ids will need to be globally unique

This is a deviation from our current design on id, currently ids must be unique in the document (not sure if this is enforced anywhere, I think we currently just pass the ID to HTML, and if more than one node exists with same id in HTML, when linking to #<the-id> browsers will scroll to the first node with the given id).

Now we will make ids unique across entire fpm package. How can we ensure that? FTD has no idea about "package". We will add check-id state on compiler, like we have stuck-on-import etc. (Tangent-1: We can maybe also tell ftd that we are interested in unique ids, and only in that case check-id state would be reached, else compiler will accept any id, so that projects that want to use ftd but do not care about this feature, they do not have to implement anything).

If id are specified they would have to be unique.

What About Auto IDs?

If id is not specified, we currently auto generate an id for headings etc, and it is passed to HTML. This is quite handy, I often find the auto generated ID by doing inspect, and use that id. In future we will make the auto generated id (and the #<id>) to doc-site etc, so they can show link on hover:

So we will need to keep auto ids. Auto IDs are not to be unique, because we do not encourage linking to them, we encourage linking using [something][id: foo] or [foo] syntax (if id is foo), as it is cleaner, and does not break if the section moves from one document to another. Despite this, auto ids need to be supported for lazy authors, most people will not give ID to most things.

Auto generated IDs will not be unique across all documents in package, and ftd will do the auto generation itself. It will use the slug (plus number if the heading is repeated within the document).

If the heading is "Hello World", the slug is hello-world. If the slug of heading is unique we will just use the slug, but if two headings in same page have same slug, we will append -1, -2 etc in the order they were found in the page to generate unique slugs.

[Arpita-Jaiswal]

We have to relax checks on a globally unique id for some cases. This includes:

• ftd.youtube which takes the id of the youtube website.

[Heulitig]

@amitu @Arpita-Jaiswal
The syntax change {<id>} to [<id>] have some conflicts:

• - [x] item1 checkboxes (this can be ignored after match)
  If the user wants to have this same effect then he/she can use do this using the first syntax [x](id: x) instead of using [x] which is reserved for checkbox in this case.
• And in this below code snippet

-- ft.code:
lang: ftd

\-- string name: Foo

\-- ftd.text: $name
$on-global-key[ctrl-,]$: $name = Arpita
$on-global-key[ctrl-3]$: $name = Ayushi
$on-global-key-seq[shift-shift]$: $name = Rajshri
$on-global-key-seq[space-dash]$: $name = Jatinderjit

Here this [ctrl-,] needs to be ignored or should this be rewritten using escape character like this inside ft.code body. Not sure if its a good idea to ignore this after match. What should be done here ?

-- ft.code:
lang: ftd

\-- string name: Foo

\-- ftd.text: $name
$on-global-key\[ctrl-,]$: $name = Arpita
$on-global-key\[ctrl-3]$: $name = Ayushi
$on-global-key-seq\[shift-shift]$: $name = Rajshri
$on-global-key-seq\[space-dash]$: $name = Jatinderjit

[amitu]

-- ft.code:
lang: ftd

\-- string name: Foo

\-- ftd.text: $name
$on-global-key[ctrl-,]$: $name = Arpita
$on-global-key[ctrl-3]$: $name = Ayushi
$on-global-key-seq[shift-shift]$: $name = Rajshri
$on-global-key-seq[space-dash]$: $name = Jatinderjit

ftd.code is not markdown. So therefore it must not change. Only text that is to be parsed as markdown should have the link logic.