Implement wikitext shortcuts as macros

[Jermolene]

It is proposed to investigate refactoring the existing wikitext parsing rules so that they generate a macrocall instead of a specific widget node. Each parse rule would map to a macrocall whose default global definition would just render the same widgets as at present, but it would be possible for users to redefine those macros so that some other sequence of widgets is generated.

For example, the prettylink parse rule currently generates the following parse tree node for internal links:

return [{
	type: "link",
	attributes: {
		to: {type: "string", value: link}
	},
	children: [{
		type: "text", text: text
	}]
}];

Under this proposal, the parse rule would instead generate the following parse tree node:

return [{
	type: "macrocall",
	name: "tm-prettylink-internal",
	params: [
		{name: "to", value: link},
		{name: "text", value: text}
	]
}];

The default definition of tm-prettylink-internal would be:

\define tm-prettylink-internal(to,text)
<$link to=<<__to__>>><$text text=<<__text__>>/></$link>
\end

Or, to render internal links as buttons, the following definition might be used:

\define tm-prettylink-internal(to,text)
<$button>
<$action-navigate $to=<<__to__>>/>
<$text text=<<__text__>>/>
</$button>
\end

This proposal reflects one of my goals for this phase of TiddlyWiki 5's development: to try to refactor more of the special, canned behaviours of the core so that they are expressed as wikitext and can be more easily customised. There are examples where this is already possible: for example, the linkcatcher widget allows navigation actions to be customised globally or locally. But there are still many examples of complex behaviour built into the core that is opaque and inaccessible.

I've experimented with this proposal and it is viable, but I'm concerned with performance -- every instance of wikitext shortcut syntax will now involve an extra parsing step. It's possible that this change will require more detailed performance evaluation and optimisation. I'll push a PR shortly that demonstrates the idea.

[pmario]

I do like this approach. ... It will increase hackability. ... But it may lead to some weird problems, that are difficult to find. ...

[pmario]

It would be nice, if aka-macros + utilities could become part of the core ;)

[Jermolene]

I do like this approach. ... It will increase hackability. ... But it may lead to some weird problems, that are difficult to find. ...

I think in most cases the resulting widget tree will be the same; it's just an extra indirection.

It would be nice, if aka-macros + utilities could become part of the core ;)

What do you mean?

[pmario]

What do you mean?

As I wrote at #3884 ... uni-link plugin uses the same mechanism, that is proposed here. So does the aka-macro, which is used for alias-handling. To support fast alias-backlinking I needed to create some utility functions, that are very similar to core link-backlinks.

Alias links also define new CSS rules, that let the user assign different link styles, depending on the alias type.

I think a generic alias handling can solve the "broken link problem" if tiddlers are renamed. ... That's why I would like it, to be part of the core. ... If we can make it solid enough.

[AnthonyMuscio]

Jeremy as a super user rather than a developer in TiddlyWiki

I think this would be great, increased hackability is always helpful, I can see that people may be inclined to change the defaults which would operate tiddlywiki wide which would be helpful in some circumstances. It would also allow people to redefine a particular rule to override the default in the current tiddler, also as a result of making the code more readable without looking deeper into the core. This once again extends the ability for people to learn about tiddlywiki by looking at how tiddlywiki does it, itself.

My only concern is that we ensure users have the ability to identify from where comes the definition of a specific macro and in a particular context and when it is redefined by another. It may be as simple as providing a custom search or a special version of dump variables such that we could find, for example tm-prettylink-internal and see the default and in which tiddler the current overriding version is defined.

It seems to me making a global change would be less common than making a local change so it would be important to be able to identify the version used in a specific context, or the default if not present. Otherwise wiki's could get awfully complex and messy in the hands of some users.

Regards
Tony

[Jermolene]

Hi @AnthonyMuscio

My only concern is that we ensure users have the ability to identify from where comes the definition of a specific macro and in a particular context and when it is redefined by another. It may be as simple as providing a custom search or a special version of dump variables such that we could find, for example tm-prettylink-internal and see the default and in which tiddler the current overriding version is defined.

A good point. We definitely need better "introspection" tools for visualising what's going on.

[twMat]

"introspection" tools for visualising what's going on.

One idea may be to have a history list for all called commands.

[pmario]

The proposed solution would remove the "backlink" information from the tiddler, because the backlinking needs
type: "link", in the parsetree.

[Jermolene]

The proposed solution would remove the "backlink" information from the tiddler, because the backlinking needs
type: "link", in the parsetree.

The parse tree would be different, but wiki.getTiddlerBacklinks() would still work as usual

[pmario]

The parse tree would be different, but wiki.getTiddlerBacklinks() would still work as usual

I don't think so. Macro definitions are not fully resolved, if a tiddler is parsed. IMO wiki.extractLinks() will search for a .type="link" in the parse-tree, which imo doesn't exist at that moment.

IMO you'd need to search for: type: "macrocall", name: "tm-prettylink-internal", to.value="link" .. to make it work again. The other thing is, that the relink plugin would have to be changed significantly. ...

There is an issue: You defined: tm-prettylink-internal, which imo has a problem. The tm- prefix is used for messages already. So this would cause confusion.

Anyway

---

I'm implementing a different concept, where the user can decide, how large parts of the "wikitext" should look like.

I want to let the user decide, how the "custom markup" should be named. .. There are some limits atm, but the concept is promising. There is a very long discussion in the dev-group with about 300 posts.

At the moment only the "block" parser is implemented in a very flexible way. The definitions are done with a \customize pragma.

There are some very interesting experiments. See: The Custom Markup edition

• Customize the HTML Details element
• Customize the list-widget

Or Create a presentation with wikitext that looks like this:

´present
´ Welcome to ''Reveal.js''
´ Embedded inside TiddlyWiki
»
  ´ Vertical Slides
  ´ Can also be managed with one level of nesting
--
´ End!
´ Have fun!
---

There is a lot of experimenting going on at the moment. So the docs lacks a little bit. ... But the story is told using Basic, Advanced and Preference as the TOC main elements.

[Jermolene]

IMO you'd need to search for: type: "macrocall", name: "tm-prettylink-internal", to.value="link" .. to make it work again.

Yes, that was my intention.

The other thing is, that the relink plugin would have to be changed significantly

Indeed, there's no obvious way around that.

[Jermolene]

There is an issue: You defined: tm-prettylink-internal, which imo has a problem. The tm- prefix is used for messages already. So this would cause confusion

Indeed, we'd need to adopt a new prefix.

[Jermolene]

I'm implementing a different concept, where the user can decide, how large parts of the "wikitext" should look like.

Yes, it's an interesting approach to customising wikitext rules, and it's encouraging how far you've been able to go.