Meta: Issues with Table of Contents Macros

[pmario]

Proposal to have a closer look at the TOC macros

• add new "auto-close-others" if a new path is selected. see code from Andreas
• remove path and itemClassFilter from <<toc-body>> macro.
  • They are not needed there.
• simplify path construction, since new code doesn't do, what it is supposed to do.
  • The problem is the <<qualify>> macro and not how the path variable was constructed
• fix toc-item - toc-item-selected class assignment
• make the exclude parameter usable
  • At the moment it is an internal parameter, which can be "misuesed"
  • misusing the param is very error prone for new users
• Table of Contents toc-caption macro awkward behaviour Table of Contents toc-caption macro awkward behaviour #3624
• summary - issues found while refactoring core TOC macros summary - issues found while refactoring core TOC macros #2646
• BUG: toc macro recursion error BUG: toc macro recursion error #3881
• Find a way to implement "auto-expand" feature with the existing TOC.
  • I did fail with this one, because the existing toc allows multiple tags to be used

[BurningTreeC]

Hi @pmario , @Jermolene ... I'd like to add a short piece of code here that comes from a short discussion in the group and looked very interesting to me, I was thinking that this could be a fast way to create toc-structures (I've changed it a bit to be two-level, but like the class_sub part I think it can be made (double-) recursive)

\define create-tiddler-title()
[[[[$(currentTiddler)$]]]]
\end

\define class_sub() 
<$list filter="[all[tiddlers]contains:tags{!!title}]"> 
<<create-tiddler-title>>
<$macrocall $name="class_sub">/> 
</$list> 
\end

<$list filter="[[TableOfContents]]">
<$wikify name="all_subclasses" text=<<class_sub>> output="text">
<$list filter="[enlist<all_subclasses>tag{!!title}]">
<<list-links filter:"[enlist<all_subclasses>tag{!!title}]">>
</$list> 
</$wikify> 
</$list>

[Jermolene]

Hi @BurningTreeC does that code work with titles that contain wikitext (e.g. I [[ am //a// title)?

[BurningTreeC]

@Jermolene no it doesn't I hadn't checked that ...

[Jermolene]

Hi @BurningTreeC I've seen similar constructions on the mailing list, but I don't know where the idea of using the wikify widget to compose filter strings came from; it's a terrible idea! Do please help spread the word if you see it being recommended anywhere.

[BurningTreeC]

It was more the structure how it was created that looked interesting... getting the whole list at start, using it again and again

I thought maybe the button and reveal structures could also be wikified at the start and then be reused over and over again

[Jermolene]

It was more the structure how it was created that looked interesting... getting the whole list at start, using it again and again

Absolutely, yes -- that was the motivation of the 'enlist' operator...

[BurningTreeC]

it's a terrible idea! Do please help spread the word if you see it being recommended anywhere.

Well then I need to be schooled, too 😁

I'll help spreading the info, but I think I'll need to get it better, too

[BurningTreeC]

Here I don't want to go too much off-topic with the wikify widget... I'll make an issue that it needs some very clear statements in the docs for what it's designed for ... and for what NOT (or better not)

[Jermolene]

Great idea, thanks @BurningTreeC -- @bimlas made a PR but I think we probably need to start again.

[BurningTreeC]

Ok, for the toc-macros, can we define the most basic recursive structure we'd use to build upon?

[BurningTreeC]

(... is that english? 😄 )

[Jermolene]

Ok, for the toc-macros, can we define the most basic recursive structure we'd use to build upon?

Yes. I do wonder if it might be worth building a little performance test rig for this, running under Node.js so that we can also readily measure memory consumption too. I don't think there's much to it: we can use the existing --render command to render a tiddler containing an "unrolled" TOC, and time it using a new --time command that executes the rest of the queued command tokens and reports the execution time, perhaps with an option for n repetitions of those commands.

[BurningTreeC]

Yes. I do wonder if it might be worth building a little performance test rig for this

that's what I had in mind, too ... creating different variants to see which works best

do you have a simple structure you would consider the base for all or should we just gather some ideas?

[Jermolene]

I've no fixed ideas beyond what we've discussed already, and the thought that it is the additional processing to generate state tiddlers that seems to be the troublesome part of what we've got.

[BurningTreeC]

I think the first thing we can do on the current toc macro is to qualify only once before starting the recursion because the qualified value is always the same

[BurningTreeC]

basically:

\define toc-linked-selective-expandable-body(tag,sort:"",itemClassFilter:" ",exclude,path)
<$set name="toc-state" value={{{ [[$:/state/toc]addsuffix<__path__>addsuffix[-]addsuffix<currentTiddler>addsuffix[$(qualified)$]] }}}>
  <$set name="toc-item-class" filter=<<__itemClassFilter__>> emptyValue="toc-item" value="toc-item-selected" >
    <li class=<<toc-item-class>>>
      <$link>
          <$list filter="[all[current]tagging[]limit[1]]" variable="ignore" emptyMessage="<$button class='tc-btn-invisible'>{{$:/core/images/blank}}</$button>">
          <$reveal type="nomatch" stateTitle=<<toc-state>> text="open">
            <$button setTitle=<<toc-state>> setTo="open" class="tc-btn-invisible tc-popup-keep">
              {{$:/core/images/right-arrow}}
            </$button>
          </$reveal>
          <$reveal type="match" stateTitle=<<toc-state>> text="open">
            <$button setTitle=<<toc-state>> setTo="close" class="tc-btn-invisible tc-popup-keep">
              {{$:/core/images/down-arrow}}
            </$button>
          </$reveal>
        </$list>
        <<toc-caption>>
      </$link>
      <$reveal type="match" stateTitle=<<toc-state>> text="open">
        <$macrocall $name="toc-selective-expandable" tag=<<currentTiddler>> sort=<<__sort__>> itemClassFilter=<<__itemClassFilter__>> exclude=<<__exclude__>> path=<<__path__>>/>
      </$reveal>
    </li>
  </$set>
</$set>
\end

\define toc-unlinked-selective-expandable-body(tag,sort:"",itemClassFilter:" ",exclude,path)
<$set name="toc-state" value={{{ [[$:/state/toc]addsuffix<__path__>addsuffix[-]addsuffix<currentTiddler>addsuffix[$(qualified)$]] }}}>
  <$set name="toc-item-class" filter=<<__itemClassFilter__>> emptyValue="toc-item" value="toc-item-selected">
    <li class=<<toc-item-class>>>
      <$list filter="[all[current]tagging[]limit[1]]" variable="ignore" emptyMessage="<$button class='tc-btn-invisible'>{{$:/core/images/blank}}</$button> <$view field='caption'><$view field='title'/></$view>">
        <$reveal type="nomatch" stateTitle=<<toc-state>> text="open">
          <$button setTitle=<<toc-state>> setTo="open" class="tc-btn-invisible tc-popup-keep">
            {{$:/core/images/right-arrow}}
            <<toc-caption>>
          </$button>
        </$reveal>
        <$reveal type="match" stateTitle=<<toc-state>> text="open">
          <$button setTitle=<<toc-state>> setTo="close" class="tc-btn-invisible tc-popup-keep">
            {{$:/core/images/down-arrow}}
            <<toc-caption>>
          </$button>
        </$reveal>
      </$list>
      <$reveal type="match" stateTitle=<<toc-state>> text="open">
        <$macrocall $name="toc-selective-expandable" tag=<<currentTiddler>> sort=<<__sort__>> itemClassFilter=<<__itemClassFilter__>> exclude=<<__exclude__>> path=<<__path__>>/>
      </$reveal>
    </li>
  </$set>
</$set>
\end

\define toc-selective-expandable-empty-message()
<$macrocall $name="toc-linked-selective-expandable-body" tag=<<tag>> sort=<<sort>> itemClassFilter=<<itemClassFilter>> exclude=<<excluded>> path=<<path>>/>
\end

\define toc-selective-expandable(tag,sort:"",itemClassFilter,exclude,path)
<$vars tag=<<__tag__>> sort=<<__sort__>> itemClassFilter=<<__itemClassFilter__>> excluded="""[enlist<__exclude__>] -[<__tag__>]""" path={{{ [<__path__>addsuffix[/]addsuffix<__tag__>] }}}>
  <ol class="tc-toc toc-selective-expandable">
    <$list filter="""[all[shadows+tiddlers]tag<__tag__>!has[draft.of]$sort$] -[<__tag__>] -[enlist<__exclude__>]""">
      <$list filter="[all[current]toc-link[no]]" variable="ignore" emptyMessage=<<toc-selective-expandable-empty-message>> >
        <$macrocall $name="toc-unlinked-selective-expandable-body" tag=<<__tag__>> sort=<<__sort__>> itemClassFilter=<<__itemClassFilter__>> exclude=<<excluded>> path=<<path>> />
      </$list>
    </$list>
  </ol>
</$vars>
\end

\define toc-start-selective-expandable(tag,sort:"",itemClassFilter,exclude,path)
<$set name="qualified" value=<<qualify>>>
<$macrocall $name="toc-selective-expandable" tag=<<__tag__>> sort=<<__sort__>> itemClassFilter=<<__itemClassFilter__>> exclude=<<__exclude__>> path=<<__path__>>/>
</$set>
\end

sorry for the bump!

[bimlas]

It might be another issue, but it's about the table of contents, so I'll mention it here: I'm thinking of implementing a different kind of ToC macro, which displays the related tags beside the children of the current "level", so it can be much more "natural" to navigate in the table of contents.

In addition, instead of the tree structure, it only displays a "level" at a time: in fact, people just interpret one "topic" at one time, unable to read each level at the same time, but only the current block.

The problem is that people are accustomed to the ToC structure of a printed book and a really "digital" table of contents is hard to get used to. So we are forced into a solution that makes it more limited to actually utilizing useful implementations.

Here is the demo that is still under development and is currently a bit slow: scroll to the "View of context - a more sophisticated toc macro powered by kin filter" heading and have a look at Reference -> Concepts -> Filters -> Filter Operators.

https://bimlas.gitlab.io/demo/tw5/kin-filter-toc-demo.html

[bimlas]

Updated the demo: try the standard search too: it will look only in the current context.

In my opinion, the term "all-in-one" can be interpreted as follows as well: you have one mind where you have all the information, no separate compartments where you store the different topics, only the context differs in the sense in which you mean a word or concept.

Based on these, a single wiki is enough to store your programming notes, reader logs, todo lists, anything, if you do not want to search the whole "brain", you can narrow your search to the topic you want.

@Jermolene What do you think?

[Jermolene]

Hi @bimlas that's great, exploring more complex ways of slicing and dicing linked data like this is fascinating. I don't think I fully understand what it's doing, though; it is very cryptic at the moment. Anyhow, this sort of thing belongs in a plugin, at least in the early days.

[00SS]

Could add this as well if it's relevant: #3675

[bimlas]

The code has evolved into a separate plugin: https://bimlas.gitlab.io/tw5-locator

[pmario]

At the moment I'm fixing the core TOC macros and add some cool stuff ;) So don't mess with it!

[00SS]

@pmario Great to hear that!!
Please also consider including the empty caption fix as it may take time for the transclude widget to have its behaviour changed so that it will treat a blank string the same as a missing string as @Jermolene suggested .

[Jermolene]

Just as an aside, I think one of the big problems with the TOC macros is that we've loaded too many features. For instance, the ability to suppress a TOC link runs through the code, and yet seems to me to be a far from universal requirement. I'd much rather we kept the existing macros as simple and streamlined as possible, moving any new features that impact performance into separate TOC variants.

[pmario]

Just as an aside, I think one of the big problems with the TOC macros is that we've loaded too many features.

That's right. eg: "Tabbed Internal" and "Tabbed External" use the TOC macros, but also have their own "story". ... But the story handling doesn't use the existing core possibilities. So it creates an overhead, which actually limits the functionality and causes problems.

I think I found a solution here, which should make the whole thing much cleaner.

[pmario]

...moving any new features that impact performance into separate TOC variants.

We got the biggest performance hit AND functionality regression, with using <<__xxx__>> instead of """xxx""".

Related problems can be seen here Using <<__xx__>> also needed to change main parts of the recursion logic.

I'll post a link to a test version soon. ... Today if everything goes right.

[pmario]

Hi folks,

Please test: toc-refactored-wip.zip

Here's a video, that shows what's new. PR will follow soon.

[pmario]

Source code can be seen with the above "WIP TOC refactor" link #3939

[pmario]

Close this one, since it is discussed with PR now.