Epic: Project/Novel Tree Visualisation

[vkbo]

There are a few feature requests for improving the tags visualisation in the Project, Novel and Outline views. This Epic tracks them and their progress.

It would be nice if we could come to a single solution for this that is flexible enough to accommodate the various use cases. Preferably something that is hidden when not used and is also simple enough that it doesn't put a processing strain on the app. The latter can mostly be mitigated by redesigning the index to handle the data collection.

Initial participants: @xahodo, @helisdevbox, @awqk as the authors of the suggestions, and I'd like input from @tmarplatt and @HeyMyian as well, and whoever others are interested in the topic.

Feature Tickets

• Allow tag tracking from Project Notes #1686
• Custom background color of novel chapter/scene based on tag #1576
• Add column settings for project and novel tree #1352

Implementation Tasks

TBD

[vkbo]

In terms of the GUI, I'm thinking about solving it by adding a panel below the Project/Novel View (with a show/hide button) named "Highlighting" where various highlighting rules can be added.

Since there is no colour information associated with tags, it may make sense to allow the user to define highlighting rules for both Project and Novel View entries (documents and headings) based on defined criteria. For instance "char contains Jane" or "pov is Jane", based on the filter rules in Thunderbird. The user can then associate a colour with it which will be used to highlight corresponding items in the tree view.

The benefit here is that the performance cost is initially 0 if the user defines no such rules. It will require a change to the index, but that is relatively low cost since the index is updated as documents are saved, so there is no heavy lifting except when the index is rebuilt from scratch.

The highlighting rules could also contain other criteria, like "word count larger than 1000" etc, which would accommodate feature requests like a way to track scene completion goals.

Edit

It may make sense to add the rules editor in the Project Settings dialog so that they can also be applied to the Outline View where the highlight panel would not be visible. It would make the panel simpler, and it could then be injected into the bottom part of the Outline View as well. There needs to be a switch/toggle for each rule to activate/deactivate them on demand.

Here's what the settings look like in Thunderbird, for those not familiar with it.

The filters would be implemented with two drop downs and a text box, like in the screen shot above, on the form: "Variable", "Condition", "Value". With a "match all" and "match any" option. A display name for the filter, and a colour setting in place of the "Perform these actions". So a lot simpler than the one in Thunderbird.

I think this should be able to cover all requested features.

[awqk]

My short take:

This would be very powerful, but I don't see myself defining rules in advance.
I might be working on some particular thing in one session, and on a completely different thing on another.

My suggestion:

I would prefer a more ad-hoc approach to this. Click on a tag or note, and show/highlight documents containing this tag. Afterwards, click on one of the highlighted documents and open it in the editor/view panel. I am not sure if both options should be available, probably the answer is yes. Think of a tiny editor symbol and a tiny eye symbol.

• no processing required until clicking a tag

• vs. Custom background color of novel chapter/scene based on tag #1576: simple, easy to understand: no highlight unless selected, one highlight colour is enough

• vs. Allow tag tracking from Project Notes #1686: The reference panel offers part of the solution, but there are some issues for me

  • it's hard to find:
    • If there is no view panel, there's no reference panel.
    • If there is a view panel, the ref panel might still be minimised.
    • The icon to open it is tiny and may change colour
  • it changes its content
    • if you click a document, the view panel changes and the reference panel changes along

• vs. Add column settings for project and novel tree #1352: I am still struggling to think of some use for the Novel Tree View. It's somewhere between the Project Tree View and the Novel Outline View, lacking features of either the one or the other.

Take this with a grain of salt or two: I am a new user looking at this with fresh eyes. Things others may take for granted may be totally obscure to me.

[vkbo]

I would prefer a more ad-hoc approach to this. Click on a tag or note, and show/highlight documents containing this tag.

Keep in mind that a document (note) can contain multiple tags, so it would have to be a specific tag and not a document in general. You mention "click on a tag". From where do you propose to activate this feature?

The problem with the "simple" approach is that it doesn't allow for any of the other feature requests, so it is very limited.

• vs. Custom background color of novel chapter/scene based on tag #1576: simple, easy to understand: no highlight unless selected, one highlight colour is enough

What if a user wants to highlight more than one? I'm trying to consider all requested features here, not just the one about finding where a tag is used.

The rest is mostly off topic here since I'm trying to arrive at new features, not discuss current ones. Improvements to those should be in new feature requests. I'll comment on your points briefly though.

• it's hard to find:
  • If there is no view panel, there's no reference panel.
  • If there is a view panel, the ref panel might still be minimised.
  • The icon to open it is tiny and may change colour
• it changes its content
  • if you click a document, the view panel changes and the reference panel changes along

It does not change colour. It is also visible by default on the view panel, so if it is hidden, the user actively closed it at some point. The reason it is tied to the view panel is that it shows the references to the currently viewed document. Which is also why it changes when you load a different document. The tags lists do not change though.

I am still struggling to think of some use for the Novel Tree View. It's somewhere between the Project Tree View and the Novel Outline View, lacking features of either the one or the other.

It is mostly useful if you don't split your novel up into individual scene documents, or haven't done so yet. Then this is where the actual novel structure is visible. It is indeed a subset of the Outline View, but visible when you have the editor and viewer available. That is the entire point of it. It's an alternative way to navigate and edit the text after you've defined your scenes. I use it mainly when editing/reviewing, but it is also useful if you define all your key scenes before starting to write. I usually don't create a document per scene at that stage, but one per act.

[vkbo]

Continuing from @awqk's comment, I think maybe a simpler highlight solution can be implemented. Here's an alternative idea that may be much easier to use, but also less powerful. Some minimalist principles may be a good idea still, to avoid going full "Microsoft" on the problem.

• Add a search box just below the tree view where a dropdown selects search mode, followed by a free text field.
• The dropdown can contain the reference categories (characters, pov, location, plot, etc), or perhaps just "reference" and not categories, and the text field can then take a tag name as input.
• It could also contain options for word count and other stats, and even tag reference counts, where the value would be an expression of a number with an "<" or ">" in front, as an option.

It achieves pretty much the same goal as the pre-defined filter list, except the filters are entered on the fly. The downside is that you can only highlight one thing at a time.

For references, it could perhaps be possible to enter multiple tags in the search box, and have the tree highlight them in different colours.

Edit: And as a quick search option, it could be possible to populate the field from a tag visible in the editor or viewer, pretty much achieving your suggestion in #1686 (comment) at the same time.

[awqk]

Thanks for your answers, even to my off-topic comments.

Keep in mind that a document (note) can contain multiple tags, so it would have to be a specific tag and not a document in general. You mention "click on a tag". From where do you propose to activate this feature?

Highlight all items in the project tree (or lines in the novel tree/novel outline) referencing a tag if

• a project note in the project tree is selected
• a tag in the novel outline is clicked (decorate the font to make it look like a link)
• a tag within the document text is clicked (bad idea, the ctrl-click action is already in use to switch the viewer)

In a tree, the highlighting would need to propagate to parent folders. Multiple tags within project notes wouldn't be a problem.

[bottom search box]
For references, it could perhaps be possible to enter multiple tags in the search box, and have the tree highlight them in different colours.

How to choose the highlight colour if more than one tag is found in a document?

[vkbo]

Highlight all items in the project tree (or lines in the novel tree/novel outline) referencing a tag if

• a project note in the project tree is selected
• a tag in the novel outline is clicked (decorate the font to make it look like a link)

It has to be an optional feature. It cannot be something that happens that automatically.

• a tag within the document text is clicked (bad idea, the ctrl-click action is already in use to switch the viewer)

That is not at all what I had in mind. It would either need to be in a right-click menu directly on a tag, or a button on the header, or both. It must not be a default feature, but something the user selects.

In a tree, the highlighting would need to propagate to parent folders.

That would be very confusing, since documents can also be parent items, and have tags defined on their own.

Multiple tags within project notes wouldn't be a problem.

You need to elaborate because how would you select which tag to highlight in your proposed solutions?

[bottom search box]
For references, it could perhaps be possible to enter multiple tags in the search box, and have the tree highlight them in different colours.

How to choose the highlight colour if more than one tag is found in a document?

A set of colours would have to be defined by the GUI theme. It cannot of course be a very long list, so the number of tags would have to be capped at a sensible value. Maybe even just three. The first tag would take priority on the colour, but the use case here is for instance showing POV character, and you won't have multiple POVs in most cases. Since this also applies to the Novel View, you can use it there for better granularity of the novel structure.

[awqk]

In a tree, the highlighting would need to propagate to parent folders.

That would be very confusing, since documents can also be parent items, and have tags defined on their own.

Indeed, bad idea. Some other decoration is needed to show that a collapsed item can be expanded to show more highlights.

Multiple tags within project notes wouldn't be a problem.

You need to elaborate because how would you select which tag to highlight in your proposed solutions?

I would have used the label of the tree view item, but now is the moment you will tell me they can be renamed. The first tag then: If it is a project note about a pov, maybe the first tag should be the pov tag the project note is about. Don't you already have a feature in the pipeline to rename an item with the tag name? Maybe it was rename by heading, but this aims in the same direction.
On another note, perhaps it shouldn't be possible to have a different name at all. Of course, this might break things for some users.

[vkbo]

I would have used the label of the tree view item, but now is the moment you will tell me they can be renamed. The first tag then: If it is a project note about a pov, maybe the first tag should be the pov tag the project note is about.

That would be a character tag. However, inside a note under "characters" they are all character tags regardless. There is no distinction. A use case for this is if I keep all my minor side characters in a single note. Then I'd have several @tag entries. For plot notes, I always add tags for each major plot point as well in my main plot note as I tag them in the scene where the plot point is occurring.

You can pick the first tag in a note, sure, but what if the user wants one of the others? So you cannot assume, but must let the user choose what to highlight. Hence my right-click action idea.

Don't you already have a feature in the pipeline to rename an item with the tag name? Maybe it was rename by heading, but this aims in the same direction. On another note, perhaps it shouldn't be possible to have a different name at all. Of course, this might break things for some users.

There is no such feature planned for tags, no. When a note is auto-generated, the label, heading and tag are the same. But there is no requirement on them staying in sync.

[helisdevbox]

Thanks for considering these features! I haven't had the time to look into the proposed changes, will dig into this next week.

[tmarplatt]

The Thunderbird-like approach to filtering tags looks like overkill, so I'm glad that was reconsidered. A search box is all we need! As to choosing tag highlight color, this should be handled at the Note containing the Tag. (See my comment on #1576).

The dropdown is a nice to-have. It would be even nicer if you could control it with the Up and Down arrow keys directly from within the search box.

Also the dropdown should follow, not precede the search box. A user clicking on the search box is thinking about the searched term and wants to type it down as soon as possible. The dropdown being first is a visual hurdle that detracts from this mental state.

[vkbo]

The Thunderbird-like approach to filtering tags looks like overkill, so I'm glad that was reconsidered.

The full Thunderbird approach is obviously overkill and was not considered. I belive the words "A lot simpler" were added 😄

A search box is all we need! As to choosing tag highlight color, this should be handled at the Note containing the Tag. (See my comment on #1576).

I agree that a search box is the better approach. It is enough, although it doesn't fulfil all requested features. Not that it has to. A compromise is fine too. My minimal suggestion doesn't require multiple colours to be defined, only one, and it is really my preferred solution.

The dropdown is a nice to-have. It would be even nicer if you could control it with the Up and Down arrow keys directly from within the search box.

Also the dropdown should follow, not precede the search box. A user clicking on the search box is thinking about the searched term and wants to type it down as soon as possible. The dropdown being first is a visual hurdle that detracts from this mental state.

These are implementation details and are best assessed after a test implementation is made. If the box has auto-complete functionality, the arrow keys will be in use for selecting that.

Edit: The dropdown box is not only "nice to have", it is required unless you want to type the context in the box, like @pov = Jane or something similar.

I also disagree that the dropdown should come after. That's what I want to take from the Thunderbird screenshot still. What to highlight, operation (optional), content to highlight. The operation can be implemented with < and > directly in the text box, and otherwise assumed to be =.

[helisdevbox]

I agree that a search box would suffice, especially if it's available from a keyboard shortcut or from a context menu. In other words, I'd like the search to be available globally from each document/note, so I don't lose the context and the train of thought that I'm having at the moment when I need to perform the search. The search results could be opened in a separate pane, as per vkbo's original suggestion. In the best case scenario the result pane could be opened directly beneath the editor pane. Admittedly this is still very much like IDEs tend to do this, but that workflow just fits my brain, and it doesn't add additional mouse clicks just to perform a search. Furthermore it doesn't require any mental context switch, which is bound to happen when you change views in an application.

I for one am not a fan of drop down menus. They tend to be cumbersome to use and add additional mouse clicks if the keyboard navigation isn't implemented properly. I don't see myself using the Thunderbird like UI, but I'm willing to take a shot if that's how this would be implemented.

[vkbo]

The search results could be opened in a separate pane, as per vkbo's original suggestion. In the best case scenario the result pane could be opened directly beneath the editor pane.

I am not proposing a search feature here. This is not a global search. That is a completely different feature request and should not be confused with this one. My use of the word "search" seems to have been misinterpreted in that way. I thought the screenshot from Thunderbird made it clear.

What I mean by "search" is a way to enter meta data that will be highlighted in the tree. That is the feature we're discussing. Lets call it the highlight "criteria" instead to avoid further mixup.

I for one am not a fan of drop down menus. They tend to be cumbersome to use and add additional mouse clicks if the keyboard navigation isn't implemented properly. I don't see myself using the Thunderbird like UI, but I'm willing to take a shot if that's how this would be implemented.

Then I need a better proposal for selecting the conditions of the criteria. pov = Jane is of course simple enough, and requires little mental energy to remember since it is the keyword used in the text, but say I wanted to see what documents have less than 1000 words, which notes have less than 5 references, or any of the other requested features?

[awqk]

The more complex settings remind me a bit of formulas in a spreadsheet, so this is something you can do now with the version 2.3 export csv feature (though not very quickly).

Suggestion: make this less epic (= daunting) by introducing an intermediate step:

1. add "simple" highlighting options
2. follow-up with "custom" highlighting choices, configured in the settings dialog

[awqk]

Thinking about this a bit more, since #1822 came up.
Suppose you want to search/highlight/visualise something like @char: Alan.

1. Consider this scene declaration (sorry, C++ programmer shining through):

@pov: Jane
@focus: Alan
@char: Alan, John, Jane

This scene is a match.

2. Consider another scene declaration:

@pov: Jane
@focus: Alan
@char: John

This scene should also be a match, both @focus and @pov imply @char.

Searching for mentions of Alan (#1822), both scenes should match as well. In other words, @char implies @mention.

If a @location tag is used, things should work the same way. In other words: any tag automatically implies it has been @mention-ed.

[vkbo]

Agreed. This is easy enough by simply listing where the tag is used. Since a character tag cannot be used for @location anyway, this is already limited.

[xahodo]

I prefer the rule based approach, this allows for quite some flexibility.

I think "match all" and "match any" are not detailed enough for matching rules.

How about allowing a combination of "and", "or", and "and not" conditions. This could require more processing power (depending on how intensive these are used). However, if implemented well, the matching would be done only once when the appropriate GUI shows up, or when a change occurs in the rules (when they are committed). For sanity reasons, precedence of operators could apply. Then, if a match occurs, the background color could be set to a preset value.

I can imagine a clash between rules. How would this be resolved? First come, first serve (easiest to deal with)? Or an error when a clash occurs (more work, but more user friendly)?

In case of a color being set solely on basis of a tag, what happens when multiple notes set colors and multiple having colors are used in the same scene?