[Discussion] Compatibility issues

[mjthoraval]

I think it would be important to clarify which features are compatible with Zotero without this plugin:

• Which features will be broken if this plugin is disabled?
• Which features are simply using existing Zotero features and will continue working?
• Which features are at risk of being broken by future Zotero developments?
• Will I loose any data by removing the plugin and reinstalling it again afterwards?
• Which features will continue working from outside of Zotero even if this plugin is not compatible with Zotero anymore?
• Is there any risk to affect my existing data in Zotero by installing this plugin and later removing it?

Before investing time in organizing my notes with this plugin, it is important for me to understand how safe it is in the long term.
The example of Jurism project is interesting in this aspect. As the developer stopped maintaining it, it became incompatibly with Zotero 6. So all the Jurism users will probably loose the ability to connect with Zotero after some time.

I really appreciate all the work put in developing this plugin. But considering the amount of time invested in organizing my research notes, I would like to be fully aware of the risks before taking that direction.

As an example, I have tested with the Note Link feature: zotero://note/{library ID]/{note ID]
https://zotero.yuque.com/books/share/f3fe159f-956c-4f10-ade3-c87559cacb60/yxpiew
I understand that the note links are not compatible with Zotero without this plugin, is that right?
It is possible to have a zotero://select/ link to a note. So why is it necessary to create a new type of links instead of using the existing one?
If I click on a note link after disabling the plugin, it crashes Zotero into an infinite loop:
[JavaScript Error: "event.target.closest is not a function" {file: "resource://zotero/note-editor/editor.js" line: 68673}]
TypeError: event.target.closest is not a function @resource://zotero/note-editor/editor.js:68673:19
I haven't investigated further this error, but it is quite worrying to me that even the simple links become obsolete without the plugin.

If some features are missing in Zotero, have you tried to get some feedback from the developers to see if this feature could be added in the future directly in Zotero, or to make sure that the way it is done maximizes the compatibility with Zotero without the plugin, to make it more future proof?

Thank you very much again.

[windingwind]

Thanks for such a detailed discussion.

Which features will be broken if this plugin is disabled?

The note links(zotero://note/). You'll no longer be able to open these links, no matter where they are(notes or outside markdown files). However, the links are still stored in your notes and you would be able to access them once you have the plugin re-installed.

If you use Zotero<6.0.14, you'll also not be able to view math equations in notes. Updating to the latest version of Zotero would solve this.

Which features are simply using existing Zotero features and will continue working?

A lot of.

Another core feature, the note template, only uses Zotero prefs to store the templates and does not rely on other Zotero features. All features related to the note template, like the quick note and bi-directional link, will not be affected by future updates of Zotero.

I also maintain a note line parser to build the heading structures, find cursor positions, and decide where to insert the contents. This also uses pure JS and does not rely on Zotero features.

The export function relies on the turndown, which is also used by Zotero.

Which features are at risk of being broken by future Zotero developments?

Those UI features, like the note link preview and the workspace layout, are most likely to be affected by UI changes but are also the easiest to recover.

Will I loose any data by removing the plugin and reinstalling it again afterwards?

No. Just notice the templates will not be synced by Zotero and you have to copy them if you start from a pure new Zotero installation. The syncing settings are not shared between devices.

Which features will continue working from outside of Zotero even if this plugin is not compatible with Zotero anymore?
Is there any risk to affect my existing data in Zotero by installing this plugin and later removing it?

Everything besides the note links will continue working.
Installing or uninstalling will not affect your data.

I understand that the note links are not compatible with Zotero without this plugin, is that right?

Yes. If you prefer to process the notes in markdown format, you could export linked notes as separate files and the zotero://note links will be converted to a relative path in your disk, like ./note1.md. Under this circumstance, there will be no data loss. Otherwise, you will lose the ability to open the linked notes in Zotero.

It is possible to have a zotero://select/ link to a note. So why is it necessary to create a new type of links instead of using the existing one?

The zotero://select can only select the item in your library, while the zotero://note open it directly. We want to open the note if we click the note link, rather than jump to the collection view and have to open it manually.

The note link also enables us to open the note in the preview pane of workspace tab, the heading indexing, and support other features like preview, embed, rename, and export.

The code of opening a note link is quite simple and independent of other features. As long as Zotero still use protocol handler to process links like zotero://select, we'll be able to use the note links in the same way.

Adding it to the Zotero is absolutely better. Currently, this plugin is still in progress and we might have more parameters in the note links. So I think it is not the time to add a possibly unmatured feature to Zotero itself.

I'm not the first to create a new type of links. The Zotfile creates a open PDF link and it is finally accepted as a Zotero format in Zotero 6. If there are enough users, arguing to the Zotero team that we should make it a Zotero standard would be more convincing.

Best,

[mjthoraval]

Thank you very much for your detailed reply.
A short summary of this compatibility statement would be useful I think to add in the README.md. This is important information to understand for new users before they decide investing more time on taking notes with this plugin.
It would also be useful to add a clear statement in the note link part of the documentation to explain that these links are currently only working with the plugin and are not supported by Zotero without the plugin, but could easily be adapted in the future if needed.
Without this information, users testing the plugin for a while could be very frustrated to loose all the note links if they decide to drop the plugin after some time for whatever reason.

I had the impression that the select link was opening the note. But I realize that it was only behaving like this when the note is opened in the right-hand side pane next to the library.
It is indeed very useful to be able to jump directly to the note.

With the information you gave here, I fell much more confident in investing more time using Better Notes. Although I am still quite confused on some of the different features.

A few more remarks (let me know if you prefer that I split these remarks in different threads):

1. Would it be possible to implement the links in a way that they do not make Zotero crash if the plugin is removed (see error in my first post above)? I still do not feel very safe to leave some links in my notes that would make Zotero crash without the plugin. Maybe the Zotero developers could give some feedback on this from the devs forum.

2. The zotero://open-pdf links follow exactly the same pattern as the zotero://select links. So, if I get a select link, I can get the open-pdf link by replacing select by open-pdf. Is there a reason not to follow the same pattern with an "open-note" link?

3. I still find it annoying to have to click twice on a link to open the target note. I would prefer to have a behaviour similar to Evernote, where a single click jumps to the target note. This is probably the same on all note taking apps? The navigation between notes would be much smoother. The current preview could still be accessible through hovering or right click if needed.

4. At the moment, I understand that there are 4 different types of links:

• External links
• zotero://select
• zotero://open-pdf
• zotero://note
  I think that it would be useful to differentiate the links more clearly in the plugin, so that the user understands visually what to expect on click. I guess it would make sense to keep the link preview step on first click (as mentioned in the previous point) for external links, for safety reasons. But for all internal links, I would prefer to skip that step and go directly to the target.
  In Evernote, they make internal links in green. Making all the internal links green would be very nice. Or maybe 3 different colours for the 3 different internal links, as they are quite different.

5. As you mention, the default behaviour of the note link is to open the note in the preview pane of workspace tab. It would be nice also to be able to open the note in a new window (Shift+Click) or in a new tab (Ctrl+Click).

6. I was expecting the two arrows in circle that appear on the right-hand side when clicking on a link to update the text of the link if the title of the target note was modified. It does not seem to do that. Maybe I misunderstood what the button is doing? Is it possible to update the text of the link when the title of the note is changed?

7. If the link is inserted at the top of the note, it cannot display properly:
   

8. I am not really convinced by the concept of "Workspace" and "Main Note". I haven't used the plugin extensively, so I could change my mind after using it for some time. But from my short experience, I was expecting the plugin to be more like an advanced "Note Viewer", with similar structure as the "PDF Viewer", opening the notes in separate tabs.

a) I would have expected that any note could be opened in a "Workspace" type view in a new tab with a double click from the main library, in the same way as the PDF viewer. The background colour of the note tab headers could be made different to distinguish between PDF Viewer tabs and Note Viewer tabs. This would allow to have multiple Workspaces opened at the same time, and also allow for the Workspaces to open in separate windows. This would allow for better interactions between the library and the notes when using multiple monitors.
[Edit: I see now that it is possible to open the Workspace in a separate window, by doing Shift+Click on the "Open Workspace" above "My Library". I was trying to do it in the same way as for PDF Viewer tabs: Right-click on the tab header -> Move Tab -> Move to New Window. Hopefully this way can also be enabled. The first way to do it explained in the README.md does not seem to be valid anymore Menu -> File -> Open in New Window.]

b) The TOC on the left-hand side should be possible to toggle and change view all from the top left, with buttons similar to the PDF viewer sidebar.

c) It could be useful to have a kind of Main Note, but it should probably just be a "pinned note", or "Shortcut" as in Evernote. I find the settings around this "Main Note" very confusing and restricting. The concept of shortcut would allow more easily to switch between multiple main notes. I would move the shortcut to these pinned notes (with the home button?) to the top bar, next to the advanced search or aligned to the left side, in a similar way as the favourites bar in a browser.
[Edit: actually, any standalone note could potentially be a main note. So simply adding a note icon on the left side of the Quick Search to show only the standalone notes in the library would probably be the best way to quickly give access to the notes. From there, a double click on any of the note would open it in a new tab, with a workspace configuration.]

d) I don't like the idea of the "Open Workspace" above "My Library". It gives the impression that you have fundamentally changed the functionalities of Zotero. Or that the Workspace is at a higher level than the library. Having the notes open in separate tabs ("Workspaces") would remove the need to have that invasive change to the core Zotero functionalities.

e) The home button in the Workspace does not seem to work. If I go to another note in the main workspace, then click on home, I was expecting it to take me back to the main note. But it does not do anything and it seems to give some error:
[JavaScript Error: "event.target.closest is not a function" {file: "resource://zotero/note-editor/editor.js" line: 68673}]
Better Notes
X No attachment found

f) If I click on the links below the "pdf" icon, it opens the note in the Main Note pane. I find it a bit confusing, as I would have expected the link to open in the same pane, just below.

I am probably misunderstanding how the features should be used. I just give some impressions on what I would feel more natural at the moment, without going through the documentation in more details yet. I will give it another try later, trying to understand better all the features.
Thank you very much for the plugin!

[IpeTek]

Many thanks for this conversation! Thinking through the fundamentals from early on will contribute to this plugin becoming a vital Zotero plugin for years to come, I hope. A few comments:

mjthoraval: Which features will be broken if this plugin is disabled?

I sincerely hope that Better Notes would not interfere with native Zotero features. At the moment the Alt+F10 does not work as expected while editing a (Better) note. Instead, it modifies the level of the heading. Breaks my work flow for underlining (Alt+F10, then Tab a couple of times).

mjthoraval: 1. ...implement the links in a way that they do not make Zotero crash if the plugin is removed...

Is this what will really happen, if removing better-notes?

mjthoraval: 6. ...when clicking on a link to update the text of the link if the title of the target note was modified. It does not seem to do that.

For me it works as expected with the first link to be updated after opening Zotero. With the following ones I have the same problem as mjthoraval.

mjthoraval: 3. I still find it annoying to have to click twice on a link to open the target note.

I do not mind clicking twice. But it is annoying that the menu box opens up too easily - even when the cursor is just placed at the end or the beginning of the link. I wish I would have to always right-click separately to open the menu box. Otherwise it becomes cumbersome to navigate in a note.

mjthoraval: 4. Or maybe 3 different colours for the 3 different internal links, as they are quite different.

I am afraid I am of the opposite opinion. I am already well under way trying Better Notes as a Zettelkasten editor. I use my Main Note as an outline. Instead of link colours indicating different link types I would prefer the option to make link colours follow their position as headlines in an outline hierarchy.

Which leads to a question: In general, how much of the settings inside Better Notes could be made user-definable? E.g. the colours of the headlines depending on their outline level.

[windingwind]

@mjthoraval

Wow, thanks for your reply!

Would it be possible to implement the links in a way that they do not make Zotero crash if the plugin is removed (see error in my first post above)? I still do not feel very safe to leave some links in my notes that would make Zotero crash without the plugin. Maybe the Zotero developers could give some feedback on this from the devs forum.

I'm sure the Zotero should not crash if the plugin is removed. Only a popup to notify you to select a way to open the link. Like this:

The only way to have it work without this plugin would be to add this zotero://note to the Zotero itself. That's what we'll try to do in the future.

The zotero://open-pdf links follow exactly the same pattern as the zotero://select links. So, if I get a select link, I can get the open-pdf link by replacing select by open-pdf. Is there a reason not to follow the same pattern with an "open-note" link?

This suggestion seems to be reasonable. I'll update the note link format to the zotero://select type at some proper time, with support for both new and old formats. This change would be imperceptible.

I still find it annoying to have to click twice on a link to open the target note. I would prefer to have a behaviour similar to Evernote, where a single click jumps to the target note. This is probably the same on all note taking apps? The navigation between notes would be much smoother. The current preview could still be accessible through hovering or right click if needed.

At the moment, I understand that there are 4 different types of links:

External links
zotero://select
zotero://open-pdf
zotero://note
I think that it would be useful to differentiate the links more clearly in the plugin, so that the user understands visually what to expect on click. I guess it would make sense to keep the link preview step on first click (as mentioned in the previous point) for external links, for safety reasons. But for all internal links, I would prefer to skip that step and go directly to the target.
In Evernote, they make internal links in green. Making all the internal links green would be very nice. Or maybe 3 different colours for the 3 different internal links, as they are quite different.
As you mention, the default behaviour of the note link is to open the note in the preview pane of workspace tab. It would be nice also to be able to open the note in a new window (Shift+Click) or in a new tab (Ctrl+Click).

This could be discussed in a new thread. The click twice to open a link is the standard behavior of all links in notes, so I guess the Zotero team must have some reason to do that. Press Ctrl/Command and click the link would open the link, as in MS Word.

Shift+click is a good suggestion. I'll mark it as todo.

I was expecting the two arrows in circle that appear on the right-hand side when clicking on a link to update the text of the link if the title of the target note was modified. It does not seem to do that. Maybe I misunderstood what the button is doing? Is it possible to update the text of the link when the title of the note is changed?

As @IpeTek said, there must be some bugs with the update function. I'll check that.

A two-arrow update would cause the linked note's first line to lose its style if you have complex styles like background color or italics, etc. We discussed this topic in the very early days of this plugin's development but haven't gotten any solution. Any suggestions?

If the link is inserted at the top of the note, it cannot display properly:

Yes, that's a known problem. Thanks for reporting that. I'll do a fix recently.

I am not really convinced by the concept of "Workspace" and "Main Note". I haven't used the plugin extensively, so I could change my mind after using it for some time. But from my short experience, I was expecting the plugin to be more like an advanced "Note Viewer", with similar structure as the "PDF Viewer", opening the notes in separate tabs.

Discussions about multiple main notes are here: #58 #137

It's not that easy to have multiple main notes because users will not be able to know which note to insert the link. Opening more notes in the tabs is a practical suggestion.

a) I would have expected that any note could be opened in a "Workspace" type view in a new tab with a double click from the main library, in the same way as the PDF viewer. The background colour of the note tab headers could be made different to distinguish between PDF Viewer tabs and Note Viewer tabs. This would allow to have multiple Workspaces opened at the same time, and also allow for the Workspaces to open in separate windows. This would allow for better interactions between the library and the notes when using multiple monitors.
[Edit: I see now that it is possible to open the Workspace in a separate window, by doing Shift+Click on the "Open Workspace" above "My Library". I was trying to do it in the same way as for PDF Viewer tabs: Right-click on the tab header -> Move Tab -> Move to New Window. Hopefully this way can also be enabled. The first way to do it explained in the README.md does not seem to be valid anymore Menu -> File -> Open in New Window.]

I'll check that, though this seem to not have a very high priority.

b) The TOC on the left-hand side should be possible to toggle and change view all from the top left, with buttons similar to the PDF viewer sidebar.

Thanks for the suggestion. Planned.

c) It could be useful to have a kind of Main Note, but it should probably just be a "pinned note", or "Shortcut" as in Evernote. I find the settings around this "Main Note" very confusing and restricting. The concept of shortcut would allow more easily to switch between multiple main notes. I would move the shortcut to these pinned notes (with the home button?) to the top bar, next to the advanced search or aligned to the left side, in a similar way as the favourites bar in a browser.
[Edit: actually, any standalone note could potentially be a main note. So simply adding a note icon on the left side of the Quick Search to show only the standalone notes in the library would probably be the best way to quickly give access to the notes. From there, a double click on any of the note would open it in a new tab, with a workspace configuration.]

Yes, I also find it hard to understand for new users. The 'main note' is too wide and abstract. As for allowing all notes to be opened in a tab, that's related to the multiple main notes discussions mentioned above. We could discuss that topic first, then turn to this one.

d) I don't like the idea of the "Open Workspace" above "My Library". It gives the impression that you have fundamentally changed the functionalities of Zotero. Or that the Workspace is at a higher level than the library. Having the notes open in separate tabs ("Workspaces") would remove the need to have that invasive change to the core Zotero functionalities.

Adding it here because, in the early development, I found new users don't know where to start after they installed the plugin. Unlike other plugins that have specific functions that can be accessed in a menu or a settings pane, this plugin serves something more complex, a note workflow. The functions are separated in different places and the 'Open Workspace' is just a notice to let users know where to start: Open it, and you'll see the main note. You can start here.

e) The home button in the Workspace does not seem to work. If I go to another note in the main workspace, then click on home, I was expecting it to take me back to the main note. But it does not do anything and it seems to give some error:
[JavaScript Error: "event.target.closest is not a function" {file: "resource://zotero/note-editor/editor.js" line: 68673}]
Better Notes X No attachment found

The home button should open or focus the workspace tab. From your description, I guess you are clicking an 'Open PDF' button. The note currently being previewed would have the home button replaced by an 'Open PDF' button, which will open its parent item's attachments. This would not work if it's a standalone note.

f) If I click on the links below the "pdf" icon, it opens the note in the Main Note pane. I find it a bit confusing, as I would have expected the link to open in the same pane, just below.

Yes, that's what I forgot to hide. The 'recent main notes' menu should only be displayed under 'home' buttons, not the 'open PDF' buttons.

[windingwind]

@IpeTek

Hi, thanks for your discussions too.

I sincerely hope that Better Notes would not interfere with native Zotero features. At the moment the Alt+F10 does not work as expected while editing a (Better) note. Instead, it modifies the level of the heading. Breaks my work flow for underlining (Alt+F10, then Tab a couple of times).

I'm not sure why this happens, but I didn't touch the alt+f1-10. The tab is used to modify heading levels, but I found it is actually rarely used. I'll have it removed in the next release.

Maybe we could discuss the shorcut issues later.

mjthoraval: 1. ...implement the links in a way that they do not make Zotero crash if the plugin is removed...
Is this what will really happen, if removing better-notes?

I'm sure it won't cause any crashes. You just cannot open the note link without the plugin.

mjthoraval: 6. ...when clicking on a link to update the text of the link if the title of the target note was modified. It does not seem to do that.

For me it works as expected with the first link to be updated after opening Zotero. With the following ones I have the same problem as mjthoraval.

This is a bug. Thanks for reporting.

Which leads to a question: In general, how much of the settings inside Better Notes could be made user-definable? E.g. the colours of the headlines depending on their outline level.

The display style of the Zotero notes could be customized using the Zotero preferences, see https://forums.zotero.org/discussion/86855/make-note-editor-beautiful. Most of the note-related features are controlled by the templates, which can be found in the Edit->Note Templates and the yellowed ones are used by the Better Notes built-in features.

[windingwind]

I only replied to those questions that I think you are asking me. If I missed anything, please let me know. Thank you all for this meaningful discussion!

We could continue to discuss some points here. However, if you find something that is off the topic of compatibility issues, we could start a new issue and discuss them there.

[windingwind]

Some bugs mentioned above are fixed. See https://github.com/windingwind/zotero-better-notes/releases/tag/0.7.5

[mjthoraval]

Thank you for your replies.
I come back to the core problem reported here, that the zotero://note links crash Zotero.
That will allow you to close this thread, and I will continue the side questions in separate threads.

Here is a screen recording of the issue.
This is recorded with all plugins disabled, so this is probably not directly controlled by the Better Notes plugin. But the links leading to the crash are produced by the Better Notes plugin. It is quite worrying that they break Zotero if clicked with the plugin disabled.
Clicking on the link is working fine if Better Notes is enabled.

This is not directly a Zotero issue, so probably should not be reported in the Zotero forums. Maybe it belongs to the zotero-dev forum? Let me know if you need more information to report the issue.

Using Zotero 6.0.16-beta.3+29dd0cf5d on Windows 10, all plugins disabled.

[IpeTek]

Interestingly, I cannot replicate the issue shown in the screen recording above. Zotero 6.0.15, Win10, only Better Notes (BN) disabled. Instead, I get "Only a popup to notify you to select a way to open the link." as windingwind stated previously.

With Better Notes disabled, when I click a BN-note link, I can choose to open the linked note e.g. in Obsidian. However, the note does not open directly and I have to search for it with the note ID copied from the link text (e.g. NFUHDTX8).

So, coming back to the core problem, at least with my configuration, BN seems sufficiently future proof at the moment. Before BN I had a habit of exporting my Zotero notes to Zettlr, then editing further there. In both Obsidian and Zettlr links of the BN-exported notes work out of the box. And I can jump from a note in Zotero to its exported copy in O. or Z. by using search.

With a two-way sync the Zotero-BN-some markdown editor -combo would make an excellent toolbox for my needs.

Concerning discussing further anything goes - either in this or separate threads.

[mjthoraval]

I confirm that the looping on typeError: event.target.closest in not a function that is crashing Zotero does not happen in Zotero 6.0.15.
So it seems to be something in the beta (which will probably affect everyone soon if not fixed).
I manage to repeat this problem with a simple zotero:// empty link, which is not directly related to this plugin, so I will report in the forums.

[mjthoraval]

Here is the report in the forums:
https://forums.zotero.org/discussion/100288/bug-6-0-16-beta-3-locked-in-looped-typeerror

[windingwind]

With Better Notes disabled, when I click a BN-note link, I can choose to open the linked note e.g. in Obsidian. However, the note does not open directly and I have to search for it with the note ID copied from the link text (e.g. NFUHDTX8).

I guess this would require an Obsidian-side plugin to help Ob to recognize which note to open.

This is interesting, but I may not have enough time to create an Ob plugin currently.

Concerning discussing further anything goes - either in this or separate threads.

I think It's OK to discuss in this thread. Once we think one topic is resolved, we could hide the comments. After all, we have already discussed a lot here.

[windingwind]

About the note links compatibility issue:

I could create a minimal realization of note links as a plugin, which allows users to open the zotero://note links with no additional functions. Would that be useful?

[IpeTek]

This could be the bare-bones service promise by Better Notes, if maintaining that plugin was easy enough through Zotero versions.

[IpeTek]

By the way, is it possible to export notes with tags (from the Zotero tagging system) written to the note text with hashtags? Mdnotes does that. Would be useful when editing notes in e.g. Obsidian.

[mjthoraval]

@windingwind
I think it would give a safety feeling about using the zotero://note links from the plugin, knowing that there is a backup solution. But it would actually only be used if for some reasons you stop developing the plugin and it becomes incompatible with more recent version of Zotero (like Jurism).
Practically, I don't know if it makes more sense to completely split a new note links plugin, or enable this behaviour within the existing Better Notes as a "sleeping mode" option. If a separate note links plugin is very easy to make and would only require very limited maintenance, that may be worth doing. But it would probably not get much testing until it is really required. If the plugin is simple enough, that would still give a good starting point for anyone to restore the links, even if there are minor issues to fix.
At the moment, as long as the Better Notes features do not affect my workflow, I am fine keeping it fully operational. So it is probably not urgent. If the move to Zotero 7 would double the work, it could probably wait until that time.

@IpeTek Hashtag tagging ("implicit note tags") seems to be under discussion at Zotero: zotero/zotero#2860
This is probably something to keep in mind for developments in this direction.

[IpeTek]

Zotero: zotero/zotero#2860

Thanks fo letting me know!

[windingwind]

As this thread is not active for a long time, I will convert it to a discussion.

Thanks you all and please take a look at the pre-release of next major version, if you are still using this plugin. See #429