Icons & Usability For Links To External Files

[metal450]

When CherryTree contains many attachments, the ctb file can become pretty enormous. This can make syncing & backups both slow & space-inefficient. Rather than storing attachments as blobs within the database, I'd like to request the option to store them externally. Attaching a file would simply copy it to an /attachments folder next to the .ctb, & the db would contain the path to that file.

This is related to & discussed in #922, but I thought I'd make a distinct request to elaborate & avoid confusing the two.

Advantages:

• Dramatically faster syncing & backups, due to much smaller database files
• More granular versioning: external backup software can see which attachment changed & needs to be versioned, rather than needing to version the entire db
• No risk of data loss when editing an attachment, as you're just editing directly on disk (i.e. even if you close CherryTree before saving the external edit, your editor just saves directly to disk).
• Attachments can be accessed on platforms where CherryTree is not yet supported (i.e. mobile)
• Shouldn't require any major change in architecture, as all other aspects of the storage can remain the same.

[metal450]

@giuspen I wonder if you might be willing to consider a 'priority bump' for a feature or two in exchange for another contribution, like last time? :)

[giuspen]

Hi @metal450 this task may look not too big but it is a bit of a pandora box because it is the first time that cherrytree would support a non single document format.
Issues like the undo/redo, password protection, would need to be taken care of too.
I think that a new data format will have to be supported where multiple files in multiple folders are used and possibility to encrypt single nodes only. This cannot happen now as I have a backlog of smallest features/improvements that I want to deploy before.

[metal450]

Hi @giuspen,

Thanks for the reply.

While such a full-featured implementation also sounds great, I had envisioned something a bit different, & much simpler. The intent wasn't really to have a "multi-file database", but to actually have attachments be direct external files - & thus, editable entirely outside of CherryTree via their native editors (i.e. the 4th bullet in the OP). This would of course not support password protection or undo/redo for the attachments themselves, but that's perfectly reasonable - if the user chooses to enable the "store files as links to external files", can simply show a dialog: "Note that if you enable this feature, attachments will be stored outside of CherryTree's database, will not be password-protected, and changes to them will not be tracked by CherryTree's UndoRedo history."

I think what you had in mind would solve the "more granular backups" issue, but if attachments somehow still support password protection, that would necessarily preclude the ability to edit them directly (bullet 4). So the intent here was really just as simple as "CT is just a helper to copy files to a directory, & facilitate opening/renaming/deleting them):

• All attachments from all notes could be in a simple flat dir, named as i.e. originalName + uuid (to avoid collisions but still be human-readable). The db just stores filename.
• When deleting an attachment & saving the ctb, delete the file from that directory
• When renaming an attachment & saving the ctb, rename the file in that directory
• When opening an attachment, open the file in that directory. External file is edited directly, so saving the changes there is sufficient (no need to save the changes in the external editor, & then save changes in CT - could even quit CT while the external editor is still opened, & all will be good).

Does that make sense? :)

[metal450]

Edited previous comment slightly (in case u read the email notification copy, instead of above 😄 )

[giuspen]

What I don't understand is why the already existing links to files cannot be used for the purpose. Links to files relative to the cherrytree document are also supported. Yes you manually have to create a folder where you keep those files but it's already there. Your request may become an improvement of the existing links to files.

[metal450]

Oooohhh, wasn't even aware that that was an option! Then yeah, seems like it's really just a UI/usability thing :)

Currently (from what I can tell), the way to achieve this is to: create the folder (one time setup), then each time we want to attach a file, navigate there with the system file explorer, move the file there, type some text in a note, give that text a hyperlink, then either select the file in the hyperlink dialog & edit the path to remove the prefix (so it's relative), or just manually type the relative path to the filename.

Essentially, what I'd hope for is a workflow that's more or less like the current attachment, & to show the files as icons rather than text links:

1. Click an "Insert (External) File" button. (no need to type text that will represent the hyperlink)
2. Select a file.
3. CT copies the file to /attachments dir, relative to the ctb file. (no need to manually copy the file, or manually type/edit the path)
4. Attachment is shown as an icon+filename, just like current attachments. Could optionally have a little link overlay on the icon to disambiguate from 'internal' attachments.
5. If the "icon" in the note is deleted, also delete the file. If it's renamed, rename the file.

If I were just to guess, I'd think that quite a few people might just not realize this is an option (I certainly didn't, & I was looking for it! :) ), so this might actually solve the 'big database' issue for many, if it were just a bit more streamlined. So from a usability standpoint, we can interact with attachments just like now, all that really changes is the storage.

[giuspen]

Yes I see the enhancement of the links to files to have dedicated auto generated folder(s) and be kept in synch as for name and removal

[metal450]

Any possibility of donating for a bump-up/solution of this one too, maybe? 😄 🙏

[giuspen]

I'll give it a try

[metal450]

Sweet, much appreciated!

[Ricent82]

I would also like to vote for this, if attachments were managed together but external the db, then backup software can version each file individually instead of versioning the entire db. Would be really great for change tracking.

[giuspen]

I think this other approach #2123 would benefit a wider audience and satisfy also this request

[metal450]

Indeed that would be even better, but it seems like a far, far bigger endeavor of which this is a subset. i.e. isn't this a "prerequisite" of that either way - as that change would require both handling attachments externally (as described here), then in addition, a complete redo to the storage of notes themselves?

Thinking of a venn diagram, to me it seems like that feature contains this one. Definitely better, but also much more involved & farther away.

[giuspen]

The introduction of a new Multi-File based data storage may be not too far away actually. I can see the benefits for versioning and for people with large databases who do not care about having all in a single file. I'm thinking about not supporting the password protection for this storage type at least at the beginning

[metal450]

Yup, having both attachments & content out of the db would be awesome. For me, getting attachments out of the db just seemed as high-value first step :)

[giuspen]

This has been implemented and while still under testing will be in 0.99.56

[metal450]

Amazing, can't wait to give it a try!!

[metal450]

This ticket was resolved, but unfortunately one of the primary motivations seems not to have been addressed (the ability to access attachments outside of CherryTree). Since I can't reopen, I copied forwards those portions of this issue to a follow-up issue: #2354