Scrapbook

[vkbo]

After reading #517, I was wondering if it would be an idea to add a new root folder type "Scrapbook". The principle of keeping ideas is supposed to be covered by allowing note files in the Novel root folder, which are considered separate from the files actually making up the novel. However, I can see the benefit in keeping these out of the "source" so to speak.

It is possible to re-purpose the Outtakes folder for this, but you may actually want to use both. In functionality internally in the code the Scrapbook folder would be exactly the same as the Outtakes folder, so it's not a complicated implementation at all.

Any thoughts @alexisargyris or @johnblommers?

[alexisargyris]

Personally, I have both. My current setup is like this:

• "Inline" notes: brief notes along with the actual text (starting with '%'). These notes are expected to be tranformed into text in the near future.
• Ideas / Notes root folder: hierarchy (files and folders) of notes with larger bits of text. For a bit of structure but less than actual folders, I use headings.
• Outtakes: pieces of text that were removed from the actual text but their fate is still uncertain. Eventually, they will either be deleted or re-inserted somewhere in the text.

[alexisargyris]

I forgot to mention that, as @johnblommers explained below, during the whole life cycle of a writing project there is need for some kind of research management functionality. For that, I use dedicated note-taking software, mainly because I need to be able to save directly a web page. I've tried to maintain links between:

• the actual text,
• notes next to the text or in separate files
• and entries in the note-taking software

but the experience has not been so great up to now.

[vkbo]

Yeah, I have similar challenges and I haven't figured out the best way to bridge the gap yet. I essentially do what you do here.

As it is, novelWriter has the functionality to cross-link story/character notes with the scene files. That is the first, and most straight-forward step. It is still lacking a way to cross-link these notes with research. I see both the value of keeping research external and internal to a project. My main interest is in writing science fiction, where you generally want to put more than one novel in a given universe and therefore need to separate general research from novel-specific notes.

[johnblommers]

Before and during the development of a novel the writer will generally keep references, citations and related supporting matter in a folder titled "Research" or "Scrapbook". That material doesn't belong in the main novel.

Indeed that material may well need to be shared among related novel writing efforts, a common Research folder if you wish. At present I have a clumsy Research folder with images, PDFs, Markdown and text files, even some movies and CBR comic book files. And I keep track of it all with a bleeding spreadhseet.

A bit more integration with that Research material in novelWriter would be outstanding.

[vkbo]

I have on purpose avoided adding features that encourage adding general research into novelWriter projects. I know other tools support this, but encouraging it immediately invites requests for supporting tables and images, something I really wanted to avoid because it significantly complicates the editor. Only supporting plain text means the content is written to disk as-is with no fuzz.

I keep all my research notes in note-taking software which are designed exactly for this kind of thing. Personally, I use Zim, which handles links and file attachments in its markdown. This is especially convenient for research that spans multiple projects.

Adding a Scrapbook folder will allow the user to have a folder that isn't tied to a specific function in novelWriter and that flies under the radar of indexing, build tool, statistics, and all of the other main "production pipelines". The Outtakes folder does that already, and as I said, adding another is trivial. The folder can be used for research if so desired, but the files will still be text only. It is also suitable for ideas, like @alexisargyris mentions above.

Which leads me to another point. I doubt I will allow adding images to the document editor directly (although the viewer can in principle support it I think). But implementing a way to link to external files from notes, be it images, office documents, PDFs, etc, is a possibility since it is just meta data. They would then need to be clickable and open via for instance xdg-open on Linux. It is doable, but I find it a bit clunky.

[johnblommers]

I know Zim. It's an old friend. Using it for research is sensible. I will have to reacquaint myself with it.

The idea that the Research folder's contents are exposed within novelWriter and that the items in it are clickable in novelWriter, and so open via xdg-open is probably the maximum amount of integration you'll want to implement.

You are going to be asked to support images within the manuscript at some point and I hope it turns out to be easy enough to implement t hat it's worth your while to include the feature. However I'm thinking out loud that I likely am not going to use the feature. Nothing I'm working on has images in it right now.

[vkbo]

As long as the user doesn't expect a WYSIWYG type experience with the editor, supporting an image tag in the markdown is not a big challenge. Allowing the user to import an image into the project should also be fine, and rendering it in the doc viewer is probably OK too although I haven't tested it. Worst case, I have to replace the current and limited QTextBrowser widget the viewer uses with a full web browser widget that supports rendering full HTML. I've avoided this because it is a much heavier object to run and it adds another dependency to the app.

As for the research, an optional tab in the project tree that simply lists the files and folders under a specified research folder would be a possible solution. It will basically be an integrated file explorer that forwards the open call of any file to the OS. I'm not sure how useful it would be compared to simply having a file explorer window open with the same folder though.

I was more thinking of allowing linking to external files from notes. A simple link text + url entry in the project XML that can be inserted as meta data for a note.

[vkbo]

Given the discussion above, I propose the following two steps:

1. Add a Scrapbook root folder with the same functionality as the Outtakes folder. This is an easy and quick addition.
2. Add a form of attachment manager that can be used to link individual files to external documents/images/etc. Each note can then be linked with such attachments. This would require:
   • A way to maintain such a link, with a set of XML entries with an attachment title and a file URL, preferably with the option to make the link either relative or absolute. (Relative links are essential for projects in file-synced folders.)
   • A way to link an attachment from a note. Either by an @-keyword, or a listbox to be displayed somewhere in connection with the documents. This needs to be visible for the editor especially, and for the viewer can be rendered as an HTML list.

[alexisargyris]

So this means you will abandon the simple text editor you have been using up to now for an HTML-capable component? If so, then maybe (some time later) we could discuss foldable (accordeon-like) sections / headings? That's a really great feature in my opinion.

[vkbo]

No, you misunderstand. The reference to HTML is about the document viewer, which is just a simple browser. It shows the generated HTML from the same converter code used for the Build Novel Project tool. Adding image tags to show images in the HTML is trivial, that's all.

I have no plans to replace the markdown editor. Associating a note with attached external files would be purely a matter of meta data, or alternatively, an @-tag reference.

Edit: As for the foldable headings, I will have to make some tests to see what's possible to do with the Python implementation, as I mentioned in the other discussion.

[johnblommers]

Music to the ears! I like it.