Folder Structure & Metadata - Brainstorming

[arunoruto]

I came back into reading and setup stump again on my nixos server to track my books.
While I am mostly satesfied with how it works, there could be ofc some imrovements here and there :)

I noticed that there are many issues which are related to how to structure the folders and files, and it will be hard to set one structure which will please everyone or even a majority here?

We could use the discussion here to collect some ideas on how to organize the folders and files and what users are even expecting. For example, I have the most basic structure with Author -> (Books | Series -> Books) and maybe this would be enough for most of the people here (contradicts my statement above, but who knows).

To start this off, here is an idea I would propose:

Note

I am comming from the arr stack and I am used that my new files get auto formatted right away when they get imported. Movies and shows usually have some good metadata sources and importing a known series we know what to expect kinda. Books are a bit diffuclt here, since there is no real "imdb" for books, but there are still sources as presented in some issues.
Lets assume we are able to get metadata from a book, which means can fill out some of the attributes of a book in the database like title, author, year, series, genre, volume/series number, isbn, etc. We can then let the user decide based on the metadata, how they want to structure their files using templating per library. This means a user can also have one template for epub books and another one for comics, as long as they are in separate libraries.
Stump would need to be able to "consume" books, either through webui uploads or a folder where the data will be consumed. Since the metadata assignment isn't perfect, the confirmation for the metadata needs to happen manual, i.e., the user gets a form to fill out for an uploaded book or consumed books from a folder. The user should also be able to use a metadata provider to search for the book use the metadata from there!
Only when the metadata has been confirmed, will the file be placed in the correct folder with the correct filename.
I understand that this is the opposite way of how stump is doing it now (Structure -> Metadata), but I think moving to Metadata -> Structure would unlock huge potential.
This doesn't have to be set in stone ofc. Stump can keep track of files which have been imported, and if a file isn't in that list, we can still consume them, fetch metadata and populate the information for that book.
If someone is using services like radarr and sonarr, this workflow should be quite familiar to them, and I guess its the most intuitive IMO.

These are my 2 cents, feel free to post yours too 😁

Some relevant issues to look into:

• [FEATURE] Folder Structure #254
• [FEATURE] Nested series #280
• [FEATURE] Metadata-only library #288
• [FEATURE] Metadata Search System #789

[aaronleopold]

I do have lots of thoughts in this realm, and will respond more meaningfully once I return from vacation.

The current plan in my mind is to improve metadata fetching as a means of funneling better experiences of metadata-based structuring of libraries (not strictly a pre-req, but would be nice for folks with less-good metadata). I added the concept of a library type already, which I planned to use to also influence how a library presents on the UI, e.g. for my own use case I'd want to see something like authors -> series / standalones / shared works -> books for libraries I configure as primarily books.

I think it makes sense to discuss and consolidate those issues, so at the end of this when consensus is reached I have no qualms closing them out for a new one with better-defined direction. I'd like to see multiple participants before calling it, though

[aaronleopold]

Some folks shared thoughts on Discord, I'll try to find some time to paste them here to keep discussion centralized

[aaronleopold]

Reply 1:

A workaround Im currently using along those lines is using seperate libraries for different comic universes/publishers with runs/oneshots seperated out into their own series. However inbuilt nested series would be quite a nice addition.

Reply 2:

I personally add metadata to the comics I place in stump, so with this structure I think the metadata serie is the one to look for.

If it's empty, it get treated as a single book if instead have a name it's a series.

Or I used to thinker with Komga before moving to stump and there you pick a folder name and every time it found a that folder name it treat everything inside as one shots. But this means you need to have the library structured in a specific way.

So the specific metadata used for comics and manga could really be series field in the ComicInfo.xml file.

[Cducharme84]

A metadata forward library option is a great compromise I think, for me metadata superseding folder structure is 100% the desired method as I have a very heavily tagged 90k+ comic archive collection but it's also not safe to assume everyone has vested that level of detail.

Stump's filtering based on metadata shows that it's currently storing and retaining a large portion of the data already which would hopefully help that library option be coded.

I love your idea of being able to possibly custom select the metadata ordering each library is utilizing (at least that's how I read your statement from a few weeks ago) for presentation some may want Author > series > title others may rely on more comic oriented Series > Volume > Issue or any other random combo that suits that library owner's need) and think that'd be a standout feature.

[arunoruto]

A metadata forward library option is a great compromise I think, for me metadata superseding folder structure is 100% the desired method as I have a very heavily tagged 90k+ comic archive collection but it's also not safe to assume everyone has vested that level of detail.

Ofc, that is why automatically fetching some good metadata source is important for this to work!

I love your idea of being able to possibly custom select the metadata ordering each library is utilizing (at least that's how I read your statement from a few weeks ago) for presentation some may want Author > series > title others may rely on more comic oriented Series > Volume > Issue or any other random combo that suits that library owner's need) and think that'd be a standout feature.

Like I said, I am coming from arr tools, which let you do that already with the title of the file, but I do not see a problem with templating the folder even.
For example in sonarr, I have the following templates depending on what kind of media it is:

A similar style could also be implemented for books/comics then. As long as the number of different media formats is manageable, we could adopt a similar schema :)

[SamKook]

What I found I would have wanted while trying to figure out how to organize my books to fit with how stump does its thing was to have the option of having sub-series(Series > Sub-Series and/or Titles > Titles) so in short, simply respect the existing folder structure when creating the library.

I know I could use the Files tab, but it doesn't have as nice a presentation as the Series or Books tab have.

The Konosuba light novels is a good example of why I would want that, since it has multiple spin-offs and also some single books not part of a series(like a TRPG manual) so clicking "Konosuba" in the Series tab and then having the main series and its spinoffs show up as series inside and the other random single books show up inside there as well would be exactly what I wish for.
My workaround is to add a "Konosuba" prefix to every series inside that main folder, which I wanted to avoid and the single books show up in a separate series only named "Konosuba", which works fine, but the Series tab is cluttered with multiple Konosuba related entries instead of being neatly all inside of one.

Since that means it's basically impossible to have a fixed structure, having the ability to manually overwrite the behavior of specific folders in the structure would be ideal, since some people might mix in authors and series at the root of a library for example or have nested series like me.