Support different consume folders for owners with multi-user support

[shamoon]

First of all: thank you so much for working on this important feature! I didn't have the time to test it but it looks like it's going to be great.

Second, I now recognized that new docs uploaded, have no Owner set. Honestly, I am fine with this. Sine all users can see uploaded docs by default, but can be hidden if this is preferred. But I wonder if this is the intended default behavior. Would it be possible to assign groups by default to new documents by a rule? Otherwise it would be a lot of work if it needs to be done for every single document - guessing that same group assignment is used for the majority of the documents and only a few will have a modified set. What do the others think about it? I guess there are a lot of different use cases if it comes to Multi-User.

Related to that, I would have a "feature suggestion" that might be linked/a partly solution to this problem: would it be possible to have subfolders in the consume folder that are linked to the users? And maybe a "general" consume where no user is needed/"open to the public".
I was thinking about the following workflow: if I scan to a network share and have the different user/team folders in the consume folder as destination presets, the document would be directly linked up to the correct user and hidden for the rest (if that is a default setting for the user). That way, a user could just choose which (sub-)folder they want to scan to (preferably their own) and the document would be correctly linked to the right user/team account. If you scan to a "general" consume, all users could potentially see the document and interact with it.
This of course only helps for the consume folder but I think this is the default ingress path for many people anyways.

Originally posted by @TheDom42 in #2147 (comment)

[Limerick-gh]

In general, I like the idea to utilize subfolders for this as it's straight forward.
The downside is that it might conflict with the setting PAPERLESS_CONSUMER_SUBDIRS_AS_TAGS= which does already use subfolders for different purpose.
Unfortunately, all other options coming to mind like naming convention for uploaded file, barcode usage are not really the way I like to use it.
Personally, I am okay to have documents being public by consume folder and personal documents when upload is done via the web interface, where authentication is already in place.

If user subfolders for consume folder is preferred, then I guess we need kind of variable like PAPERLESS_CONSUMER_SUBDIRS=<tags|owner> where just one of those could be set. Or does one see option to use both side by side?

[TheDom42]

This also works, but my suggestion was actually a bit different: on level 1, I would have only have two (verbatim "users" and e.g. "teams", if that is wanted) or even only one (verbatim "owner") reserved subfolders. And then on level 2 you would have the user-named folders and on level 3 the tags for that user. That way you would only need to reserve the terms "user" and "teams" or "owner" instead of the actual user names.
I think your solution is also good and possbile, I just wanted to clarify the other one.

[Limerick-gh]

Now I got you! 😀
Probably your approach is easier to understand as it's following a clear structure without any exceptions. 👍
Let's wait for further voices.

[BjoernSchotte]

Really like both ideas. With a new scanner in place which supports more than 1 network folder profile connection (scenario: my wife and me), I would also prefer a way to have different consumption directories for different users/tems.

Solution from @TheDom42 seems to be more flexible as it takes also other scenarios like teams into account.

As far as I've understood @TheDom42 this would be like:

consumption/
'- users/
|  '- name1/
|     '- tag1/
|     '- tag2/
|     '- tag3/
|  '- name2/
|     '- scan.pdf
'- teams/
|  '- marketing/
|  '- customerservice/
'- globalscan.pdf

?

[andreheuer]

Hi @BjoernSchotte,

I think, this has to be a bit more detailed out. A document in 1.14 can have an owner (afaik who controls the permissions - this can only be a user, but not a group). Additionally, the document has specific permissions: view and edit permissions to which a set of users or groups can be assigned to.

So based on your above folder structure: should this be the "detected" user be the owner, get view or edit permissions? Almost the same for teams: should these user get view or edit permissions?

I am also not sure, what would be the best solution here 🫤

BR
André

[BjoernSchotte]

I think owner permissions are the first, then view/edit permissions follow.

That would mean:

1. within /users/username there's no problem: Owner is the specific user who can then distribute view/edit permissions to other users gradually
2. within /teams/groupname (maybe the folder should be renamed to /groups/ ) there should be a "default owner" when creating a group inside paperless-ngx. So ownership would be this specific user with view and/or edit rights distributable to other users of the group.

Maybe for 2.) there needs to be some "default options" when users have been assigned to this group: who is default owner of documents for this group and who can have additional view/edit rights.

Group: marketing

User	Ownership	View	Edit
bjoern		X	X
marina		X
office	X	X	X

So there's a general "office" user who is the owner of the document. marina can only view the documents while bjoern can also edit them.

Does that make sense?

What happens when the owner leaves the company? I.e. imagine "marina" has ownership, leaves the company and her user will be deactivated/deleted. Will ownership of already existing documents be redistributed to other users?

[TheDom42]

Two weeks have gone by so far and unfortunately we did not hear many more voices with input on the topic. What's your take on going on with this, @shamoon?

[shamoon]

We use these discussions to gauge community interest in what features to work on but it’s not like there’s a timeline on that since we’re all doing this on a volunteer basis. If it continues to gain popularity perhaps someone will pick it up eventually but I wouldn’t necessarily expect that any time soon

[uhoenig]

Just want you to know that I'd be still interested (and maybe also a silent majority).
My suggestion would be to create two subfolders. The first one for the user with the possibility to have an every_user folder, too and the second one for the tags.
Thanks for the nice work and I hope this implementation isn't very hard to achieve.

[andreheuer]

When thinking today about my future "scanning process", I also stumbled across the same issue. I would also really appreciate and like the approach of having dedicated folders for specific users! That would made really sense!

[ruppsn6923]

+1

[ph1248]

Very interesting discussion!

I would like to suggest a slightly different approach. (At least, I think it's different. My apologies if this has been suggested before :-)) I am aware that the following approach will probably require quite some work to implement. However, I think that it might be a nice and flexible approach which could bring a lot of benefits over time and cover a lot of use cases. Maybe it could be implemented step by step.

Note: The details as described here, e.g. how the conf file could be structured, are just suggestions for the illustration of my proposal. They don't have to be implemented like this, there are lots of other possibilities to implement them.

The scenario I would prefer is as follows:

1. In the config file paperless.conf it would be allowed to have more than one (ideally: unlimited) definitions of consumption folders.
2. To this end, instead of just a line there would be for each consumption folder a whole section in the config file. A bit similar to the way how more than one share can be defined in a samba smb.conf. Each such section would be started by [consumption_folder] or something like that.
3. In these sections we would have the following settings (which could then be different for each consumption folder):

• NAME=<name> --> new setting, to designate the consumption folders later in the GUI if needed. Required.

• PATH=<path> --> replaces PAPERLESS_CONSUMPTION_DIR. Required.

• OWNER=<json> --> new setting, a list of the owners of the documents after they have been consumed. Optional. Can be set to "*" to set all current users as owners. If this setting is missing, the superuser would be the default owner.
  Edit: From the answers below I was wrong in assuming that multiple owners are possible. Therefore, this setting should be just a string instead of a json list. To allow more than one user to work on new documents (e.g. for teams), we could think about setting also default viewing and editing permissions per consumption folder.

• DEFAULT_TAGS=<json> --> new setting, a list of tags which all documents from this folder get by default. Optional.

  The following settings could also be moved from the general settings into the section defining a consumption folder. Thus, they could be set differently for each consumption folder

• CONSUMER_DELETE_DUPLICATES=<bool>

• CONSUMER_RECURSIVE=<bool>

• CONSUMER_SUBDIRS_AS_TAGS=<bool>

• CONSUMER_IGNORE_PATTERNS=<json>

• PRE_CONSUME_SCRIPT=<filename>

• POST_CONSUME_SCRIPT=<filename>

• FILENAME_DATE_ORDER=<format>

• IGNORE_DATES=<string>

• DATE_ORDER=<format>

• CONSUMER_POLLING=<num>

• CONSUMER_POLLING_RETRY_COUNT=<num>

• CONSUMER_POLLING_DELAY=<num>

• CONSUMER_INOTIFY_DELAY=<num>

Example:
EDIT: Please note that the OWNER setting in this example should rather be a single string, as discussed below.

(...)

[consumption_folder]
NAME="Alice's mobile phone"
PATH="/home/alice/mobile_phone_sync/DCIM/scan_it"
OWNER=["Alice"]
DEFAULT_TAGS=["Tax", "International", "Confidential", "*IMPORTANT*"]
CONSUMER_INOTIFY_DELAY=5
\# since Alice's mobile phone scanning app tends to create unwanted subfolders, 
\# we want to consume them, but we don't want to use them as tags:
CONSUMER_RECURSIVE=true
CONSUMER_SUBDIRS_AS_TAGS=false

[consumption_folder]
NAME="Bob's scanner"
PATH="/home/bob/scanner"
\# Since Bob reports to Alice, she needs to see his scans as well:
OWNER=["Bob", "Alice"]
DEFAULT_TAGS=["Tax", "Confidential"]
CONSUMER_INOTIFY_DELAY=5
CONSUMER_RECURSIVE=true
CONSUMER_SUBDIRS_AS_TAGS=true

[consumption_folder]
NAME="Department letter scanner (non confidential)"
PATH="/mnt/document_server_nfs/scanner_incoming_letters_public"
\#It's the incoming paper letters, so everyone in the department should be able to see them and sift through them:
OWNER=[*]
DEFAULT_TAGS=["Letters", "Non-confidential"]
\# It is a network drive, so we need polling:
CONSUMER_POLLING=30

(...)

5. The system would treat each of these consumption folders basically as now. That means, the folders are polled or watched via inotify, subfolders could be converted as tags, pre- or post consumption scripts could be executed, etc.
6. Regarding the setting "DEFAULT_TAGS, I can think of quite a few use cases in which this might be beneficial. E.g., for some scenarios, instead of trying to move all documents to tagged subfolders in one consumption folder, it would be easier to use an additional consumption folder, maybe on a different drive or even a network drive, and mark the documents coming from this folder with some special tags.
7. As a future feature, one could also think to set the OCR settings on a per consumption folder basis. This could be put on the roadmap, but it's not that important in my opinion, so for the time being they can be left unchanged. The same goes for the barcode settings.

What do you think? Could this be a target picture?

[deviantintegral]

Multiple users are in the beta version and every document has an owner set. It's a single value field.

[ph1248]

@shamoon, @deviantintegral, thank you for pointing out that owner is just a single value (actually I should have guessed that). I will update my post above for clarification.

[deviantintegral]

I do like the idea of specifying consume folders and them having their own attributes, though as noted, implementing that in general is more complex than simply having different folders

I think starting with different folders is a good first step. When it comes to cross-OS filesystem compatibility, folders really are the only reliable method of inferring metadata. But, with individual folders, it would at least solve per-user consume folders. I imagine many users would expose those folders via a network share, where it would be easy enough to limit access to each folder to a specific user.

[vistalba]

Just got the update to the new version which allowes multiple users! A big step forward!
Now I just came to this FR as it is exactly what I'm looking for. I do scan documents to different folders per user. So would be nice if they would get the default owner

/consume/user1 --> user1
/consume/user2 --> user2

If a document is placed in /consume do not set a owner -> this is how it works today.
If a document is placed in /consume/user-not-existing1 do not set a owner. --> if the username as folder does not exist in paperless... just don't set any owner, so everyone will be able to claim the ownership of that file.

[ruppsn6923]

Just a comment on that. I suggest to use the user's Id (primary key or something that is more stable than the name) as long as the configuration is not entirely possible via the UI. Why? Imagine the user name is changed, this will have side effects so that the consumption process will no longer work. No big deal if you remember to change the consume sub folder's name as well.

[nosebeggar]

I just wanted to weigh in and say I'd love this. Multi-user is pretty useless for me as it is, because the consume folder is our main way of adding documents to paperless. Currently I have no choice but to run two paperless instances.

[feuerwasser85]

I would also love to have multiple consume folders. My only need is to set the ownership, so no fancy stuff like tags or so for me :-)
Currently we set an inbox tag which leads to setting the ownership manually. This is ok but multiple consume folders would cut away this one step and would have a high w.a.f. ;-)

[philipp-koch]

It seems indisputable that for numerous users, it's a significant use case to have documents be attributed to different users when imported through the consume directory. I am one of them, too! :-) For me, it would be perfect to simply designate subfolders to users (consume/philipp --> documents in here are automatically given ownership of the user philipp; consume/wife --> those are given the ownership of user wife).

I personally don't use the mentioned tagging-by-subdirectory feature, so I'm not sure if I understand the use case correctly. But maybe those directories could start with a hashtag sign, for instance consume/#invoice, and in order to distinguish tags from usernames (since they could be identical), maybe usernames could be prefixed as well, say, with an exclamation mark:

consume/
    !philipp/
        #invoice
    !wife/
        #invoice
    #invoice

• Files placed directly in consume would get no owner assigned (which is currently the case, so no workflow relying on that would be broken).
• Files placed in consume/!philipp would be assigned to the user philipp (if it exists, or else have no owner).
• Files placed in wife/#invoice would be both assigned to the user wife (if it exists, or else have no owner), and be tagged as invoice (if no such tag exists yet, it would be created).
• Files placed in consume/#invoice would be assigned to user, but tagged as invoice.

---

Now that I've written the above, I've come to an IMHO much better / easier solution:

• Allow for an arbitrary number of subfolders in consume.
• Each folder may have a file named .paperless-conf (or similar) with some simple format (maybe YAML) to declare how files placed there are to be consumed:
  • declare an owner (or don't)
  • declare any number of tags to be assigned (or none).
  • declare which user may view the document (or none)
  • declare which group may view the document (or none)
  • declare which user may edit the document (or none)
  • declare which group may edit the document (or none)

This way, it would be really easy to define any number of complex consumption scenarios, keep them separate from each other and use them by just placing a file (or have your scanner place your file) in a respective directory.

Example for .paperless-conf in consume/philipps-dubious-horse-invoices:

owner:
  - philipp
view:
  - theUsergroupAllowedToView
  - wife
edit:
  - wife
tags:
  - invoice
  - battery
  - horse
  - staple 

If no .paperless-conf is present, files placed in such a directory would be assigned no user, no tags, etc. In other words, nothing would change for existing workflows, but special cases could be added anytime by creating a subfolder with a configuration file.

[kungfoolfighting]

I would also very much appreciate this feature. I think having an optional consume directory per user, that must then differ from the general consume directory and is customizable in the user settings, would make everything very easy to implement in the UI and not clash with the subdirectories as tags feature.
If someone wants all user consume directories to be in subfolders, they can just create them as sub-directories of their consume directory and target those sub-directories via the per-user consume directory setting.
I am currently at the point where the wife and I want to digitalize all our documents and are using network folder scanning for this. Without this feature it would be a lot of manual tagging of documents.

[nosebeggar]

I urge everyone in this thread to go here on github and upvote this topic.
More replies does not necessarily warrant feature integration.
It seems the thread upvotes count!

[bhotrock]

I'm just getting started with Paperless-NGX. Planning to use it with a small office of about 12 people. Paperless-NGX wasn't even on my consideration list until the multi-user support was added. Glad to see the enthusiastic development team hard at work on this great project.

Multiple consume folders would definitely reduce the amount of manual work required as we digitize years of paper documents. I think either of the methods suggested by @ph1248 (adding [consumption_folder] sections to paperless.conf) or @philipp-koch (adding a .paperless-conf file to subfolders of the consume folder) would provide for a flexible system that would be adaptable to many different use cases. I'm not sure if one, or the other, would be easier/quicker to implement. While it would be IDEAL to expose this configuration in the GUI, I see that as only icing on the cake. Setting this up manually through config files is just fine with me, since for the most part, it's something you don't setup/change very often.

I would love to see this capability added if possible. I understand the developers are volunteers, and I'm grateful for all the work that they put in!

[andreheuer]

Actually, I have started some straight forward implementation. My capabilities are also quite limited, so backend development is fine, but front-end is not my topic ;-)

So my current implementation works as follows:

• configure a setting to enable user / sub-folder matching
• create a sub-folder in the consume folder matching a user name
• place a file in the that new sub-folder
  • if there is a matching username, the document owner is set to this username
  • if there is no matching username, it is either ignored or (depending on the setting) the document is tagged with the sub-folder name

As said, this is still work-in-progress and I still need to test the implementation and also things might get changed. As soon as it its done, I will follow-up.

[shamoon]

See #4196

[nosebeggar]

Fantastic! Will this be in the 2.0.0 release?

[github-actions[bot]]

This discussion has been automatically locked since there has not been any recent activity after it was closed. Please open a new discussion for related concerns.