Add ability to specify which collections the filer can read from & write to

[Material-Scientist]

Hi,

I have several filers connected to the same store (Redis). Meaning, they all have the same view of the data. But suppose I have several groups of users, some of which should only be exposed to certain collections.

Currently, I can start a filer with the -collection argument to specify which collection the filer should write all files to.

Specify read permissions on filer-level

However, would it make sense to also be able to specify which collections the filer is able to read from? E.g. specified as list of comma-separated string: -collections_readable="collection1,collection2,collection3"

Specify write permissions on filer-level

And while -collection is useful for writing to a default collection, would it also make sense to be able to specify all collections the filer is allowed to write to if the default were overwritten in a request? E.g. -collections_writeable="collection1,collection2,collection3"

Emulate ACL

That way, you could configure user_group1 to e.g. only have read-access to collection1, user_group2 to have read & write access to collection1, and user_group3 to have read & write access to all collections.

Setup with 3 filers:
weed filer -master="<master_ip>:9333" -filerGroup="default" -ip="<filer_ip>" -port="<filer_port1>" -collections_readable="collection1"

weed filer -master="<master_ip>:9333" -filerGroup="default" -ip="<filer_ip>" -collection="collection1" -port="<filer_port2>" -collections_readable="collection1" -collections_writeable="collection1"

weed filer -master="<master_ip>:9333" -filerGroup="default" -ip="<filer_ip>" -collection="collection3" -port="<filer_port3>" -collections_readable="collection1,collection2,collection3" -collections_writeable="collection1,collection2,collection3"

All 3 could share the same filer.toml where the store is set to Redis:

[redis3] # beta
enabled = true
address = "<redis_ip>:<redis_port>"
password = "<redis_pw>"
database = 0

Collection-specific vs. directory/file-speficic access roles

This 'collection-centric' approach could also help with authorization, as it's currently not possible to secure SPECIFIC directories and/or files with JWT.

Instead of configuring directories/files, you could set different JWTs with different filers (and therefore sets of collections).
Finally, you could route a user to a specific filer, based on what permissions you want to give them, and require them to also submit the correct JWT token for that filer.

E.g. user_A belongs to user_group1, and can therefore only read collection1 from the filer at <filer_ip>:<filer_port1>.

This approach doesn't require any updates to the underlying data or the metadata store. If you want to stop serving a specific collection to a user group, you can just remove the collection from the filer startup parameters and restart the filer.

Data versioning/recovery

This would also allow you to version data under different collection names. E.g. if you made a change to a file on collection1, you could push the old data to collection1_old1 with a ttl as described here, and then just launch a new filer with read permissions to this collection to retrieve it.

Potential collision

However, while this approach helps with specifying who has access to what data, 2 groups who have access to different collections could write a file with the same filename to the same directory, in which case you get this problem for user-groups who have access to both collections. It may be helpful to issue a warning/error on write, or indicate to the user-group who sees both collections which version is from which collection (prepend/append/ or show in parenthesis).

Alternatives

I suppose an alternative could be to use the tag system, which is currently only supported for PUT/DELETE operations. However, that would require writing metadata for each file, which is maybe not so ideal.

Worth exploring?

What do you think? Is this something worth exploring or are there any immediate arguments against this approach?

[Material-Scientist]

Realized this is possible with S3/iam 👍