Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Circles, contact groups and user groups #13478

Open
MorrisJobke opened this issue Jan 10, 2019 · 24 comments

Comments

@MorrisJobke
Copy link
Member

commented Jan 10, 2019

Problem

We have so many grouping mechanisms and they are confusing for people that don't know the technical background of them. We should unify the way those are handled into a single way of grouping users.

  • circles
  • contact groups
  • (system) user groups
  • federated groups
  • ...

Idea

Idea is to have some grouping entity that is the only way of organizing a group of users and contacts.

Additionally a layer is added to the server to share to some grouping entity only and not to groups, users or federated users anymore, because those are 3 concepts that basically do the same: share something to 1...n users.

  • each user is at least in one of those groupings (the grouping with only the user itself) <- that is used to replace user shares

Attributes

  • this grouping entity has a name (random one by default)
  • this grouping entity has a type (single user, group of users)? - can be build with permissions (no adding/removing permission for anybody for the single user type)
  • this grouping entity has a permission who can manage it (joining, adding, removing users, change name)
  • this grouping entity has a visibility

Steps

  • discuss the idea (right here in the ticket)
  • define the scope and specs
  • implement this grouping mechanism (which exists in parallel to the existing ones and undisclosed to any UI)
  • implement sharing part
  • migrate user shares, groups, circles, contact groups
  • remove existing shares, rooms, circles, ...

Note: the last two steps need to be done in one release. The other ones can be done in separate major releases to be able to move this forward and not have one huge changeset that nobody can review properly.

Problems

  • the UI needs to make it easy what this grouping entity is about (admins, permissions, visibility)
  • the UI needs to make it obvious where this grouping entity comes from to avoid phishing attacks by creating a similar looking entity via federated groups for example
  • for migration: what about existing collisions? if there is a circle, a group and 3 contact groups named the same? - during migration, we can add something at the end of the name
  • how are the groups managed? User mgmt and Contacts? And additionally inherent grouping using "Linking Collaboration Things"? (Ideally the name "Circles" does not show up – it’s just "users" and "groups")

Other approaches

Linking collaboration things™ - #11015

  • links a random entity (room, file, board, ...) to another one and each user that has access to any of them then they also have access to the linked ones
  • it's still different, because the goal is not to group users but to group data objects

Design forward: How we handle Users / Groups / Circles - #4493

  • this one is another approach to not evolve but rethink the whole concept to not be limited by the existing structures

cc @skjnldsv @juliushaertl @rullzer @jancborchardt @ChristophWurst @danielkesselberg @blizzz @schiessle @nickvergessen @daita for feedback on this topic.

@jancborchardt

This comment has been minimized.

Copy link
Member

commented Jan 11, 2019

Does it make most sense to talk about it more at the Contributor Week in person? Or is that too late, and we should rather have another (or two) calls about this?

I ask mainly because of:

Linking collaboration things™ - #11015
it's still different, because the goal is not to group users but to group data objects

Technically that’s what it does, but UX-wise it’s really similar. That’s why I think we need to coordinate this here and that one, so we don’t end up in 2 different concepts again. :)

Design forward: How we handle Users / Groups / Circles - #4493

This includes a whole lot of other things (federation of contacts, own profile etc), so this here is one part of it. Probably that issue should be renamed. :)

@juliushaertl

This comment has been minimized.

Copy link
Member

commented Jan 11, 2019

Technically that’s what it does, but UX-wise it’s really similar. That’s why I think we need to coordinate this here and that one, so we don’t end up in 2 different concepts again. :)

Well, the approach this PR describes is more a generalization of our model of sharing entities (file/calendar) to a user/group/userdefined-group, while #11015 is about combining related entities. Yes, the linking will have the effect of allowing users to access those entities as well, but the main concept is about grouping related elements.

Does it make most sense to talk about it more at the Contributor Week in person? Or is that too late, and we should rather have another (or two) calls about this?

This issue is something we won't be able to tackle for 16 I think, regarding #11015 we already discussed that at the last hackweek and the concept is different from sharing, so I see no collision there.

@daita

This comment has been minimized.

Copy link
Member

commented Apr 16, 2019

Finally, I found a way to do something nice: IEntitiesManager !

While this is a group management solution, we will take the file sharing as an example for a better overview.
When you share a file, you share to an entity, could it be:

  • a single local user,
  • a single mail address,
  • a group of multiple local users and/or mail address.

(we can imagine a list of other types like federated cloud, federated users, ...)

Now, let's talk about the databases
Please note that this is the general structure, some fields might pop up before and during the development.

table 'entities'

This table will be the core of any request regarding a user or a group of users.

- id             string    - a unique uuid to ident the entity
- owner_id       string    - id from 'entitied_accounts' for the owner
- type           string    - type of the entity: no_member, unique, group, admin_group
- visibility     small int - visible to all, visible to owner only, visible to members
- access         small int - free, invite only, request needed
- name           string    - name of the entity
- creation       datetime
  • 'no_member' means that the entity contains no members at all: the owner is the entity. As an example, when you share your file to a local user, you're sharing to the 'no_member' entity that have the local user as owner_id
  • 'unique' means that the entity will have a unique member: this unique member is the entity and the owner is the one that can use this entity. As an example, when you share to a mail address, you're sharing to the 'unique' entity that have your userId as owner_id and the mail address the unique member of the entity (see the table 'entities_members') and 'name' should be the mail address of that unique member.
  • 'group' means that the entity is a group of multiple accounts. works like 'unique' but is not limited to a single account and the 'name' will be editable.
  • 'admin_group' is a 'group' with admin rights on the Nextcloud.

Uniqueness is not managed by the database.

table 'entities_accounts'

This table contains all 'accounts' that can be added to an entity: Local Users and Mail Addresses

- id             string    - a unique uuid to ident the account
- type           string    - local_user, mail_address, guest_user
- account        string    - account/user_id
- creation       datetime

'id' is linked to 'entities.owner_id' or to 'entities_members.account_id'.

  • 'local_user' means it is a local account identified by the userId
  • 'mail_address' is a mail address.
  • 'federated_user' and 'federated_cloud' can be one of the possible entries as 'type'.

Uniqueness is managed by the database using 'type' and 'account', meaning that the same mail address can be used by different owner in the 'entities' table. Of course, when sharing to a mail address, we provide completion based on the 'entities.owner_id', meaning that we only returns the known mail address if it was previously used by the user.

table 'entities_members'

This table will help to link 'entities_accounts' to 'entities'

- id             string    - a unique uuid
- entity_id      string    - id from 'entities'
- account_id     string    - id from 'entities_accounts'
- status         string    - invited, requesting, member
- level          small int - 1=member, 4=moderator, 8=admin  
- creation       datetime

'entity_id' is linked to 'entities.id'
'account_id' is linked to 'entities_accounts.id'

  • Status and Level will help manage the current rights in the group for each members: like in Circles, moderator can add/remove users, admin can add/remove moderators.

table 'entities_types'

This table should returns interface and class used by some of the types in the previously defined table. This looks like a descent way to have some extension to the whole system.

- id             int       - incremented and primary key, nothing more
- type           string    - string that define the type
- interface      string    - type of the type (sic)
- class          string    - class to be called to manage the service

This table needs some improvement during the developement of the project, my guess is that the 'entities_accounts.type' will be the first field to use this table with type='local_user', interface='IEntitiesAccounts', class='OC\Entities\Accounts\LocalUser::class'

Please provide feedback and comments.

@MorrisJobke

This comment has been minimized.

Copy link
Member Author

commented Apr 16, 2019

Looks good so far. I haven't re-assembled all details in my head fully, but roughly it seem to be what we discussed in the past.

Maybe @nickvergessen @kesselb @blizzz @rullzer @ChristophWurst @skjnldsv @juliushaertl also want to check this out, because that is what we want to replace groups, circles and stuff like that with.

@nickvergessen

This comment has been minimized.

Copy link
Member

commented Apr 17, 2019

My biggest question for talk is how we should use this.
This needs to be circular proof:

  • A talk conversation should be an entity, because you can share files into a conversation
  • We want to be able to add entities to a conversation, so you can invite e.g. a group, email, circle, ... into a conversation.
  • But at the same time we need to be able to detect if a user is removed from a conversation (while staying in the original entity that was invited) etc.

Hope you can put those thoughts into the thingy as well :)

@MorrisJobke

This comment has been minimized.

Copy link
Member Author

commented Apr 17, 2019

  • But at the same time we need to be able to detect if a user is removed from a conversation (while staying in the original entity that was invited) etc.

That's quite interesting ... because it would mean that it changes either the original entity or creates a copy without this one user. 🤔

@daita

This comment has been minimized.

Copy link
Member

commented Apr 17, 2019

@nickvergessen : About adding entites, this could be done in 2 ways:

  • In a sharing approach, I was thinking to add a 2-level (and non-recursive) way to group groups. The idea is that you could add an slave entity as part of a master entity, without being too heavy on resources.

  • In a grouping approach, a better approach would be to create 1-way link between entities.

Not sure there is a real difference from a user point of view, I need to think about the back-end perspective.

@MorrisJobke : We could use the status field in the entities_members database, even better if a sql request on bit flag value is doable. If not, we can have a text field to store that kind of data in a json that will be stored by the EntityManager, but used by the Talk app. A third solution would be that the Talk app manage the in/out conversation flag using the uuid of the entities_member (which is unique per user per group)

Both are just quick answers!

@blizzz

This comment has been minimized.

Copy link
Member

commented Apr 17, 2019

GroupAdmins

If I understand it correctly and want to apply it to group admins:

  • the target entity is of type group
  • regular admins have ownership built-in, so they don't need to be in owner_id
  • a "virtual" entity of type admin_group is created and all group admins (each entity no_member) are members of it
  • this virtual groupadmin entity is the owner of the target entity

That's a bit cumbersome. To have n owners another relational table could be used.

External users and groups

If I understand correctly, backends that might bring users and groups from somewhere else like LDAP and SAML do register theirs as a type in "entities_types"?

Since, right now there is one service for all users and another for all groups, so duplicating them with "local_users", "ldap_users", "saml_users", …, is perhaps not necessary, unless there is a reason for having it like that?

@daita

This comment has been minimized.

Copy link
Member

commented Apr 18, 2019

@blizzz

GroupAdmins are just a group member with Admin rights on the current group, right ? This should be managed by the IEntityManager itself when the admin level is assigned to a member.
The 'admin_group' type of an entity are groups that provide Nextcloud Admin rights to its members. Like the 'admin' group in today's Nextcloud.
Users within an admin_group also have Admin Level in all 'group' entities.

External users and groups wont need any specific stuff.

Edit: I now understand the misunderstood: 'local_user' is 'local' as from the Nextcloud. the way it authed is not important.

@skjnldsv

This comment has been minimized.

Copy link
Member

commented Apr 18, 2019

Can we do a table with different examples so I can properly get a grasp on it? It's a tad complex! :)
Please edit the following one ?
@daita

Shares entity_type account_type ... ...
User share unique local_user
Mail share unique mail_address
Talk file share unique? ??
Talk conversation share unique? ??
Circle share group? ??
Public share unique? guest_user
Group share group? ??
(Contact share)?
...
@nickvergessen

This comment has been minimized.

Copy link
Member

commented Apr 18, 2019

Mind explaining Talk thread share ? Do you mean a file shared into a Talk conversation? Or a Talk conversation in itself?

@skjnldsv

This comment has been minimized.

Copy link
Member

commented Apr 18, 2019

Yeah, conversation was too long, I was mixed between lazyness and it being too wide for the table 😉
I added the differentiation of the two in the table

@blizzz

This comment has been minimized.

Copy link
Member

commented Apr 18, 2019

@daita you're right, i missed that field in the members table, sorry.

External users and groups wont need any specific stuff.

But then 'local_users' is also unnecessary?

@daita

This comment has been minimized.

Copy link
Member

commented Apr 18, 2019

* We want to be able to add entities to a conversation, so you can invite e.g. a group, email, circle, ... into a conversation.

* But at the same time we need to be able to detect if a user is removed from a conversation (while staying in the original entity that was invited) etc.

So !

First, on install of the Talk app, we have a new entry in the entities_types:

type='room'
interface='IEntity'
class='\OCA\Talk\Entity'

When you create a new talk room, it creates a new entity in the 'entities' table :

id='1a2b3c'
owner_id='9a8b7c'
type='room'
visibility=(not relevant right now
access='invite only'
name='your room name'

For each entity (user or group) you want to include in your room

entity_id='1a2b3c'
account_id='[a...f0-9]' (in case of an account)
slave_entity_id=[a...f0-9]' (in case of an group)
status='member'
level=1

The Talk app would also need its own table (talk_left_convo) to manage people leaving the conversation, like this:

entity_id='1a2b3c'
account_id='[a...f0-9]'
left_conversation=true

Now, IEntity would be structured with methods that would returns members but also IQueryBuilder.

protected method getMembersQueryBuilder() {
    $qb = $this->dbConnection->getQueryBuilder();
    $qb->select()->from();

[...]

    return $qb;
}

public method getMembers() {
    $qb = $this->getMembersQueryBuilder();
    $data = $qb->execute();
    
[...]

    return $users;
}

extending it would allow to rewrite the getMembers() method:

public method getMembers() {
    $qb = $this->getMembersQueryBuilder();
    $qb->select('tlc.left_conversation');
    $qb->leftJoin(
        'entities_members, 'talk_left_convo', 'tlc',
	$qb->expr()->eq('account_id', 'tlc.account_id')
    );
    $data = $qb->execute();
    
[...]   // for each entries, check the value of 'left_conversation'

    return $users;
}

Note: adding 'slave_entity_id' in entities_members to link group_entity within a group_entity (with a limit to a certain depth)

@daita

This comment has been minimized.

Copy link
Member

commented Apr 18, 2019

@blizzz : yes, sorry, I edited my answer half an hour ago about it:

I think I understand the misunderstood: 'local_user' is 'local' as from the instance of Nextcloud. the way it authed is not important. We could name it 'user'.

@daita

This comment has been minimized.

Copy link
Member

commented Apr 18, 2019

@skjnldsv you're right, let's do some case studies !

  • User shares :
    you are sharing to an entity with:

(entities) type='no_member'
(entities) owner_id='user account'
(entities) name='user display name'

  • Mail shares
    You are sharing to an entity with:

(entities) type='unique' :
(entities) owner_id='your account'
(entities) name='recipient mail address'
(entities_accounts) type='mail_address'
(entities_accounts) account 'recipient@example.net'
(entities_members) entity_id=(entities) id
(entities_members) account_id=(entities_accounts) id

  • Group shares
    You are sharing to an entity with:

(entities) type='group'
(entities) owner_id='owner of the group'
(entities) visibility='all'
(entities) access='limited'
(entities) name='name of the group'
(entities_accounts) type='local_user'
(entities_accounts {entry_1}) account 'userA'
(entities_accounts {entry_2}) account 'userB'
(entities_accounts {entry_3}) account 'userC'
(entities_members) entity_id=(entities) id
(entities_members {entry_1}) account_id=(entities_accounts) id
(entities_members {entry_2}) account_id=(entities_accounts) id
(entities_members {entry_3}) account_id=(entities_accounts) id

  • Personal Circles
    You are sharing to an entity with:

(entities) type='group'
(entities) owner_id='your account'
(entities) visibility='hidden'
(entities) access='limited'
(entities) name='name of the personal circle'
(entities_accounts {entry_1}) type='local_user'
(entities_accounts {entry_1}) account 'userA'
(entities_accounts {entry_2}) type='local_user'
(entities_accounts {entry_2}) account 'userB'
(entities_accounts {entry_3}) type='mail_address'
(entities_accounts {entry_3}) account 'userC@example.net'
(entities_members) entity_id=(entities) id
(entities_members {entry_1}) account_id=(entities_accounts) id
(entities_members {entry_2}) account_id=(entities_accounts) id
(entities_members {entry_3}) account_id=(entities_accounts) id

  • Public Circles
    You are sharing to an entity with:

(entities) type='group'
(entities) owner_id='owner of the circle'
(entities) visibility='all'
(entities) access='free'
(entities) name='name of the circle'
(entities_accounts {entry_1}) type='local_user'
(entities_accounts {entry_1}) account 'userA'
(entities_accounts {entry_2}) type='local_user'
(entities_accounts {entry_2}) account 'userB'
(entities_accounts {entry_3}) type='mail_address'
(entities_accounts {entry_3}) account 'userC@example.net'
(entities_members) entity_id=(entities) id
(entities_members {entry_1}) account_id=(entities_accounts) id
(entities_members {entry_2}) account_id=(entities_accounts) id
(entities_members {entry_3}) account_id=(entities_accounts) id

  • Private Circles
    You are sharing to an entity with:

(entities) type='group'
(entities) owner_id='owner of the circle'
(entities) visibility='all'
(entities) access='request_needed'
(entities) name='name of the circle'
(entities_accounts {entry_1}) type='local_user'
(entities_accounts {entry_1}) account 'userA'
(entities_accounts {entry_2}) type='local_user'
(entities_accounts {entry_2}) account 'userB'
(entities_accounts {entry_3}) type='mail_address'
(entities_accounts {entry_3}) account 'userC@example.net'
(entities_members) entity_id=(entities) id
(entities_members {entry_1}) account_id=(entities_accounts) id
(entities_members {entry_2}) account_id=(entities_accounts) id
(entities_members {entry_3}) account_id=(entities_accounts) id

  • Hidden Circles
    You are sharing to an entity with:

(entities) type='group'
(entities) owner_id='owner of the circle'
(entities) visibility='members_only'
(entities) access='all'
(entities) name='name of the circle'
(entities_accounts {entry_1}) type='local_user'
(entities_accounts {entry_1}) account 'userA'
(entities_accounts {entry_2}) type='local_user'
(entities_accounts {entry_2}) account 'userB'
(entities_accounts {entry_3}) type='mail_address'
(entities_accounts {entry_3}) account 'userC@example.net'
(entities_members) entity_id=(entities) id
(entities_members {entry_1}) account_id=(entities_accounts) id
(entities_members {entry_2}) account_id=(entities_accounts) id
(entities_members {entry_3}) account_id=(entities_accounts) id

  • Talk Discussion
    You are sharing to an entity with:

(entities) type='room'
(entities) owner_id='owner of the room'
(entities) visibility='members_only'
(entities) access='invite_only'
(entities) name='name of the room'
(entities_accounts {entry_1}) type='local_user'
(entities_accounts {entry_1}) account 'userA'
(entities_accounts {entry_2}) account 'userB'
(entities_accounts {entry_3}) account 'userC'
(entities_members) entity_id=(entities) id
(entities_members {entry_1}) account_id=(entities_accounts) id
(entities_members {entry_2}) account_id=(entities_accounts) id
(entities_members {entry_3}) account_id=(entities_accounts) id

Note: visibility/access can be changed if you want anyone to be able to join your conversation.

  • Public Share (wip)
  • Contact Share (wip)
@skjnldsv

This comment has been minimized.

Copy link
Member

commented Apr 18, 2019

@daita can't you use a table please? This is not very much understandable ^^

@daita

This comment has been minimized.

Copy link
Member

commented Apr 18, 2019

it isunderstandable from the sql point of view !

anyway userA is the user sharing stuff:

Type of Shares entity.type entity.owner_id entity.visibility entity.access entity_accounts.type entity_accounts.account
User - Sharing to userB no_member userB
Mail - Sharing to user@example.net unique userA mail_address user@example.net
Group - Sharing to GroupA group OwnerA all limited local_user userId
Personal Circle - Sharing to CircleA group UserA hidden limited local_user / mail_address userId or mail address
Public Circle - Sharing to CircleB group UserB all free local_user / mail_address userId or mail address
Private Circle - Sharing to CircleC group UserC all request_needed local_user / mail_address userId or mail address
Hidden Circle - Sharing to CircleD group UserD members_only free local_user / mail_address userId or mail address
Talk - Sharing/Talking to RoomA room (extends group) OwnerA * * local_user userId
@skjnldsv

This comment has been minimized.

Copy link
Member

commented Apr 18, 2019

it isunderstandable from the sql point of view !

But I'm not a sql expert ;)
Thanks for the tabel!! 🤗

@sunjam

This comment has been minimized.

Copy link

commented Apr 25, 2019

  • Comments. I would love file and folder Comments to be treated as part of a project. This would greatly increase their usefulness, and would probably rely on ironing out comment federation as well.
  • Social ActivityPub would also be useful if you incorporate your Mastodon comments or add followers into a project.
@fwolfst

This comment has been minimized.

Copy link

commented Apr 25, 2019

Just a very quick feedback from an outsider perspective. We all know, naming is hard. To me it seems that Entity might not be a good name (it is as good as Object would be). I dont want to propose something better, as I am not involved and definately do not want to inspire to move away from the focused constructive discussion in this issue, but after reading this, I couldnt refrain from raising concerns about the name (Entity).

@daita

This comment has been minimized.

Copy link
Member

commented May 3, 2019

@fwolfst you'll have to come with some better names :-)

@fwolfst

This comment has been minimized.

Copy link

commented May 7, 2019

some better names :-)

ShareEntity ShareGroup SharingEntity Recipients RecipientEntity SharePool - but probably there are even better ones :)

@daita

This comment has been minimized.

Copy link
Member

commented May 28, 2019

rebased, squashed, documented

@MorrisJobke any chance you would have a look to the current status of the project ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
9 participants
You can’t perform that action at this time.