Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support multiple orgs on the same instance #658

Open
engelgabriel opened this issue Sep 1, 2015 · 147 comments

Comments

Projects
None yet
@engelgabriel
Copy link
Member

commented Sep 1, 2015

Transparently divide a single server into several different instances shared between different groups of users.

Evaluate
https://atmospherejs.com/mizzao/partitioner

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

@engelgabriel engelgabriel added this to the v1.1 milestone Sep 1, 2015

@mhurwi

This comment has been minimized.

Copy link
Contributor

commented Sep 13, 2015

Hi! We might be interested in helping on this one.

Is it to be the same 'teams' feature as on Slack? Has anyone started on it yet?

@Sing-Li

This comment has been minimized.

Copy link
Member

commented Sep 13, 2015

Cool. Please join the #contributors channel on the demo chat. Can definitely use the help, especially if you guys have experience in working with mizzao:partioner already!

@mhurwi

This comment has been minimized.

Copy link
Contributor

commented Sep 13, 2015

Not used it, but reading the docs and discussion, seems like a nice solution. Will see you in the chat room soon!

@Sing-Li

This comment has been minimized.

Copy link
Member

commented Sep 13, 2015

Great, @mhurwi ! Catch you on demo soon. Can't wait to see what sort of real world performance hit partitioning brings. Day job's got me running 'round in circles on a Sunday 😀

@engelgabriel

This comment has been minimized.

Copy link
Member Author

commented Sep 13, 2015

THe good news is, I just talked to 2 companies that want this feature so bad, they will allocate their own developers to contribute and build it :)

One of the companies if from around the block, so they will just come to the office for a few days and work from here.

@geekgonecrazy

This comment has been minimized.

Copy link
Member

commented Sep 13, 2015

👍 this is awesome

@Sing-Li

This comment has been minimized.

Copy link
Member

commented Sep 13, 2015

Haha what a team! @mhurwi , you just got yourself some teamates from the real sunny south! 😀

@liuliming2008

This comment has been minimized.

Copy link

commented Sep 14, 2015

Partition is the same as "team" at slack?
At slack, one user(email)can create and join multiple teams, can can switch team.
If Partition is only a solution like load balance, rocket cloud add team feature at one instance first.

@mhurwi

This comment has been minimized.

Copy link
Contributor

commented Sep 14, 2015

One thing about mizzao:meteor-partitioner: it seems set up to allow one user to belong to only one group.

https://github.com/mizzao/meteor-partitioner/blob/master/grouping.coffee#L19-L23

Partitioner.setUserGroup = (userId, groupId) ->
  check(userId, String)
  check(groupId, String)
  if Grouping.findOne(userId)
    throw new Meteor.Error(403, "User is already in a group")

This makes sense, compared to Slack. From my experience, Slack requires a unique new password for each team. That could mean that each team represents a new instance and behind the scenes, on Slack I have a separate account for each team I belong to.

Maybe using meteor-partitioner would require creating a new account for a user when they want to join a new group?

Not sure what you all think about this?

@Sing-Li

This comment has been minimized.

Copy link
Member

commented Sep 14, 2015

Just copying a post from the other thread (use a context id):

Can we 'virtualise' the real email + an appearance id ... and then gen a unique guid email to be used by meteor?? just a thought

the appearance id might just be a 'context identifier', from which white-labeled instance (or a team name) the user is logging in from (as), for example:

(singli@hisemail.com + 'rocketchat team') -> kskfjsafosajdfasf@dsalfsakfaslf.com
(singli@hisemail.com + 'sings own family chat') -> o24i3i435o35345@oi324323432.com

@engelgabriel

This comment has been minimized.

Copy link
Member Author

commented Sep 14, 2015

I think we should look into mizzao:partioner, but not use it, just understand it and apply some of the logic into our project. The main reason being that we want this behaviour to be optional. Some organization will just want a very simple Rocket.Chat deployment, for a single team. So we need to add the hooks on the right places, and only activate them via settings.

@liuliming2008

This comment has been minimized.

Copy link

commented Sep 15, 2015

I suggest to add the "team" feature as optional settings.
And then consider how to expand one instance to more.

@geekgonecrazy

This comment has been minimized.

Copy link
Member

commented Sep 15, 2015

@liuliming2008 that's a completely separate thing and is being tracked here. #520

High Availability / Scaling to multiple instances.

@engelgabriel

This comment has been minimized.

Copy link
Member Author

commented Oct 13, 2015

Lets implement that using the new abstraction layer at
Rocket.Chat/packages/rocketchat-lib/server/models/_Base.coffee

@engelgabriel

This comment has been minimized.

Copy link
Member Author

commented Oct 13, 2015

Althou we will use the word "TEAMS" to denominate the enviroment variable, the property name on the schema should be configurable, so we can keep compatibility with https://atmospherejs.com/mizzao/partitioner as they use _groupId

@joaocarloscabral

This comment has been minimized.

Copy link

commented Oct 14, 2015

Hello guys.
I'll work on this issue on the next days!
As @engelgabriel said before, we have a product that needs this feature and will be a pleasure for me to help.

Yesterday I was talking with @engelgabriel and @rodrigok about the first steps to develop and the suggestion was to start changing this file: https://github.com/joaocarloscabral/Rocket.Chat/blob/master/packages/rocketchat-lib/server/models/_Base.coffee

The idea is intercept all calls to DB, add a property named as _teamId and check if the property was not changed from the client.

Right? :)

@engelgabriel

This comment has been minimized.

Copy link
Member Author

commented Oct 14, 2015

@joaocarloscabral Take a look on the link below for how to grab the connection object for that fiber.
https://github.com/RocketChat/Rocket.Chat/compare/EnvironmentConnection#diff-e214084ef39ef04ea8cfce4bd72a420bR26

@geekgonecrazy

This comment has been minimized.

Copy link
Member

commented Oct 15, 2015

@engelgabriel when I think teams i'm thinking of an organizational unit. A way to say this user is part of this team, this channel is part of this team. A way of visually organizing and displaying things. But essentially its still all in the same organization, and same instance and they could directly interact with other teams.

Like Company X has:

  • Sales Team
  • Marketing Team
  • Engineering Team

Etc, but they can still talk in general shared channels.

When I read through this thread, this looks like you guys are talking multi-organizational support? Or partitioning of a single server to be able to take care of multiple organizations. Completely separate but on the same instance.

I'm I correct in thinking this is actually multi-organization support not really teams?

@mhurwi

This comment has been minimized.

Copy link
Contributor

commented Oct 20, 2015

We are in favor of 'multi-organization'. We use slack this way-- each team is a different customer organization that we manage. These different organizations do not talk to each other.

'sales team' and 'marketing team' seem like people that belong to the same organization. They can use private channels to stay separate, but they can still direct message whoever they want and also join in the public channels.

@geekgonecrazy

This comment has been minimized.

Copy link
Member

commented Oct 20, 2015

@mhurwi right that makes sense that you would have the option to separate out to a multi-org if you want isolation between teams.

@engelgabriel I'm going to go ahead and change this title so we its a little more clear.

@geekgonecrazy geekgonecrazy changed the title Support multiple teams on the same instance Support multiple orgs on the same instance Oct 20, 2015

@liuliming2008

This comment has been minimized.

Copy link

commented Oct 23, 2015

hi @joaocarloscabral and @engelgabriel:
If do ti at Rocket.Chat/packages/rocketchat-lib/server/models/_Base.coffee, can not support one user create or join mutiple organizations like slack, right?

@GreatEmerald

This comment has been minimized.

Copy link

commented Mar 16, 2019

"Multi team per Org" is what I'd be looking for. Completely isolating instances can already be achieved by using several subdomains, as mentioned. But it would be a great feature to be able to group channels so that they are organised better, while still being able to talk to everyone. (You can work around this right now in Rocket.chat by using standardised prefixes, but they get rather long and cumbersome.)

Matrix seems to do well in that with the "community" concept. Mattermost "teams" are a bit worse in that there is no support for cross-team channels so far, so it's more isolating.

Probably ideal would be to have a hierarchy of grouped channels, so we could have general channels for everyone, channels for a particular organisational unit, and within it channels for a particular topic. Everyone could still access any channel but they can find them easier by following the hierarchy.

@Sing-Li

This comment has been minimized.

Copy link
Member

commented Mar 16, 2019

@GreatEmerald The new Threaded Discussions feature in release 1.0-BETA may already do what you need. Please take a look.

image

The UI (including nesting) is still evolving.

With Federation, cross-domain isolations can be relaxed in a selective and controlled manner.

@juliomac

This comment has been minimized.

Copy link

commented Mar 16, 2019

Hi @Sing-Li , thanks for the information. But I quite did not understand how this "threads" implementation relates to "multiple orgs on same instance". How would you think we could leverage this for that purpose? It would be excellent if that could solve it!

@Sing-Li

This comment has been minimized.

Copy link
Member

commented Mar 16, 2019

@juliomac That was a reply very specific to @GreatEmerald 's exploration.

Throughout this arduous journey of almost 4 years (since the start of this issue - and multiple public/private attempts to implement something usable), we've learnt the hard way that NOT EVERYONE has the same requirements in this area. And that a sizable population of the real requirements fall towards what @GreatEmerald suggests - and might be solvable with this approach.

@GreatEmerald

This comment has been minimized.

Copy link

commented Mar 16, 2019

Threads is a great feature that is a big added value to the platform. But threads are very short-lived, they're complementary with the concept of "communities"/"teams". Basically as far as I can tell their use is to structure short topics within a room, rather than to act as rooms within a group of rooms.

@Sing-Li

This comment has been minimized.

Copy link
Member

commented Mar 16, 2019

@GreatEmerald you've gotta try our implementation of Threads then.
Lifecycle is in your control. Can be as long lived as the server itself.
It is also absolutely rooms within room - that's how it is implemented, actually.
We've spent a lot of time working out how to make "threading" actually usable by a majority of users - and this particular implementation might do it.

@GreatEmerald

This comment has been minimized.

Copy link

commented Mar 16, 2019

Perhaps to make it a bit more visual, I'm thinking about a Rocket.Chat instance for a university. There are multiple departments or faculties; each department has multiple chair groups; each chair group organises a number of courses. Each course can have several rooms (e.g. #announcements and #questions-answers). Each room can have a big amount of threads, for instance a question on #questions-answers.

As it is, we have the lowest part of the hierarchy, and we could work around the lack of the higher levels of hierarchy with prefixes, but then the name of the #questions-answers room name would be #esg/grs/geoscripting2019/questions-answers, which is unwieldy and error-prone for misspelling.

@Sing-Li

This comment has been minimized.

Copy link
Member

commented Mar 16, 2019

@GreatEmerald just quoting the earlier post

"Multi team per Org" is what I'd be looking for. Completely isolating instances can already be achieved by using several subdomains, as mentioned. But it would be a great feature to be able to group channels so that they are organised better, while still being able to talk to everyone. (You can work around this right now in Rocket.chat by using standardised prefixes, but they get rather long and cumbersome.)

Not the entire namespace needs to be partitioned via threads. Perhaps, the use of isolated instances (subdomains) can help with the higher levels. Sharing, between these "hard partitioned" spaces, can possibly be relaxed in a controlled manner via Federation. (allows sharing of users and channels between instances).

It is only in the lower levels - that this "rooms within room" thread implementation will be useful.

The "too many threads", "unwieldy" problem might be solvable with some UI ingenuity.

@GreatEmerald

This comment has been minimized.

Copy link

commented Mar 16, 2019

In my case, the hard partitioning Slack style isn't desirable, since it would be bad for cross-organisation collaboration. Also, we have just one LDAP instance for the whole university.

But yes, I agree that support from the protocol level isn't necessary, UI trickery could as well work. But it would be convenient, because you want different people to be admins of the different groups. (Though I guess that's a separate issue.)

@torgeirl

This comment has been minimized.

Copy link
Contributor

commented Mar 16, 2019

@Sing-Li: this approach might be a solution for some, but it isn't comparable to Slack workspaces or Mattermost teams.

One scenario that warrants something comparable is when you have smaller teams/workgroups that use it more internally:

  • general channels becomes noise and steals UI real-estate.
  • it's hard to control something similar to team/Workspace admins (admins without sysadmin privileges) for that team/Workspace.
  • it becomes easy to post secrets in open channels by mistake
  • managing bots for that team/workspace gets messy
  • it might also become harder to make sure uploads related to that team gets deleted after its done (GDPR)

As mentioned earlier you can do this by running each team in containers, but that complicates other stuff again and is not a practice solution for many.

Please stop looking at halfway fixes, and rather focus this discussion around how Rocket.Chat could get its alternative to Slack workspaces /Mattermost teams. 🙏

@Sing-Li

This comment has been minimized.

Copy link
Member

commented Mar 16, 2019

Right. It is a very separate issue 🙁 This "thread" (pardon the pun) had split into two rather mutually exclusive prongs:

  1. those who wants MULTI-TENANCY on one server
  2. those who actually just needed some level of scoping of object visibility, accessibility and manageability within a very large server instance

Admittedly, some requirements overlap the two, But my exploration thus far, is completely addressing only a (good sized) subset of (2).

@GreatEmerald

This comment has been minimized.

Copy link

commented Mar 16, 2019

Yes, it's what @geekgonecrazy said, there's "Multi Org" and "Multi team per Org" ideas that are quite different in scope.

@juliomac

This comment has been minimized.

Copy link

commented Mar 17, 2019

Thanks @Sing-Li ! If I understood it well, threads is some sort of "conversation grouping" so people can relate to a specific subject within a larger channel, right? That's a fantastic improvement! However, like you said, it may not solve the 2 main requests you mentioned.

I am trying to achieve some sort of "multi-org scoping" for LiveChat, using departments. With each department being a different organization. So far, from what I am seeing, it almost completely solves the team/workspace/multi-tenacy request. But only works for LIveChat. In fact, the goal of departments was to get some scoping of visibility (your number 2) for departments.

I would say that the "department" solution could be further explored. Maybe just creating an "organization" field on top of department and include it on all messages/rooms/channels and control its access by roles, just like is done in livechat, could be just as fine. Going all the way with mizzao/partitioner seems daunting.

What do you think?

@juliomac

This comment has been minimized.

Copy link

commented Mar 17, 2019

@torgeirl

This comment has been minimized.

Copy link
Contributor

commented Mar 17, 2019

@juliomac: the URL is broken.

@Sing-Li

This comment has been minimized.

Copy link
Member

commented Mar 17, 2019

I would say that the "department" solution could be further explored. Maybe just creating an "organization" field on top of department and include it on all messages/rooms/channels and control its access by roles, just like is done in livechat, could be just as fine. Going all the way with mizzao/partitioner seems daunting.

@juliomac That's precisely the direction that we're exploring at the moment. A note to those who are on "Prong 1" though. That the number of departments we're working with will be 10s or low 100s at the limit.

@juliomac

This comment has been minimized.

Copy link

commented Mar 17, 2019

Hi @torgeirl , strange... it works for me.

Please test this: develop...leonardoterrao:develop

@juliomac

This comment has been minimized.

Copy link

commented Mar 17, 2019

Dear @Sing-Li , please let me know how could I help there. I have some experience with Meteor.

@mwells3work

This comment has been minimized.

Copy link

commented Mar 21, 2019

Is there a good branch that we could evaluate what has bee done so far with the multi-org development, as we have a need for this, we could devote some time to getting it working.
thanks

@juliomac

This comment has been minimized.

Copy link

commented Mar 22, 2019

Hi @mwells3work ! Also for me! I would gladly help. By the way, have you seen the branch I just posted above?

@mwells3work

This comment has been minimized.

Copy link

commented Mar 22, 2019

@juliomac I have not looked at it yet, If you think this is a good place to start, I will check it out and test the multi-org functionality. Thanks

@ggruber4711

This comment has been minimized.

Copy link

commented Apr 27, 2019

For implementing a multi-org feature, wouldn't a naive approach be to have:

  • organisations as new entity
  • obligation to link each user to an organisation, existing installations could be updated by a "core" organisation, where every existing user is automatically linked.
  • add functionality into core that users would only see messages, users etc. only from own organsation
  • add special role to allow cross-organisation communication/administration for sysadmin users

That would probably solve the most important requirements in the multi-org domain.

@ggruber4711

This comment has been minimized.

Copy link

commented Apr 28, 2019

A poor man's solution could also be:

  • add custom field 'organisation' to both user and room
  • implement a proxy around rocket.chat server which intercepts any request and will filter out users and rooms based on the organisation custom field.

Goal would be that a user, which has set the organisation custom field would only see users and rooms of the same org.

@mindey

This comment has been minimized.

Copy link

commented Jun 21, 2019

How to just disable listing of users in /directory, if they are not in the groups that the user is part of? That would solve us all the problems, without having to create any abstraction layers on top. It's solved by PR #7830 (Permission: view-outside-room).

@ggruber4711

This comment has been minimized.

Copy link

commented Jul 4, 2019

How to just disable listing of users in /directory, if they are not in the groups that the user is part of? That would solve us all the problems, without having to create any abstraction layers on top. It's solved by PR #7830 (Permission: view-outside-room).

OK, needed some time to understand this. This essentially means, that for the Role 'user' we would remove the view-outside-room permission.

Then for each organisation/customer which is added to the single Rocket.Chat instance, we would auto-create a #Customer-General channel, where all users of the customer would be added and add ther users to there 'general' channel.

If then a user of a customer searches for other users, he would need to use the company-specific general channel in order to find his colleagues. He would not be able to find colleague with the directory though :-(

In order for this scenario to work properly, I think we would need to change the directory in such a way, that at least all users of the rooms the current user is currently in are displayed. Essentially it means collecting all the rooms of the user and merging the other users of those rooms into one set and displaying it in the directory.

Created a new issue for that UX Problem #14930

@ggruber4711

This comment has been minimized.

Copy link

commented Jul 4, 2019

I did some more research and came up with a pretty good configuration scenario removing these permissions:

  • view-c-room (view channels)
  • view-outside-room (view users outside of room)
  • create-c (create channels)
  • create-p (create private rooms)

Now a user can only see members of the groups he is in and he cannot create new groups or channels.

Only 2 drawbacks:

  • Directory / search functionality does not show any user #14930
  • Dialog to start a discussion allows to reference a user outside of scope (who is not part of the referenced channels) #14931
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.