Tool needed: Matrix Admin - mass invite to parity channels during onboarding

[lovelaced]

Describe your problem and idea for the tool

A matrix "moderation" bot which ensures correct room permissions, manages user power levels, access control (ACLs), etc for internal rooms.

How do you solve this problem now?

The PeopleOps team manually invites users to rooms when onboarded; Arnold and I have the keys for the r00t account. There is no automation here.

Importance and impact

• Low. Would be nice to have it at some point in the future
• Medium. Lack of this instrument really prevents some people from working effectively
• High. If we don't make it by the deadline — we're doomed!

Deadline

Date, month or event reference to understand urgency.

Additional info

There could be some more interesting things done regards to this, like having the bot have an internal concept of 'teams' and integrating paritytech/opstooling#13 per-team, or something like that. Lots of possibilities. We should maybe ask the general company (matrix users) to weigh in on this, as I'm sure there has been plenty of pain.

[lovelaced]

I've rated this as a "medium" since this has been an ongoing issue for quite some time (ref https://github.com/paritytech/devops/issues/421 ), especially in regards to onboarding users. I have a list of channels all users should join when they're first onboarded to the server (we don't want all users created to be autojoined to rooms, because this gets really annoying when creating bot accounts - a command issued to the bot by a bot admin to specify to join a user to a set of rooms, for example, would be a good way to do this).

[joao-paulo-parity]

Below is an excerpt from a Matrix discussion about the use of r00t. It might add some insight to what this bot would be trying to solve.

Kirill: (By the way, dvdplm, you'd probably need to add @r00t into this room as admin when you're back, so we won't hit the same roadblock ever again with this room)

David: how do I do that? /invite @root doesn't work, nor does "root" show up in the invite users GUI.

Erin: //invite @r00t:matrix.parity.io

David invited r00t

Erin: someone has to log into it and accept the invite so it'll join when that happens

Erin: (If you're curious, the "matrix way" to do this would be to run it as an actively listening bot which autojoins channels if invited and keeps a list of authorized users which can control it via private message rather than having a shared password)

Source: https://matrix.to/#/!HqjQmVCvsgiLfZigmJ:matrix.parity.io/$KCZI6tHoNmd3yeh9Q1w46q4Asn_jlhNeUkNPqYG-sso

[lovelaced]

I feel like the amount of effort this would take to implement for base-level features like onboarding would save a LOT of people ops time in the coming months given the pace of hiring.

[lovelaced]

Here's a loose example of a bot which manages room invites and a bunch of other things: https://github.com/gracchus163/hopeless - for reference this bot was created by a couple people who barely ever program in less than a week of spare hours/volunteer time and was deployed to manage over 1000 users with 100% success.

it's based on this template which already supports everything Matrix and only needs the logic itself implemented: https://github.com/anoadragon453/nio-template

[helenekranz]

Adding some context on an "extra issue" we've been facing for the last few months:

When we log in to R00t to add a new person to all the core channels, we go into each of those 12 channels (listed here - there are 13, but people get added to "Substrate" automatically already) and add the person. For a while now we face the issue that after adding them to 5 rooms, adding them to the 6th room gets very slow (takes several minutes). This happens to both Hela and myself and it always happens after 5 rooms, even if we change the order of the rooms we add the person to.

This considerably slows down the whole process, since we can only continue adding the person to the remaining rooms after that current invite is done. The process being automated would save us a lot of time, especially since we now onboard at a high volume, which will only increase further. If automating the whole process can't be done quickly, we would at least need to get rid of the lag if at all possible, since that really slows things down.

[lovelaced]

Can we change the "impact" of this issue accordingly? :D

[Vovke]

@lovelaced incremented the impact accordingly, thanks for pointing out

[helenekranz]

Another data point from our end: On Monday I looked at the time it took me to add one person to all the rooms. It took me 8 minutes to add Jen (with some room invites going through in a second, others taking several minutes).

[joao-paulo-parity]

Now there's a new "Spaces" feature in our Matrix instance. Can the current pain be lessened for the time being if we grouped all the auto-join channels in a single space and asked newcomers to join them by themselves? Or maybe I'm misunderstanding how Spaces work and this is unfeasible, sorry.

[lovelaced]

@joao-paulo-parity Yes indeed this would solve a lot of our woes; however the room overlord is still needed for automation of room creation, cascading ACLs (fro example, adminning someone like Gav in the main Parity space and he becomes admin in all sub-spaces) and other more subtle things like that.

Using spaces is indeed something that will fix the specific issue of discoverability and inviting people upon onboarding but there's a bunch of other stuff we need as well. I'll have a better sense of requirements very soon.

[lovelaced]

Also keep in mind that every single room we want to behave in a space-native way will have to be upgraded to room version 9 (most are only on version 5 at the moment) so we would likely need some automation to upgrade all rooms. @fevo1971 knows more about this process but we'll need to settle on something as a company at large before doing so as it's a disruptive process.

[lovelaced]

somewhat related https://github.com/paritytech/devops/issues/1268

[helenco]

Hi team,
Can I ask for a status update on this?
To reiterate it would be very helpful and impactful for People Ops if this can be implemented. Considering the effort and time goes into this with the current volume of hiring, average of ca. 24 people monthly in the last 3 months and in the upcoming months this won't slow down.

These are currently the 12 core channels that People Ops currently adds people to manually (excl. Substrate and Parity Forum Notifications): Element Page on Notion

Thanks!

[lovelaced]

@helenco I am in contact with the Element people and we are currently working on a channel/Space resdesign for the whole of the company, which this problem will be worked into as a high priority item. We are unsure yet whether that will take the form of a management bot or just using Spaces to their highest potential.

[helenco]

@lovelaced Thanks for your prompt reply! I really appreciate this being worked into as a high priority item. Much thanks, Helen.

[koenw]

I've lately been playing around with a matrix bot and thinking about possible approaches a management bot could take here. I'll share some thoughts here in case they're helpful in closing in on a solution.

I see two general approaches: an 'imperative' approach where People Ops would tell the bot to do certain mutations (e.g. 'Add the following users to all default channels') or alternatively a more fully automated 'declarative' approach where the bot would periodically (and/or when triggered) make sure room membership is as it should be (which could mean some mutations made by users themselves would be undone if we want).

I think when correctly configured the declarative approach would result in least manual labor for People Ops. We would essentially have an authoritative Matrix configuration somewhere (I imagine something like a mapping of users/teams to rooms) and People Ops would only have to update the configuration. We might even be able to re-use existing administration of this kind like retrieving the list of teams/users from bamboo resulting in even less work for People Ops.

Alternatively a more imperative approach could be relatively simple to implement. A bot where you would just tell the bot to 'add the following users to the following rooms' would (based on @helenekranz' comment) already save People Ops quite some time. Because more manual labor is required in this case (e.g. typing out the list of people) there is also a higher chance of human error. We could depending on what fits with People Ops' processes also combine this imperative approach with some declarative configuration, where you could for example tell the bot to 'Add all users beginning this week to all default rooms'.

@helenco @lovelaced Do we already have an 'authoritative' list of matrix rooms and/or users somewhere that we could re-use? @helenco Would People Ops have a preference for how they'd like to interact with the bot (i.e. imperative, declarative or a particular mix)?

[lovelaced]

Thank you for your thoughts @koenw and I agree this is a great way to start. I think a declarative approach here would be very good, and doing it right the first time could allow us to do much more powerful things down the line. In my head, I had an idea of a bot which had a list of admins (people ops) who could then use basic commands in order to add people to $ALL_PARITY_ROOMS, $TEAM, and based on $TEAM they would also be added to rooms in $ENGINEERING_COMMON and $TEAM_ROOMS or something like that. The bot could work on a basis of "spidering" Spaces, so the bot could dynamically retrieve all rooms currently in a given space, and the person could then be invited to them without having to explicitly declare every single room. The management bot would have to be present in all rooms and spaces and also have admin (room admin, not server admin) privs. Based on other variables we could also then set Admin/Mod rights etc throughout the entire organization in a declarative way which would be amazing.

Sorry for the disorganized thoughts on my part, we should have a chat about this.

[lovelaced]

https://github.com/devture/matrix-corporal also exists as a reference but I think this is way overkill/corporate.

[lovelaced]

OK, I don't know when the Matrix people are going to get back to us so I'm going to say let's go ahead and start on this if possible. I'm happy to sit down with whomever and discuss our requirements, any questions about Matrix, and the best way to make something like this future-proof, but with appropriate scope (as it will probably develop into more than just a room-manager for people).

[Vovke]

Happy this got unblocked, @mordamax please work with Erin on the requirements part

[lovelaced]

Templates (both by core matrix team people):
https://github.com/turt2live/matrix-bot-sdk-bot-template

https://github.com/anoadragon453/nio-template

Some extra thoughts: you probably don't actually want to use the Synapse admin API if you can help it, because it's safer to not have a bot account which has server admin privs and I don't think it's actually needed since the bot is only performing user actions for now (inviting/chatting).

[lovelaced]

We should also use a supported cloud db for this (postgres or MySQL) so we can easily back up the current state.