Users with Role Admin cannot login (Ubuntu snap)

[hackbard-sealion]

Description:

User with Role Admin cannot login to self hosted rocketchat instance. The famous three dots are shown and browser goes into death loop until showing out of memory message

Steps to reproduce:

1. Make User Admin
2. Refresh /home
3. Does not Work

Expected behavior:

Normal Login

Actual behavior:

Server Setup Information:

• Version of Rocket.Chat Server: 6.6.2
• Operating System: Rocky Linux release 8.9 (Green Obsidian)
• Deployment Method: snap
• Number of Running Instances: 1
• DB Replicaset Oplog:
• NodeJS Version: v14.21.2
• MongoDB Version: 5.0.15 / wiredTiger (oplog Enabled)

Client Setup Information

• Desktop App or Browser Version: App 3.9.14 Browser Firefox 122 / Edge 122.0.2365.66
• Operating System: WIndows 11

Additional context

if i use
db.users.update({username: "admin_name"}, { $pull: { roles: "admin"}})

the user can immediatly access rocketchat after ctrl+r

Relevant logs:

Cannot enter Workspace, cannot provide Logs.

Thank you for helping!

[luizjr]

I'm having exactly the same problem.
I noticed that when I remove the user's admin ROLE the interface loads normally for him

[luizjr]

@hackbard-sealion Do you currently have an active message removal policy?

[hackbard-sealion]

Hi @luizjr,

You mean this? https://docs.rocket.chat/use-rocket.chat/workspace-administration/settings/retention-policy

I don't think so, but since i can not login as an admin, i can not verify at this point...

[luizjr]

Hi @luizjr,

You mean this? https://docs.rocket.chat/use-rocket.chat/workspace-administration/settings/retention-policy

I don't think so, but since i can not login as an admin, i can not verify at this point...

Yes, that was right, I enabled it 4 days before these problems started. But there was an update too!

[luizjr]

Hello, friend @hackbard-sealion, I have news, I tried a lot looking for solutions.

I ended up getting to one alone:
Admins usually receive banners (alerts) I made a drop in the tables:

• rocketchat_banner
• rocketchat_banner_dismiss

Commands here below, via the mongo shell:

db.rocketchat_banner.drop()
db.rocketchat_banner_dismiss.drop()

And believe me, it did the trick for me.

[hackbard-sealion]

db.rocketchat_banner.drop()
db.rocketchat_banner_dismiss.drop()

Wow, thank you so much. that did the trick also for me! Actually i was on a similar path some days ago where i deletes the banner messages from the profiles, but that did not help. Dropping the complete table did it.

Thank you!

[danielfariati]

We are also affected by this bug... It was making RocketChat completely unusable (literally... browser crashes / infinite loading on the app) to anyone that had the admin role.

We solved it by forcing a fixed response with an empty array of banners for any request to /api/v1/banners.
So yeah, the problem is definitely caused by those banners that are being shown to admins.

[grzebo]

We solved it by forcing a fixed response with an empty array of banners for any request to /api/v1/banners. So yeah, the problem is definitely caused by those banners that are being shown to admins.

Can you please describe in more detail how to do it?

[danielfariati]

We solved it by forcing a fixed response with an empty array of banners for any request to /api/v1/banners. So yeah, the problem is definitely caused by those banners that are being shown to admins.

Can you please describe in more detail how to do it?

Our RocketChat instance is served from an AWS ELB, so we just included a listener rule to do that. Something like:

Rule 1:
- Priority: 1
- Conditions: HTTP Host Header is chat.myorg.com AND Path Pattern is /api/v1/banners
- Then: 
    - Response code: 200
    - Response body: {"banners":[]}
    - Response content type: application/json

Rule 2:
- Priority: 2
- Conditions: HTTP Host Header is chat.myorg.com
- Then: 
    - Forward to target group: rocketchat

But you can achieve the same result by adding Apache Httpd or Nginx server in front of your chat instance with a fixed response rule for that route.

You could also change the RocketChat API directly in the code, if you prefer.
In that case, you would have to change the code in this file: https://github.com/RocketChat/Rocket.Chat/blob/master/apps/meteor/app/api/server/v1/banners.ts

[kirayamatosf]

I am facing this issue too.
May I know how to do this?
I am using snap install rocketchat.
And run below command to get in mongodb-shell
rocketchat-server.mongo

However I cannot locate db.rocketchat_banner
Is there anything I missed ? Please help.

[grzebo]

You should open up mongo shell, and then:
show dbs
this will show you your databases

then switch to rocketchat:
use rocketchat

then this will show you your collections:
show collections

then you can check the contents of the banner collection:
db.getCollection("rocketchat_banner").find({})

and then you can set all the banners to "false":
db.rocketchat_banner.updateMany({ },{ $set: { active: false }})

You don't need to check everything on the way, just this two commands are enough:
use rocketchat
db.rocketchat_banner.updateMany({ },{ $set: { active: false }})

[kirayamatosf]

You should open up mongo shell, and then: show dbs this will show you your databases

then switch to rocketchat: use rocketchat

then this will show you your collections: show collections

then you can check the contents of the banner collection: db.getCollection("rocketchat_banner").find({})

and then you can set all the banners to "false": db.rocketchat_banner.updateMany({ },{ $set: { active: false }})

You don't need to check everything on the way, just this two commands are enough: use rocketchat db.rocketchat_banner.updateMany({ },{ $set: { active: false }})

Oh. Thanks a lot. I can get back in my admin page.
Can I ask one more thing?
I notice some of the private channel on iOS & Android App will be freeze cannot get in.
However on PC rocketchat client without problem.
Is there any solution for this? Thank you very much

[kirayamatosf]

You should open up mongo shell, and then: show dbs this will show you your databases
then switch to rocketchat: use rocketchat
then this will show you your collections: show collections
then you can check the contents of the banner collection: db.getCollection("rocketchat_banner").find({})
and then you can set all the banners to "false": db.rocketchat_banner.updateMany({ },{ $set: { active: false }})
You don't need to check everything on the way, just this two commands are enough: use rocketchat db.rocketchat_banner.updateMany({ },{ $set: { active: false }})

Oh. Thanks a lot. I can get back in my admin page. Can I ask one more thing? I notice some of the private channel on iOS & Android App will be freeze cannot get in. However on PC rocketchat client without problem. Is there any solution for this? Thank you very much

After update App today. seem fix my problem.
Thanks for your help !

[TLINDEN]

Is there a way to disable these messages on mobile devices?

[TLINDEN]

db.rocketchat_banner.updateMany({ },{ $set: { active: false }})

Doesn't work. After a restart, there are new active banner messages and the banner re-appears:

 "_id" : "65fb99c5edfe320bb8956cf5", "_updatedAt" : ISODate("2024-03-22T11:08:55.098Z"), "active" : true,[..]

[grzebo]

After a restart, there are new active banner messages

Yes, the banner messages are added back in after a restart and once every 12 hours.

There are probably many more elegant ways to do it, but I just added this to my crontab, so that every 10 minutes the banners are cleared out, in case they reappear:

/usr/bin/mongosh --eval 'use rocketchat' --eval 'db.rocketchat_banner.updateMany({active: true },{ $set: { active: false }})'

[bakaevmm]

We are experiencing the same issue. Administrators are unable to log in and are facing an infinite loading screen. The workaround described here helped us resolve it. It seems like a bug. How can we notify the Rocket.Chat team about the need for a fix?

[bakaevmm]

My RocketChat installation is deployed using Docker. The issue is relevant for us as well.

[sfood]

We have the same problem. Removing or disabling banners, it works!
Banners are restored after:

1. If you click on the "Sync license update" button in the "Subscription" section
2. Reboot the server.
3. Every 12 hours.

Another way is to change the schedule, instead of 12 hours, I set it to 1 year. If you don't reboot the server, then it works.

[bakaevmm]

I have a cron job for cloud workspace sync that runs every 12 hours, but the banners have been in the false state for several days now.
I executed the command - db.rocketchat_banner.updateMany({ },{ $set: { active: false }})

[grzebo]

For me they reappear every 12 hours, I log whenever they reappear. Only on March 28th I got just one reappearance instead of two.

Banners removed on:

• Tue Mar 26 00:02:29 CET 2024
• Tue Mar 26 12:00:16 CET 2024
• Wed Mar 27 00:02:18 CET 2024
• Wed Mar 27 12:01:12 CET 2024
• Thu Mar 28 00:01:10 CET 2024
• Fri Mar 29 00:01:06 CET 2024
• Fri Mar 29 12:01:11 CET 2024
• Sat Mar 30 00:01:07 CET 2024
• Sat Mar 30 12:00:07 CET 2024
• Sun Mar 31 00:00:11 CET 2024
• Sun Mar 31 12:01:03 CEST 2024
• Mon Apr 1 00:01:03 CEST 2024
• Mon Apr 1 12:00:06 CEST 2024
• Tue Apr 2 00:01:03 CEST 2024
• Tue Apr 2 12:00:06 CEST 2024

[reetp]

it's a bit hard to know what to say here as there seem to be multiple versions exhibiting issues so it is difficult to sort the wood from the trees.

If you have an issue we need to know the version, deployment method, licence type etc so please fill in a bug template properly rather than just saying 'me too'

Please make sure you test on the latest available version as well.

Thanks.

[github-actions]

This issue has been marked as stale because there has been no further activity in the last 10 days. If the issue remains stale for the next 4 days (a total of 14 days with no activity), then it will be assumed that the question has been resolved and the issue will be automatically closed.

[luizjr]

Still waiting for response.

[reetp]

Still waiting for response.

Still waiting for your specific information as I mentioned above.

There are several "metoo" comments but it needs specific info for each.

Also note some work on banners - which seems to be a theme here - lands in 6.10

#32018

[luizjr]

@reetp What I can say is that a notification arrives every day, but if I click close it doesn't close, I can only do it by going to mongodb and removing it by hand.

[reetp]

Again, see the other bug which may well fix it.

First RCs for 6.10 will be out any day - please test.

[github-actions]

This issue has been marked as stale because there has been no further activity in the last 10 days. If the issue remains stale for the next 4 days (a total of 14 days with no activity), then it will be assumed that the question has been resolved and the issue will be automatically closed.

[luizjr]

Still no solution

[reetp]

Still no solution

Still not related to this issue regarding logins so no there won't be a solution here.

See the previously mentioned bug which relates to your problem, and note that there are fixes in 6.10

Test on that, and comment on the correct issue.

Thanks.

[github-actions]

This issue has been marked as stale because there has been no further activity in the last 10 days. If the issue remains stale for the next 4 days (a total of 14 days with no activity), then it will be assumed that the question has been resolved and the issue will be automatically closed.

[MartinSchoeler]

Hey @hackbard-sealion @luizjr @bakaevmm when the client is infinitely loading do you get any errors on the browser console? Thanks!

[luizjr]

Hey @hackbard-sealion @luizjr @bakaevmm when the client is infinitely loading do you get any errors on the browser console? Thanks!

yes, this is the behavior, no error, nothing, it just crashes for anyone who is an admin.
It only works when I run the commands to clean the tables mentioned above, this problem still exists today

[MartinSchoeler]

I see, could you share the entries on the DB that causes issue, I think we might have some malformed data that is breaking the UI, I'll try to reproduce the issue locally with the DB entries.
Thanks!

@hackbard-sealion @luizjr @bakaevmm

[eisberg]

I have a similar problem, everything freezes on any clients, including iOS. Clearing banners in mongo helps, before the next update.
In the last update, the hangup has gone away, but it is impossible to close the banner about exceeding the limit of push notifications per month.

You click on the cross - the request goes away, but nothing happens, the banner remains.
In the rocketchat_banner_dismiss database, the number of records increases. There are several thousand records in rocketchat_banner, and their number does not change.

I completely clean rocketchat_banner and rocketchat_banner_dismiss with my hands, after which the banner disappears.

Until the next update...

[MartinSchoeler]

@eisberg

You click on the cross - the request goes away, but nothing happens, the banner remains.
In the rocketchat_banner_dismiss database, the number of records increases. There are several thousand records in rocketchat_banner, and their number does not change.

This should be fixed by #32552 are you in version 6.10?

have a similar problem, everything freezes on any clients, including iOS. Clearing banners in mongo helps, before the next update.

Could you share the records that causes the issue? Thanks!

[eisberg]

@MartinSchoeler

This should be fixed by #32552 are you in version 6.10?

Yes, 6.10.0 (Community), docker installation, MongoDB 6.0.13
The problem with banners was repeated during the last update ;(

I attached a log file, but I'm not sure if there is anything suitable there.
The banner table was cleared on 2024-07-20 06:28:31
rocketchat.log.zip

[Gustrb]

Hey @eisberg which banner is the one who can't be dismissed?
#32552 fixed the specific case for the banner that would show when one was close to the limit of members, if there are different banners that are causing the same issue I guess we need to look at them case-by-case

[eisberg]

"You have reached your monthly mobile push limit"

[MartinSchoeler]

Hey, just an update, I could not reproduce the issue locally, I will probably need a sample of the database entry that is causing the issue. If anyone wants to help, I believe you can force a recheck by asking for a license sync on the subscription admin page

And if you don't want to share your db entries on a public forum, feel free to dm me at https://open.rocket.chat/direct/martin.schoeler

Many thanks!

[github-actions]

This issue has been marked as stale because there has been no further activity in the last 10 days. If the issue remains stale for the next 4 days (a total of 14 days with no activity), then it will be assumed that the question has been resolved and the issue will be automatically closed.

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.

[KodoGit]

And if you don't want to share your db entries on a public forum, feel free to dm me at https://open.rocket.chat/direct/martin.schoeler

Hello Martin. Unfortunately that direct chat says that no such room. Looks like permission problem.
I would like to help to solve the problem.