Update to v5.0.4 failing with indexes errors

[iamfasal]

Description:

We were running an outdated version v3.18.1 on our CentOS 7 server deployed manually. Today when we tried to update to the latest v5.0.4, the rocketchat service throwing below errors and getting failed when we start the service.

systemd[1]: Started The Rocket.Chat server.
rocketchat[32389]: Some indexes for collection 'rocketchat_integration_history' could not be created:
rocketchat[32389]: Index with name: _updatedAt_1 already exists with different options
rocketchat[32389]: Some indexes for collection 'rocketchat_nps_vote' could not be created:
rocketchat[32389]: Index with name: npsId_1_identifier_1 already exists with different options
systemd[1]: rocketchat.service: main process exited, code=killed, status=11/SEGV
systemd[1]: Unit rocketchat.service entered failed state.

The MongoDB was updated to v4.2 which is compatible with this RC version as per release data. Since this is a major version update, I think there could be some DB level changes in past versions, and not sure about these index errors thrown in Systemd. Could someone shed some light on this or let me know the solution to apply.

Steps to reproduce:

Expected behavior:

Actual behavior:

Server Setup Information:

• Version of Rocket.Chat Server: v5.0.4
• Operating System: CentOS 7
• Deployment Method: tar
• Number of Running Instances: 1
• DB Replicaset Oplog: Enabled
• NodeJS Version: 14.19.3
• MongoDB Version: 4.2

[iamfasal]

As mentioned here #24347 , I tried dropping those indexes using:

mongo
use rocketchat
db.rocketchat_integration_history.dropIndexes()
db.rocketchat_nps_vote.dropIndexes()

Now these are not showing while restarting RC service, however, it is still getting failed without any errors in logs.

Aug 27 15:21:10  systemd[1]: Started The Rocket.Chat server.
Aug 27 15:21:24  systemd[1]: rocketchat.service: main process exited, code=killed, status=11/SEGV
Aug 27 15:21:24  systemd[1]: Unit rocketchat.service entered failed state.
Aug 27 15:21:24  systemd[1]: rocketchat.service failed.

Any idea?

[debdutdeb]

What about a fresh install? Does that behave similarly?

[himpierre]

Release notes are saying:
Upgrade to version 5.0 can be done only from version 4.x (#26100)

[debdutdeb]

Ah that's right. Thanks @himpierre !

@Fasal-Official Please update to 4.x first before jumping to 5.x.

[iamfasal]

@debdutdeb @himpierre Thanks for this hint. I've now upgraded RC from v3.18.2 to v4.6.4 successfully. However, it is still failing to start when I upgrade from v4.6.4 to v5.0.4 with below errors:

Aug 29 12:53:42  systemd[1]: Started The Rocket.Chat server.
Aug 29 12:53:51  rocketchat[11275]: Some indexes for collection 'rocketchat_integration_history' could not be created:
Aug 29 12:53:51  rocketchat[11275]: Index with name: _updatedAt_1 already exists with different options
Aug 29 12:53:51  rocketchat[11275]: Some indexes for collection 'rocketchat_nps_vote' could not be created:
Aug 29 12:53:51  rocketchat[11275]: Index with name: npsId_1_identifier_1 already exists with different options
Aug 29 12:53:57  systemd[1]: rocketchat.service: main process exited, code=killed, status=11/SEGV
Aug 29 12:53:57  systemd[1]: Unit rocketchat.service entered failed state.
Aug 29 12:53:57  systemd[1]: rocketchat.service failed.

Any thoughts about this?

[debdutdeb]

Ah this is Centos7 - and segfaulting. There is no solution for this as of now unfortunately. We are discussing how to manage it since centos7's core is very much outdated

[iamfasal]

Yes, this is CentOS 7 server. CentOS 7 is still receiving updates until 2024 as per RedHat, so not sure what exactly outdated on it. I either don't see any segfault in the kernel logs, and only logs I've are mentioned above with some index conflicts.

If you are referring to segfault, we had an issue before with v4.0 which was discussed over #23346 . Is this the same situation? Any ETA available for the solution? @debdutdeb

[iamfasal]

Any update on this case @debdutdeb @sampaiodiego

[debdutdeb]

Hi, sorry, no ETAs as of now.