4.4.0 Upgrade: Error creating index: users -> { bio: 1 } #24347

oliob · 2022-01-29T14:28:11Z

Description:

After upgrading from 4.3.2 to 4.4.0 i get the message on each start of rocketchat server:
Jan 29 15:19:18 chat rocketchat[180809]: Error creating index: users -> { bio: 1 } { sparse: 1 } MongoError: Index with name: bio_1 already exists with different options

Steps to reproduce:

Start Rocketchat server 4.4.0

Expected behavior:

no error message

Actual behavior:

see above

Server Setup Information:

Version of Rocket.Chat Server: 4.4.0
Operating System: Ubuntu
Deployment Method: tar
Number of Running Instances: 1
DB Replicaset Oplog: Enabled
NodeJS Version: 14.18.3
MongoDB Version: 4.4.12

Client Setup Information

Desktop App or Browser Version:
Operating System:

Additional context

Relevant logs:

Jan 29 15:19:14 chat systemd[1]: Started The Rocket.Chat server.
Jan 29 15:19:18 chat rocketchat[180809]: Error creating index: users -> { bio: 1 } { sparse: 1 } MongoError: Index with name: bio_1 already exists with different options
Jan 29 15:19:18 chat rocketchat[180809]: at MessageStream.messageHandler (/opt/Rocket.Chat/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:272:20)
Jan 29 15:19:18 chat rocketchat[180809]: at MessageStream.emit (events.js:400:28)
Jan 29 15:19:18 chat rocketchat[180809]: at MessageStream.emit (domain.js:475:12)
Jan 29 15:19:18 chat rocketchat[180809]: at processIncomingData (/opt/Rocket.Chat/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:144:12)
Jan 29 15:19:18 chat rocketchat[180809]: at MessageStream._write (/opt/Rocket.Chat/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:42:5)
Jan 29 15:19:18 chat rocketchat[180809]: at writeOrBuffer (internal/streams/writable.js:358:12)
Jan 29 15:19:18 chat rocketchat[180809]: at MessageStream.Writable.write (internal/streams/writable.js:303:10)
Jan 29 15:19:18 chat rocketchat[180809]: at Socket.ondata (internal/streams/readable.js:731:22)
Jan 29 15:19:18 chat rocketchat[180809]: at Socket.emit (events.js:400:28)
Jan 29 15:19:18 chat rocketchat[180809]: at Socket.emit (domain.js:475:12)
Jan 29 15:19:18 chat rocketchat[180809]: at addChunk (internal/streams/readable.js:293:12)
Jan 29 15:19:18 chat rocketchat[180809]: at readableAddChunk (internal/streams/readable.js:267:9)
Jan 29 15:19:18 chat rocketchat[180809]: at Socket.Readable.push (internal/streams/readable.js:206:10)
Jan 29 15:19:18 chat rocketchat[180809]: at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
Jan 29 15:19:18 chat rocketchat[180809]: at TCP.callbackTrampoline (internal/async_hooks.js:130:17) {

The text was updated successfully, but these errors were encountered:

lazyandproud · 2022-01-29T17:37:19Z

same here.

jthomaschewski · 2022-01-30T11:07:48Z

Same problem here.

Looks like Meteor was upgraded from v2.2 to v2.5 in RocketChat v4.4.0.
So this is likely caused by the following Meteor issue: meteor/meteor#11666

btw: just downgrading back to v4.3.3 seems to work without issues as it looks like there are no breaking db migrations in v4.4.0

JakubJachym · 2022-02-01T07:25:04Z

We have similar issue, but different index.

MongoError: An equivalent index already exists with the same name but different options. Requested index: { v: 2, unique: true, key: { username: 1 }, name: "username_1", sparse: true }, existing index: { v: 2, unique: true, key: { username: 1 }, name: "username_1", ns:"rocketchat.users", sparse: 1 }

Rolled back to 4.3.3.

sampaiodiego · 2022-02-01T17:48:17Z

just to let you guys know, even though there is this index warning, the system still works.

ixlandia · 2022-02-01T18:01:59Z

No, it doesn't. The server job ends immediately.

ixlandia · 2022-02-01T18:04:27Z

I tried deleting the index generating the error but quickly realized it is Every. Single. Index.

MongoError: An equivalent index already exists with the same name but different options. Requested index: { v: 2, unique: true, key: { emails.address: 1 }, name: "emails.address_1", sparse: true }, existing index: { v: 2, unique: true, key: { emails.address: 1 }, name: "emails.address_1", sparse: 1 }
    at MessageStream.messageHandler (/opt/Rocket.Chat/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:272:20)
    at MessageStream.emit (events.js:400:28)
    at processIncomingData (/opt/Rocket.Chat/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:144:12)
    at MessageStream._write (/opt/Rocket.Chat/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:42:5)
    at writeOrBuffer (internal/streams/writable.js:358:12)
    at MessageStream.Writable.write (internal/streams/writable.js:303:10)
    at TLSSocket.ondata (internal/streams/readable.js:731:22)
    at TLSSocket.emit (events.js:400:28)
    at addChunk (internal/streams/readable.js:293:12)
    at readableAddChunk (internal/streams/readable.js:267:9)
    at TLSSocket.Readable.push (internal/streams/readable.js:206:10)
    at TLSWrap.onStreamRead (internal/stream_base_commons.js:188:23) {
  ok: 0,
  code: 85,
  codeName: 'IndexOptionsConflict',
  '$clusterTime': {
    clusterTime: Timestamp { _bsontype: 'Timestamp', low_: 7, high_: 1643573064 },
    signature: {
      hash: Binary {
        _bsontype: 'Binary',
        sub_type: 0,
        position: 20,
        buffer: Buffer(20) [Uint8Array] [
          249, 231,  77, 137, 42,  10,
           95, 116, 174,  86, 33, 110,
          221, 118,  68,  81, 67, 114,
           28, 163
        ]
      },
      keyId: Long { _bsontype: 'Long', low_: 2, high_: 1630216647 }
    }
  },
  operationTime: Timestamp { _bsontype: 'Timestamp', low_: 7, high_: 1643573064 }
}
/opt/Rocket.Chat/programs/server/node_modules/fibers/future.js:313
                                                throw(ex);
                                                ^

sampaiodiego · 2022-02-01T18:08:53Z

that's weird, on our Open server, we get the error only for bio and prid indexes, but it doesn't halt the startup.. it just shows a warning message and keeps going.

anyways, we'll take a look on why this is happening..

I'll get back in case I need additional information.. thx

JakubJachym · 2022-02-01T19:54:50Z

I can confirm our server was dead too. Full log:

systemd[1]: Started The Rocket.Chat server.
rocketchat[3371362]: /opt/Rocket.Chat/programs/server/node_modules/fibers/future.js:313
rocketchat[3371362]: #011#011#011#011#011#011throw(ex);
rocketchat[3371362]: #011#011#011#011#011#011^
rocketchat[3371362]: MongoError: An equivalent index already exists with the same name but different options. Requested index: { v: 2, unique: true, key: { username: 1 }, name: "username_1", sparse: true }, existing index: { v: 2, unique: true, key: { username: 1 }, name: "username_1", ns: "rocketchat.users", sparse: 1 }
rocketchat[3371362]:     at MessageStream.messageHandler (/opt/Rocket.Chat/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:272:20)
rocketchat[3371362]:     at MessageStream.emit (events.js:400:28)
rocketchat[3371362]:     at processIncomingData (/opt/Rocket.Chat/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:144:12)
rocketchat[3371362]:     at MessageStream._write (/opt/Rocket.Chat/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:42:5)
rocketchat[3371362]:     at writeOrBuffer (internal/streams/writable.js:358:12)
rocketchat[3371362]:     at MessageStream.Writable.write (internal/streams/writable.js:303:10)
rocketchat[3371362]:     at Socket.ondata (internal/streams/readable.js:731:22)
rocketchat[3371362]:     at Socket.emit (events.js:400:28)
rocketchat[3371362]:     at addChunk (internal/streams/readable.js:293:12)
rocketchat[3371362]:     at readableAddChunk (internal/streams/readable.js:267:9)
rocketchat[3371362]:     at Socket.Readable.push (internal/streams/readable.js:206:10)
rocketchat[3371362]:     at TCP.onStreamRead (internal/stream_base_commons.js:188:23) {
rocketchat[3371362]:   ok: 0,
rocketchat[3371362]:   code: 85,
rocketchat[3371362]:   codeName: 'IndexOptionsConflict',
rocketchat[3371362]:   '$clusterTime': {
rocketchat[3371362]:     clusterTime: Timestamp { _bsontype: 'Timestamp', low_: 3, high_: 1643625509 },
rocketchat[3371362]:     signature: {
rocketchat[3371362]:       hash: Binary {
rocketchat[3371362]:         _bsontype: 'Binary',
rocketchat[3371362]:         sub_type: 0,
rocketchat[3371362]:         position: 20,
rocketchat[3371362]:         buffer: Buffer(20) [Uint8Array] [
rocketchat[3371362]:           0, 0, 0, 0, 0, 0, 0,
rocketchat[3371362]:           0, 0, 0, 0, 0, 0, 0,
rocketchat[3371362]:           0, 0, 0, 0, 0, 0
rocketchat[3371362]:         ]
rocketchat[3371362]:       },
rocketchat[3371362]:       keyId: 0
rocketchat[3371362]:     }
rocketchat[3371362]:   },
rocketchat[3371362]:   operationTime: Timestamp { _bsontype: 'Timestamp', low_: 3, high_: 1643625509 }
rocketchat[3371362]: }
systemd[1]: rocketchat.service: Main process exited, code=exited, status=1/FAILURE
systemd[1]: rocketchat.service: Failed with result 'exit-code'.

sampaiodiego · 2022-02-01T19:56:42Z

interesting.. apparently the ones that kill the server are coming from Meteor and not from Rocket.Chat code, because we do have a try/catch to prevent the server from crashing.

sampaiodiego · 2022-02-03T19:19:28Z

I'm trying to reproduce the other index errors other than the bio index error..

I've started a rocket.chat version 2.0.0 and starting doing updates up to 4.4.0, and still didn't have such errors.

I'm now trying to figure what other variables may take place here.. for you with errors preventing the server from starting up, what mongo version do you use? what is your deployment method? do you remember which was the first rocket.chat version deployed using the same database name?

JakubJachym · 2022-02-03T21:22:55Z

@sampaiodiego We deploy using the steps described in Manual Installation for Ubuntu. We have a bash script for it:

#!/bin/bash
version=${1:-latest}

curl -L https://releases.rocket.chat/$version/download -o /tmp/rocket.chat.tgz && \
tar --no-same-owner -xzf /tmp/rocket.chat.tgz -C /tmp && \
sudo chown -R rocketchat:rocketchat /tmp/bundle && \
cd /tmp/bundle/programs/server && \
/usr/bin/npm install

Then we replace /opt/Rocket.Chat with /tmp/bundle and start the service.

We're currently running MongoDB 5.0.5.

We're using same update process for the Rocket.chat since 12/2018 when we've started using it. Don't really remember what version it was.

ixlandia · 2022-02-03T21:31:44Z

I have a feeling we might have to wait for upstream to fix meteor/meteor#11666
Reading the comments on that issue, it looks like the errors occur during meteor startup, before Rocket.Chat code is even executed.
Our current server was set up in Sept 2018, using Rocket.Chat 0.69.1 and ~~MongoDB 3.4~~ (actually I'm not certain of the MongoDB version).
It seems more likely that it is due to the version of MongoDB being used at that point. It looks like ~~3.7~~ 3.4 was the first version to validate index options instead of just silently ignoring invalid ones. (https://docs.mongodb.com/manual/release-notes/3.4-compatibility/#stricter-validation-of-index-specifications)

jthomaschewski · 2022-02-08T02:36:02Z

@sampaiodiego I can confirm that the fix in #24409 does work as @ixlandia already assumed.
I just tested upgrading from v3.3.3 to v4.4.1 and its still failing.

This has to be fixed in Meteor though. Untill then a workaround is probably to drop the problematic indexes and let meteor recreate them on startup.
I'm not sure though which rocketchat indexes gets created automatically on startup and which only in migrations - as there are ensureIndex calls both in migrations and model constructors...
Has anyone found a workaround/recreated those indexes?

Some logs/info
I do have the error with a different index (username_1) but I think the underlying issue is, that the startup fails before migrations are run.
So e.g. the new migration 254 has still not been run:

rs0:PRIMARY> db.migrations.find()
{ "_id" : "control", "version" : 252, "locked" : false, "lockedAt" : ISODate("2022-01-30T11:08:34.249Z"), "buildAt" : "2022-01-28T22:36:02.409Z" }

startup log

MongoError: An equivalent index already exists with the same name but different options. Requested index: { v: 2, unique: true, key: { username: 1 }, name: "username_1", sparse:
true }, existing index: { v: 2, unique: true, key: { username: 1 }, name: "username_1", ns: "rocketchat.users", sparse: 1 }
  at MessageStream.messageHandler (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:272:20)
  at MessageStream.emit (events.js:400:28)
  at processIncomingData (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:144:12)
  at MessageStream._write (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:42:5)
  at writeOrBuffer (internal/streams/writable.js:358:12)
  at MessageStream.Writable.write (internal/streams/writable.js:303:10)
  at Socket.ondata (internal/streams/readable.js:731:22)
  at Socket.emit (events.js:400:28)
  at addChunk (internal/streams/readable.js:293:12)
  at readableAddChunk (internal/streams/readable.js:267:9)
  at Socket.Readable.push (internal/streams/readable.js:206:10)
  at TCP.onStreamRead (internal/stream_base_commons.js:188:23) {
ok: 0,
code: 85,
codeName: 'IndexOptionsConflict',
'$clusterTime': {
  clusterTime: Timestamp { _bsontype: 'Timestamp', low_: 1, high_: 1644286935 },
  signature: {
    hash: Binary {
      _bsontype: 'Binary',
      sub_type: 0,
      position: 20,
      buffer: Buffer(20) [Uint8Array] [
        0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0
      ]
    },
    keyId: 0
  }
},
operationTime: Timestamp { _bsontype: 'Timestamp', low_: 1, high_: 1644286935 }
}
/app/bundle/programs/server/node_modules/fibers/future.js:313
                                              throw(ex);
                                              ^

MongoError: An equivalent index already exists with the same name but different options. Requested index: { v: 2, unique: true, key: { username: 1 }, name: "username_1", sparse: true }, existing index: { v: 2, unique: true, key: { username: 1 }, name: "username_1", ns: "rocketchat.users", sparse: 1 }
  at MessageStream.messageHandler (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:272:20)
  at MessageStream.emit (events.js:400:28)
  at processIncomingData (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:144:12)
  at MessageStream._write (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:42:5)
  at writeOrBuffer (internal/streams/writable.js:358:12)
  at MessageStream.Writable.write (internal/streams/writable.js:303:10)
  at Socket.ondata (internal/streams/readable.js:731:22)
  at Socket.emit (events.js:400:28)
  at addChunk (internal/streams/readable.js:293:12)
  at readableAddChunk (internal/streams/readable.js:267:9)
  at Socket.Readable.push (internal/streams/readable.js:206:10)
  at TCP.onStreamRead (internal/stream_base_commons.js:188:23) {
ok: 0,
code: 85,
codeName: 'IndexOptionsConflict',
'$clusterTime': {
  clusterTime: Timestamp { _bsontype: 'Timestamp', low_: 1, high_: 1644286995 },
  signature: {
    hash: Binary {
      _bsontype: 'Binary',
      sub_type: 0,
      position: 20,
      buffer: Buffer(20) [Uint8Array] [
        0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0
      ]
    },
    keyId: 0
  }
},
operationTime: Timestamp { _bsontype: 'Timestamp', low_: 1, high_: 1644286995 }
}

jthomaschewski · 2022-02-08T02:57:49Z

Workaround for everyone looking for a quick solution

RocketChat v4.4.1 will start after manually dropping all old indexes. On start most (hopefully all) indexes should be recreated as (try)EnsureIndex calls are placed in startup/constructor code of models throughout the code.
There is no guarantee though, some indexes could be created e.g. only in migrations...

I did the following and RocketChat started successfully:

  /**
   * WARNING: be careful, especially with huge rocket instances
   * this could take a while, cause heavy load and some indexes might not be recreated...
   */
// first stop rocketchat & backup db
// then run this in mongodb:
db.users.dropIndexes()
db.rocketchat_room.dropIndexes()
db.rocketchat_message.dropIndexes()
// start rocketchat => it should work and indexes should be recreated

sampaiodiego · 2022-02-08T13:33:05Z

thanks @jthomaschewski .. indeed the indexes you had that were failing were from Meteor, and they don't have a try/catch to "ignore" the errors, that's why they prevent the server to start up.

it is safe to remove indexes from DB as you did since they're re-created on every startup.. the ones we have on migrations are the ones that have changed the shape, in this case the one on the model would fail and then the migration would run and fix it.

I'll be adding a special to the change log so people can find out how to remove indexes that are preventing the server to start up.

jthomaschewski · 2022-02-08T14:42:34Z

Thanks for confirming that indexes will be recreated automatically @sampaiodiego

Actually not only indexes from Meteor where failing. I started by dropping the index which failed during startup, restarting and so on. Many indexes where causing this issue - also some/all of rocketchat_room/rocketchat_message collections.
Thats why I dropped indexes of those as well.

Maybe recreating those very old indexes is a good idea anyways :-)

robbyoconnor · 2022-02-11T05:35:06Z

This should be reopened -- or retitled because it's not fixed.

oliob · 2022-02-11T15:21:39Z

for me version 4.4.1 fixed the bio index problem. I cannot speak for the other index cases above.

robbyoconnor · 2022-02-11T18:42:35Z

@oliob I'm shocked these weren't caught to be honest -- I'm not attacking you, just making sure it's mentioned

jthomaschewski · 2022-02-11T19:50:02Z

@oliob I'm shocked these weren't caught to be honest -- I'm not attacking you, just making sure it's mentioned

The issue is, that the issue seems to occur for every index, which was created with a quite old MongoDB version (+ probably old Meteor version).
This can't be reproduced by running tests with fairly up-to-date tech stack - all migrations will run successfully.

Also at least some indexes (those maintained by Meteor itself on users collection) cause these failures before the applications starts can thus the application code can't do anything about it.

It's a very hard one to be honest as it's more an issue of the underlying framework (Meteor) in combination with long running instances.
Best Rocket.Chat can do right now is probably to document this issue in the release/upgrade notes and provide a workaround (e.g. dropping the indexes as I mentioned in #24347 (comment))

Also one can hope for a fix in the Meteor framework which might make manual steps unnecessary in the future.

robbyoconnor · 2022-02-11T23:50:41Z

To be clear: it doesn't make it not work, just screams when migrations are run

wojtokuba · 2022-03-03T16:01:07Z

Having the same problem. Firstly I had an index start issue, now the chat keeps restarting every few hours.

No more logs provided just exit code:

Mar  3 15:58:03 localhost systemd[1]: rocketchat.service: Main process exited, code=killed, status=4/ILL
Mar  3 15:58:03 localhost systemd[1]: rocketchat.service: Failed with result 'signal'.
Mar  3 15:58:08 localhost systemd[1]: rocketchat.service: Service RestartSec=5s expired, scheduling restart.
Mar  3 15:58:08 localhost systemd[1]: rocketchat.service: Scheduled restart job, restart counter is at 15.

Whats more, mobile application is missing avatars and sometimes a chat refresh on phone leads to whole RocketChat app crash.

main-sonn · 2022-04-01T13:10:05Z

We tested the upgrade from 3.18.2 to version 4.5.5 and got these index errors:

Error creating index: canned_response -> { shortcut: 1 } { unique: true } MongoError: Index with name: shortcut_1 already exists with different options

Error creating index: users -> { bio: 1 } { unique: 1 } MongoError: E11000 duplicate key error collection: rocketchat.users index: bio_1 dup key: { bio: null }

Should we wait for a fix before installing the upgrade on the production server?

@wojtokuba is your restarting issue fixed in newer versions? How long has the server been running before it keeps restarting?

bognerf · 2022-04-11T09:57:32Z

Upgraded 3.18.5 to 4.6.1 (manual installation), same error messages in log:

Error creating index: canned_response -> { shortcut: 1 } { unique: true } MongoError: Index with name: shortcut_1 already exists with different options

and

Error creating index: users -> { bio: 1 } { unique: 1 } MongoError: E11000 duplicate key error collection: rocketchat.users index: bio_1 dup key: { bio: null }

But, indeed, the server is/keeps running and seems to work.

TheFatal · 2022-06-17T20:09:15Z

Workaround for everyone looking for a quick solution

RocketChat v4.4.1 will start after manually dropping all old indexes. On start most (hopefully all) indexes should be recreated as (try)EnsureIndex calls are placed in startup/constructor code of models throughout the code. There is no guarantee though, some indexes could be created e.g. only in migrations...

I did the following and RocketChat started successfully:
  /**
   * WARNING: be careful, especially with huge rocket instances
   * this could take a while, cause heavy load and some indexes might not be recreated...
   */
// first stop rocketchat & backup db
// then run this in mongodb:
db.users.dropIndexes()
db.rocketchat_room.dropIndexes()
db.rocketchat_message.dropIndexes()
// start rocketchat => it should work and indexes should be recreated

worked for me, ty 👍

whitetiger264 · 2022-06-28T13:06:46Z

I seem to have a similar problem when upgrading from 4.7.2 -> 4.8.1:

Jun 28 14:51:50 core rocketchat: Error creating indexes for integration_history MongoError: Index with name: _updatedAt_1 already exists with different options
Jun 28 14:51:50 core rocketchat: at MessageStream.messageHandler (/opt/Rocket.Chat/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:272:20)
Jun 28 14:51:50 core rocketchat: at MessageStream.emit (events.js:400:28)
Jun 28 14:51:50 core rocketchat: at MessageStream.emit (domain.js:475:12)
Jun 28 14:51:50 core rocketchat: at processIncomingData (/opt/Rocket.Chat/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:144:12)
Jun 28 14:51:50 core rocketchat: at MessageStream._write (/opt/Rocket.Chat/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:42:5)
Jun 28 14:51:50 core rocketchat: at writeOrBuffer (internal/streams/writable.js:358:12)
Jun 28 14:51:50 core rocketchat: at MessageStream.Writable.write (internal/streams/writable.js:303:10)
Jun 28 14:51:50 core rocketchat: at Socket.ondata (internal/streams/readable.js:731:22)
Jun 28 14:51:50 core rocketchat: at Socket.emit (events.js:400:28)
Jun 28 14:51:50 core rocketchat: at Socket.emit (domain.js:475:12)
Jun 28 14:51:50 core rocketchat: at addChunk (internal/streams/readable.js:293:12)
Jun 28 14:51:50 core rocketchat: at readableAddChunk (internal/streams/readable.js:267:9)
Jun 28 14:51:50 core rocketchat: at Socket.Readable.push (internal/streams/readable.js:206:10)
Jun 28 14:51:50 core rocketchat: at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
Jun 28 14:51:50 core rocketchat: at TCP.callbackTrampoline (internal/async_hooks.js:130:17) {
Jun 28 14:51:50 core rocketchat: operationTime: Timestamp { _bsontype: 'Timestamp', low_: 1, high_: 1656420702 },
Jun 28 14:51:50 core rocketchat: ok: 0,
Jun 28 14:51:50 core rocketchat: code: 85,
Jun 28 14:51:50 core rocketchat: codeName: 'IndexOptionsConflict',
Jun 28 14:51:50 core rocketchat: '$clusterTime': {
Jun 28 14:51:50 core rocketchat: clusterTime: Timestamp { _bsontype: 'Timestamp', low_: 1, high_: 1656420702 },
Jun 28 14:51:50 core rocketchat: signature: { hash: [Binary], keyId: 0 }
Jun 28 14:51:50 core rocketchat: }
Jun 28 14:51:50 core rocketchat: }

It seems this is not solved.

Version of Rocket.Chat Server: 4.7.2
Operating System: CentOS 7
NodeJS Version: v14.18.3
MongoDB Version: 4.2.21 / wiredTiger (oplog Enabled)
NPM: 6.14.15

wreiske · 2022-07-03T00:16:29Z

I had to run:

db.rocketchat_nps_vote.dropIndexes()
db.users.dropIndexes()
db.rocketchat_room.dropIndexes()
db.rocketchat_message.dropIndexes()
db.rocketchat_integration_history.dropIndexes()
db.rocketchat_apps_logs.dropIndexes()

And from #26089 (comment)

db.rocketchat_settings.insert({ 
  "_id" : "Rate_Limiter_Limit_RegisterUser",
  "createdAt" : ISODate("2022-07-01T05:32:48.791Z"),
  "value" : 1,
  "packageValue" : 1,
  "valueSource" : "packageValue",
  "secret" : false,
  "enterprise" : false,
  "i18nDescription" : "Rate_Limiter_Limit_RegisterUser_Description",
  "autocomplete" : true,
  "sorter" : 0,
  "ts" : ISODate("2022-07-01T05:32:48.791Z"),
  "type" : "int",
  "group" : "Rate Limiter",
  "section" : "Feature_Limiting",
  "enableQuery" : "{\"_id\":\"API_Enable_Rate_Limiter\",\"value\":true}",
  "i18nLabel" : "Rate_Limiter_Limit_RegisterUser",
  "hidden" : false,
  "blocked" : false,
  "requiredOnWizard" : false,
  "env" : false,
  "public" : false,
  "_updatedAt" : ISODate("2022-07-01T05:38:12.246Z")
})

ggodreau · 2022-07-03T17:52:45Z

@wreiske - thanks 🙇🏿 Was able to complete my full migration from 3.18 directly to 4.8.1 using your 6 index drops in addition to this rate limiter code. 'Only' dropping the 3 indexes did not resolve my issue. HTH somebody ❤️ 💋

Fiodin · 2022-07-10T17:46:09Z

Unfortunately this did not help me solve my problem....

And within the last command rate_limiter I get an writeError:
"code" : 11000, "errmsg" : "E11000 duplicate key error collection: parties.rocketchat_settings index: _id_ dup key: { _id: \"Rate_Limiter_Limit_RegisterUser\" }"

Could be that I am blind now after the whole day on computer and lines, but I do not see the error.

wreiske · 2022-07-10T17:50:06Z

Unfortunately this did not help me solve my problem....

And within the last command rate_limiter I get an writeError: "code" : 11000, "errmsg" : "E11000 duplicate key error collection: parties.rocketchat_settings index: _id_ dup key: { _id: \"Rate_Limiter_Limit_RegisterUser\" }"

Could be that I am blind now after the whole day on computer and lines, but I do not see the error.

For anyone else having other index issues not listed in my comment above, look at the collection:
E11000 duplicate key error collection: parties.rocketchat_settings

In this case, the database is parties and the collection is rocketchat_settings. We'll want to drop the indexes on rocketchat_settings so that when Rocket.Chat starts it will create the correct indexes.

Connect to mongo using mongo or mongosh depending on your MongoDB Version.

use parties // select your RC database. this is usually rocketchat, but in the case above Its probably parties
db.rocketchat_settings.dropIndexes();

Exit mongo and then reload your Rocket.Chat service. Rinse and repeat depending on if new errors arise. You may have multiple other indexes that need to be resolved depending on how you use Rocket.Chat.

Fiodin · 2022-07-10T17:58:25Z

So I repeat the last command since there is no more error?

wreiske · 2022-07-10T17:59:24Z

So I repeat the last command since there is no more error?

If there are no more errors you shouldn't need to repeat any commands. Just start up Rocket.Chat and it should run all migrations and do the upgrade. This could take a few minutes depending on how powerful your server is.

whitetiger264 · 2022-07-10T18:02:41Z

@wreiske is there any solution for this perhaps: LINK

wreiske · 2022-07-10T18:05:05Z

@wreiske is there any solution for this perhaps: LINK

Seeing as that ticket is from the same person, it's possible the redirect loop mentioned here is because of the rocketchat_settings needing to have indexes dropped.

Fiodin · 2022-07-10T18:19:29Z

@wreiske is there any solution for this perhaps: LINK

Seeing as that ticket is from the same person, it's possible the redirect loop mentioned here is because of the rocketchat_settings needing to have indexes dropped.

That all leads to the same problem, looking at the logs...

Fiodin · 2022-07-10T18:25:18Z

See, ....
I stop the service for snap.rocketchat.server
then log-in the db with rocketchat-server.mongo
then use parties
and perform all the Indexdropping from above.
Then exit the db
and restart the service.

And the server is back in the reboot-loop.

I do the same and add the settings.insert and I get the error from above even with dropping the settings_indexes
and guess what: the server is in a reboot-loop

It is confusing and frustrating.

maxhaslbeck · 2023-01-04T11:28:11Z

Hi,
I also got the

Error creating index: canned_response -> { shortcut: 1 } { unique: true } MongoError: Index with name: shortcut_1 already exists with different options

error
I deleted all Indexes in the rocketchat database following:

https://stackoverflow.com/questions/24241742/drop-all-indexes-from-all-collections-in-a-mongodb-database-using-the-command-li

now the error does not occur anymore. Was this a safe action? I'm new to rocketchat so I dont know!

sampaiodiego · 2023-01-10T17:21:58Z

@maxhaslbeck it is safe if you restart rocket.chat after removing all indexes since rocket.chat will create all required indexes during startup

sampaiodiego added this to the 4.4.1 milestone Feb 1, 2022

jthomaschewski mentioned this issue Feb 3, 2022

MongoError: Index with name: username_1 already exists with different options meteor/meteor#11666

Closed

sampaiodiego mentioned this issue Feb 3, 2022

[FIX] Startup errors creating indexes #24409

Merged

sampaiodiego closed this as completed in #24409 Feb 3, 2022

CAberry mentioned this issue Mar 2, 2022

[4.5.0][CE] MongoError: index already exist after upgrade #24672

Closed

sgohl mentioned this issue Mar 3, 2022

LDAP - User not found - since 4.5.0 (aka better logging needed) #24694

Open

rvva mentioned this issue May 20, 2022

4.7.0 Upgrade: "Error creating index: users -> meteor.users index: bio_1 dup key: { bio: null }" #25576

Open

wreiske mentioned this issue Jul 3, 2022

"numRequestsAllowed" property in rateLimiter not set after upgrade from 3.x to 4.x #26089

Closed

Jenso-DE mentioned this issue Aug 8, 2022

read_receipts MongoError: Index build failed #26259

Open

iamfasal mentioned this issue Aug 27, 2022

Update to v5.0.4 failing with indexes errors #26716

Closed

4.4.0 Upgrade: Error creating index: users -> { bio: 1 } #24347

4.4.0 Upgrade: Error creating index: users -> { bio: 1 } #24347

Comments

oliob commented Jan 29, 2022

Description:

Steps to reproduce:

Expected behavior:

Actual behavior:

Server Setup Information:

Client Setup Information

Additional context

Relevant logs:

lazyandproud commented Jan 29, 2022

jthomaschewski commented Jan 30, 2022 • edited Loading

JakubJachym commented Feb 1, 2022

sampaiodiego commented Feb 1, 2022

ixlandia commented Feb 1, 2022

ixlandia commented Feb 1, 2022 • edited Loading

sampaiodiego commented Feb 1, 2022

JakubJachym commented Feb 1, 2022

sampaiodiego commented Feb 1, 2022

sampaiodiego commented Feb 3, 2022

JakubJachym commented Feb 3, 2022

ixlandia commented Feb 3, 2022 • edited Loading

jthomaschewski commented Feb 8, 2022

jthomaschewski commented Feb 8, 2022 • edited Loading

Workaround for everyone looking for a quick solution

sampaiodiego commented Feb 8, 2022

jthomaschewski commented Feb 8, 2022

robbyoconnor commented Feb 11, 2022

oliob commented Feb 11, 2022

robbyoconnor commented Feb 11, 2022

jthomaschewski commented Feb 11, 2022

robbyoconnor commented Feb 11, 2022

wojtokuba commented Mar 3, 2022 • edited Loading

main-sonn commented Apr 1, 2022

bognerf commented Apr 11, 2022

TheFatal commented Jun 17, 2022

Workaround for everyone looking for a quick solution

whitetiger264 commented Jun 28, 2022

wreiske commented Jul 3, 2022

ggodreau commented Jul 3, 2022

Fiodin commented Jul 10, 2022 • edited Loading

wreiske commented Jul 10, 2022 • edited Loading

Fiodin commented Jul 10, 2022

wreiske commented Jul 10, 2022

whitetiger264 commented Jul 10, 2022

wreiske commented Jul 10, 2022

Fiodin commented Jul 10, 2022

Fiodin commented Jul 10, 2022 • edited Loading

maxhaslbeck commented Jan 4, 2023 • edited Loading

sampaiodiego commented Jan 10, 2023

jthomaschewski commented Jan 30, 2022 •

edited

Loading

ixlandia commented Feb 1, 2022 •

edited

Loading

ixlandia commented Feb 3, 2022 •

edited

Loading

jthomaschewski commented Feb 8, 2022 •

edited

Loading

wojtokuba commented Mar 3, 2022 •

edited

Loading

Fiodin commented Jul 10, 2022 •

edited

Loading

wreiske commented Jul 10, 2022 •

edited

Loading

Fiodin commented Jul 10, 2022 •

edited

Loading

maxhaslbeck commented Jan 4, 2023 •

edited

Loading