Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] errmsg: 'E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key #17045

Open
ankar84 opened this issue Mar 27, 2020 · 19 comments

Comments

@ankar84
Copy link

ankar84 commented Mar 27, 2020

Description:

From time to time rocket chat instances show that error:
errmsg: 'E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key: { : "6mkv7tvrJAmi4AJrR", : "pE4ztmmZFNJ5TZXFv", : "raufRbopbb9NDcGsQ" }',

But @frdmn advised me that it is an old bug, that was already fixed in #10061 by @sampaiodiego
But our server version is 3.0.3 and we have that bug on a regular basis.
How to fix it? Delete duplicate from mongodb? How to do it correctly?

Expected behavior:

Not have error E11000 duplicate key error index

Actual behavior:

Error and maybe rocket chat instance crash and server resource consume.

Server Setup Information:

  • Version of Rocket.Chat Server: 3.0.3
  • Operating System: CentOS7
  • Deployment Method: docker
  • Number of Running Instances: 12
  • DB Replicaset Oplog: Enabled
  • NodeJS Version: v12.14.0
  • MongoDB Version: 4.0.13

Client Setup Information

  • Desktop App Version: 2.17.9
  • Operating System:

Additional context

Show Read Receipts and Detailed Read Receipts are enabled
image
Allow Message Editing and Allow Message Deleting are enabled for 10 minutes
image

Relevant logs:

Logs from Kibana:

	March 27th 2020, 12:51:55.623	s-rc-srv-03	rocketchat_rocketchat_3_1	=================================
	March 27th 2020, 12:51:55.622	s-rc-srv-03	rocketchat_rocketchat_3_1	Setting EXIT_UNHANDLEDPROMISEREJECTION will cause the process to exit allowing your service to automatically restart the process
	March 27th 2020, 12:51:55.622	s-rc-srv-03	rocketchat_rocketchat_3_1	Errors like this can cause oplog processing errors.
	March 27th 2020, 12:51:55.622	s-rc-srv-03	rocketchat_rocketchat_3_1	Future node.js versions will automatically exit the process
	March 27th 2020, 12:51:55.621	s-rc-srv-03	rocketchat_rocketchat_3_1	[Symbol(mongoErrorContextSymbol)]: {}
	March 27th 2020, 12:51:55.621	s-rc-srv-03	rocketchat_rocketchat_3_1	---------------------------------
	March 27th 2020, 12:51:55.621	s-rc-srv-03	rocketchat_rocketchat_3_1	}
	March 27th 2020, 12:51:55.620	s-rc-srv-03	rocketchat_rocketchat_3_1	},
	March 27th 2020, 12:51:55.620	s-rc-srv-03	rocketchat_rocketchat_3_1	}
	March 27th 2020, 12:51:55.615	s-rc-srv-03	rocketchat_rocketchat_3_1	lastOp: [Object]
	March 27th 2020, 12:51:55.614	s-rc-srv-03	rocketchat_rocketchat_3_1	nModified: 0,
	March 27th 2020, 12:51:55.614	s-rc-srv-03	rocketchat_rocketchat_3_1	nRemoved: 0,
	March 27th 2020, 12:51:55.614	s-rc-srv-03	rocketchat_rocketchat_3_1	nMatched: 0,
	March 27th 2020, 12:51:55.614	s-rc-srv-03	rocketchat_rocketchat_3_1	upserted: [],
	March 27th 2020, 12:51:55.613	s-rc-srv-03	rocketchat_rocketchat_3_1	nUpserted: 0,
	March 27th 2020, 12:51:55.613	s-rc-srv-03	rocketchat_rocketchat_3_1	nInserted: 0,
	March 27th 2020, 12:51:55.613	s-rc-srv-03	rocketchat_rocketchat_3_1	insertedIds: [Array],
	March 27th 2020, 12:51:55.612	s-rc-srv-03	rocketchat_rocketchat_3_1	writeConcernErrors: [],
	March 27th 2020, 12:51:55.607	s-rc-srv-03	rocketchat_rocketchat_3_1	writeErrors: [Array],
	March 27th 2020, 12:51:55.606	s-rc-srv-03	rocketchat_rocketchat_3_1	result: {
	March 27th 2020, 12:51:55.606	s-rc-srv-03	rocketchat_rocketchat_3_1	ok: 1,
	March 27th 2020, 12:51:55.605	s-rc-srv-03	rocketchat_rocketchat_3_1	}
	March 27th 2020, 12:51:55.605	s-rc-srv-03	rocketchat_rocketchat_3_1	result: BulkWriteResult {
	March 27th 2020, 12:51:55.605	s-rc-srv-03	rocketchat_rocketchat_3_1	},
	March 27th 2020, 12:51:55.604	s-rc-srv-03	rocketchat_rocketchat_3_1	messageId: 'raufRbopbb9NDcGsQ',
	March 27th 2020, 12:51:55.604	s-rc-srv-03	rocketchat_rocketchat_3_1	ts: 2020-03-27T09:51:55.555Z
	March 27th 2020, 12:51:55.604	s-rc-srv-03	rocketchat_rocketchat_3_1	userId: 'pE4ztmmZFNJ5TZXFv',
	March 27th 2020, 12:51:55.603	s-rc-srv-03	rocketchat_rocketchat_3_1	roomId: '6mkv7tvrJAmi4AJrR',
	March 27th 2020, 12:51:55.601	s-rc-srv-03	rocketchat_rocketchat_3_1	_id: 'DyBPDuqasguFt2BCD',
	March 27th 2020, 12:51:55.597	s-rc-srv-03	rocketchat_rocketchat_3_1	errmsg: 'E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key: { : "6mkv7tvrJAmi4AJrR", : "pE4ztmmZFNJ5TZXFv", : "raufRbopbb9NDcGsQ" }',
	March 27th 2020, 12:51:55.597	s-rc-srv-03	rocketchat_rocketchat_3_1	op: {
	March 27th 2020, 12:51:55.596	s-rc-srv-03	rocketchat_rocketchat_3_1	err: {
	March 27th 2020, 12:51:55.596	s-rc-srv-03	rocketchat_rocketchat_3_1	index: 0,
	March 27th 2020, 12:51:55.596	s-rc-srv-03	rocketchat_rocketchat_3_1	code: 11000,
	March 27th 2020, 12:51:55.595	s-rc-srv-03	rocketchat_rocketchat_3_1	name: 'BulkWriteError',
	March 27th 2020, 12:51:55.595	s-rc-srv-03	rocketchat_rocketchat_3_1	driver: true,
	March 27th 2020, 12:51:55.595	s-rc-srv-03	rocketchat_rocketchat_3_1	},
	March 27th 2020, 12:51:55.594	s-rc-srv-03	rocketchat_rocketchat_3_1	ts: 2020-03-27T09:51:55.555Z
	March 27th 2020, 12:51:55.592	s-rc-srv-03	rocketchat_rocketchat_3_1	messageId: 'raufRbopbb9NDcGsQ',
	March 27th 2020, 12:51:55.590	s-rc-srv-03	rocketchat_rocketchat_3_1	userId: 'pE4ztmmZFNJ5TZXFv',
	March 27th 2020, 12:51:55.589	s-rc-srv-03	rocketchat_rocketchat_3_1	_id: 'DyBPDuqasguFt2BCD',
	March 27th 2020, 12:51:55.589	s-rc-srv-03	rocketchat_rocketchat_3_1	roomId: '6mkv7tvrJAmi4AJrR',
	March 27th 2020, 12:51:55.588	s-rc-srv-03	rocketchat_rocketchat_3_1	code: 11000,
	March 27th 2020, 12:51:55.588	s-rc-srv-03	rocketchat_rocketchat_3_1	errmsg: 'E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key: { : "6mkv7tvrJAmi4AJrR", : "pE4ztmmZFNJ5TZXFv", : "raufRbopbb9NDcGsQ" }',
	March 27th 2020, 12:51:55.588	s-rc-srv-03	rocketchat_rocketchat_3_1	op: {
	March 27th 2020, 12:51:55.587	s-rc-srv-03	rocketchat_rocketchat_3_1	at processTicksAndRejections (internal/process/task_queues.js:75:11) {
	March 27th 2020, 12:51:55.587	s-rc-srv-03	rocketchat_rocketchat_3_1	index: 0,
	March 27th 2020, 12:51:55.587	s-rc-srv-03	rocketchat_rocketchat_3_1	at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/pool.js:397:18
	March 27th 2020, 12:51:55.586	s-rc-srv-03	rocketchat_rocketchat_3_1	at handler (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/topologies/replset.js:1155:22)
	March 27th 2020, 12:51:55.581	s-rc-srv-03	rocketchat_rocketchat_3_1	at resultHandler (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/bulk/common.js:501:23)
	March 27th 2020, 12:51:55.580	s-rc-srv-03	rocketchat_rocketchat_3_1	BulkWriteError: E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key: { : "6mkv7tvrJAmi4AJrR", : "pE4ztmmZFNJ5TZXFv", : "raufRbopbb9NDcGsQ" }
	March 27th 2020, 12:51:55.580	s-rc-srv-03	rocketchat_rocketchat_3_1	at OrderedBulkOperation.handleWriteError (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/bulk/common.js:1112:11)
	March 27th 2020, 12:51:55.566	s-rc-srv-03	rocketchat_rocketchat_3_1	=== UnHandledPromiseRejection ===
	March 27th 2020, 12:39:26.826	s-rc-srv-01	rocketchat_rocketchat_2_1	Setting EXIT_UNHANDLEDPROMISEREJECTION will cause the process to exit allowing your service to automatically restart the process
	March 27th 2020, 12:39:26.826	s-rc-srv-01	rocketchat_rocketchat_2_1	=================================
	March 27th 2020, 12:39:26.826	s-rc-srv-01	rocketchat_rocketchat_2_1	Future node.js versions will automatically exit the process
	March 27th 2020, 12:39:26.822	s-rc-srv-01	rocketchat_rocketchat_2_1	Errors like this can cause oplog processing errors.
	March 27th 2020, 12:39:26.820	s-rc-srv-01	rocketchat_rocketchat_2_1	[Symbol(mongoErrorContextSymbol)]: {}
	March 27th 2020, 12:39:26.820	s-rc-srv-01	rocketchat_rocketchat_2_1	}
	March 27th 2020, 12:39:26.820	s-rc-srv-01	rocketchat_rocketchat_2_1	---------------------------------
	March 27th 2020, 12:39:26.819	s-rc-srv-01	rocketchat_rocketchat_2_1	lastOp: [Object]
	March 27th 2020, 12:39:26.819	s-rc-srv-01	rocketchat_rocketchat_2_1	},
	March 27th 2020, 12:39:26.819	s-rc-srv-01	rocketchat_rocketchat_2_1	}
	March 27th 2020, 12:39:26.819	s-rc-srv-01	rocketchat_rocketchat_2_1	upserted: [],
	March 27th 2020, 12:39:26.818	s-rc-srv-01	rocketchat_rocketchat_2_1	nRemoved: 0,
	March 27th 2020, 12:39:26.818	s-rc-srv-01	rocketchat_rocketchat_2_1	nMatched: 0,
	March 27th 2020, 12:39:26.818	s-rc-srv-01	rocketchat_rocketchat_2_1	nModified: 0,
	March 27th 2020, 12:39:26.815	s-rc-srv-01	rocketchat_rocketchat_2_1	nUpserted: 0,
	March 27th 2020, 12:39:26.814	s-rc-srv-01	rocketchat_rocketchat_2_1	writeConcernErrors: [],
	March 27th 2020, 12:39:26.814	s-rc-srv-01	rocketchat_rocketchat_2_1	insertedIds: [Array],
	March 27th 2020, 12:39:26.814	s-rc-srv-01	rocketchat_rocketchat_2_1	nInserted: 0,
	March 27th 2020, 12:39:26.814	s-rc-srv-01	rocketchat_rocketchat_2_1	writeErrors: [Array],
	March 27th 2020, 12:39:26.813	s-rc-srv-01	rocketchat_rocketchat_2_1	result: BulkWriteResult {
	March 27th 2020, 12:39:26.813	s-rc-srv-01	rocketchat_rocketchat_2_1	ok: 1,
	March 27th 2020, 12:39:26.813	s-rc-srv-01	rocketchat_rocketchat_2_1	},
	March 27th 2020, 12:39:26.813	s-rc-srv-01	rocketchat_rocketchat_2_1	result: {
	March 27th 2020, 12:39:26.806	s-rc-srv-01	rocketchat_rocketchat_2_1	}
	March 27th 2020, 12:39:26.804	s-rc-srv-01	rocketchat_rocketchat_2_1	roomId: 'GMqJTNEx74S3oGGLZxGRrsJoTqEi9cFYbN',
	March 27th 2020, 12:39:26.804	s-rc-srv-01	rocketchat_rocketchat_2_1	userId: 'GMqJTNEx74S3oGGLZ',
	March 27th 2020, 12:39:26.804	s-rc-srv-01	rocketchat_rocketchat_2_1	messageId: 'gu9BrisadJMaEcFRn',
	March 27th 2020, 12:39:26.804	s-rc-srv-01	rocketchat_rocketchat_2_1	ts: 2020-03-27T09:39:26.770Z
	March 27th 2020, 12:39:26.803	s-rc-srv-01	rocketchat_rocketchat_2_1	code: 11000,
	March 27th 2020, 12:39:26.803	s-rc-srv-01	rocketchat_rocketchat_2_1	op: {
	March 27th 2020, 12:39:26.803	s-rc-srv-01	rocketchat_rocketchat_2_1	errmsg: 'E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key: { : "GMqJTNEx74S3oGGLZxGRrsJoTqEi9cFYbN", : "GMqJTNEx74S3oGGLZ", : "gu9BrisadJMaEcFRn" }',
	March 27th 2020, 12:39:26.803	s-rc-srv-01	rocketchat_rocketchat_2_1	_id: 'o9W4gbNQCnWvPa23R',
	March 27th 2020, 12:39:26.802	s-rc-srv-01	rocketchat_rocketchat_2_1	index: 0,
	March 27th 2020, 12:39:26.802	s-rc-srv-01	rocketchat_rocketchat_2_1	err: {
	March 27th 2020, 12:39:26.798	s-rc-srv-01	rocketchat_rocketchat_2_1	driver: true,
	March 27th 2020, 12:39:26.797	s-rc-srv-01	rocketchat_rocketchat_2_1	name: 'BulkWriteError',
	March 27th 2020, 12:39:26.796	s-rc-srv-01	rocketchat_rocketchat_2_1	userId: 'GMqJTNEx74S3oGGLZ',
	March 27th 2020, 12:39:26.796	s-rc-srv-01	rocketchat_rocketchat_2_1	},
	March 27th 2020, 12:39:26.796	s-rc-srv-01	rocketchat_rocketchat_2_1	messageId: 'gu9BrisadJMaEcFRn',
	March 27th 2020, 12:39:26.796	s-rc-srv-01	rocketchat_rocketchat_2_1	ts: 2020-03-27T09:39:26.770Z
	March 27th 2020, 12:39:26.795	s-rc-srv-01	rocketchat_rocketchat_2_1	_id: 'o9W4gbNQCnWvPa23R',
	March 27th 2020, 12:39:26.795	s-rc-srv-01	rocketchat_rocketchat_2_1	roomId: 'GMqJTNEx74S3oGGLZxGRrsJoTqEi9cFYbN',
	March 27th 2020, 12:39:26.795	s-rc-srv-01	rocketchat_rocketchat_2_1	op: {
	March 27th 2020, 12:39:26.794	s-rc-srv-01	rocketchat_rocketchat_2_1	code: 11000,
	March 27th 2020, 12:39:26.794	s-rc-srv-01	rocketchat_rocketchat_2_1	index: 0,
	March 27th 2020, 12:39:26.794	s-rc-srv-01	rocketchat_rocketchat_2_1	errmsg: 'E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key: { : "GMqJTNEx74S3oGGLZxGRrsJoTqEi9cFYbN", : "GMqJTNEx74S3oGGLZ", : "gu9BrisadJMaEcFRn" }',
	March 27th 2020, 12:39:26.791	s-rc-srv-01	rocketchat_rocketchat_2_1	at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/pool.js:397:18
	March 27th 2020, 12:39:26.791	s-rc-srv-01	rocketchat_rocketchat_2_1	at processTicksAndRejections (internal/process/task_queues.js:75:11) {
	March 27th 2020, 12:39:26.790	s-rc-srv-01	rocketchat_rocketchat_2_1	at resultHandler (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/bulk/common.js:501:23)
	March 27th 2020, 12:39:26.790	s-rc-srv-01	rocketchat_rocketchat_2_1	at OrderedBulkOperation.handleWriteError (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/bulk/common.js:1112:11)
	March 27th 2020, 12:39:26.790	s-rc-srv-01	rocketchat_rocketchat_2_1	at handler (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/topologies/replset.js:1155:22)
	March 27th 2020, 12:39:26.789	s-rc-srv-01	rocketchat_rocketchat_2_1	BulkWriteError: E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key: { : "GMqJTNEx74S3oGGLZxGRrsJoTqEi9cFYbN", : "GMqJTNEx74S3oGGLZ", : "gu9BrisadJMaEcFRn" }
	March 27th 2020, 12:39:26.777	s-rc-srv-01	rocketchat_rocketchat_2_1	=== UnHandledPromiseRejection ===
	March 27th 2020, 12:36:13.659	s-rc-srv-02	rocketchat_rocketchat_2_1	Setting EXIT_UNHANDLEDPROMISEREJECTION will cause the process to exit allowing your service to automatically restart the process
	March 27th 2020, 12:36:13.659	s-rc-srv-02	rocketchat_rocketchat_2_1	=================================
	March 27th 2020, 12:36:13.659	s-rc-srv-02	rocketchat_rocketchat_2_1	Errors like this can cause oplog processing errors.
	March 27th 2020, 12:36:13.659	s-rc-srv-02	rocketchat_rocketchat_2_1	Future node.js versions will automatically exit the process
	March 27th 2020, 12:36:13.658	s-rc-srv-02	rocketchat_rocketchat_2_1	}
	March 27th 2020, 12:36:13.658	s-rc-srv-02	rocketchat_rocketchat_2_1	---------------------------------
	March 27th 2020, 12:36:13.658	s-rc-srv-02	rocketchat_rocketchat_2_1	[Symbol(mongoErrorContextSymbol)]: {}
	March 27th 2020, 12:36:13.657	s-rc-srv-02	rocketchat_rocketchat_2_1	},
	March 27th 2020, 12:36:13.655	s-rc-srv-02	rocketchat_rocketchat_2_1	}
	March 27th 2020, 12:36:13.654	s-rc-srv-02	rocketchat_rocketchat_2_1	lastOp: [Object]
	March 27th 2020, 12:36:13.653	s-rc-srv-02	rocketchat_rocketchat_2_1	nModified: 0,
	March 27th 2020, 12:36:13.653	s-rc-srv-02	rocketchat_rocketchat_2_1	upserted: [],
	March 27th 2020, 12:36:13.653	s-rc-srv-02	rocketchat_rocketchat_2_1	nRemoved: 0,
	March 27th 2020, 12:36:13.652	s-rc-srv-02	rocketchat_rocketchat_2_1	nMatched: 0,
	March 27th 2020, 12:36:13.652	s-rc-srv-02	rocketchat_rocketchat_2_1	nInserted: 0,
	March 27th 2020, 12:36:13.652	s-rc-srv-02	rocketchat_rocketchat_2_1	nUpserted: 0,
	March 27th 2020, 12:36:13.651	s-rc-srv-02	rocketchat_rocketchat_2_1	writeConcernErrors: [],
	March 27th 2020, 12:36:13.651	s-rc-srv-02	rocketchat_rocketchat_2_1	insertedIds: [Array],
	March 27th 2020, 12:36:13.650	s-rc-srv-02	rocketchat_rocketchat_2_1	writeErrors: [Array],
	March 27th 2020, 12:36:13.646	s-rc-srv-02	rocketchat_rocketchat_2_1	ok: 1,
	March 27th 2020, 12:36:13.645	s-rc-srv-02	rocketchat_rocketchat_2_1	},
	March 27th 2020, 12:36:13.645	s-rc-srv-02	rocketchat_rocketchat_2_1	}
	March 27th 2020, 12:36:13.645	s-rc-srv-02	rocketchat_rocketchat_2_1	result: BulkWriteResult {
	March 27th 2020, 12:36:13.645	s-rc-srv-02	rocketchat_rocketchat_2_1	result: {
	March 27th 2020, 12:36:13.644	s-rc-srv-02	rocketchat_rocketchat_2_1	userId: 'QFNqm28joYJqhmBCA',
	March 27th 2020, 12:36:13.644	s-rc-srv-02	rocketchat_rocketchat_2_1	ts: 2020-03-27T09:36:13.620Z
	March 27th 2020, 12:36:13.644	s-rc-srv-02	rocketchat_rocketchat_2_1	_id: 'tz4GPFEmF2NmwHpzY',
	March 27th 2020, 12:36:13.644	s-rc-srv-02	rocketchat_rocketchat_2_1	roomId: 'xnGcER23hDhCaXS6G',
	March 27th 2020, 12:36:13.644	s-rc-srv-02	rocketchat_rocketchat_2_1	messageId: 'Ewi50RwMhZLGEJfv6',
	March 27th 2020, 12:36:13.643	s-rc-srv-02	rocketchat_rocketchat_2_1	op: {
	March 27th 2020, 12:36:13.641	s-rc-srv-02	rocketchat_rocketchat_2_1	code: 11000,
	March 27th 2020, 12:36:13.641	s-rc-srv-02	rocketchat_rocketchat_2_1	index: 0,
	March 27th 2020, 12:36:13.641	s-rc-srv-02	rocketchat_rocketchat_2_1	errmsg: 'E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key: { : "xnGcER23hDhCaXS6G", : "QFNqm28joYJqhmBCA", : "Ewi50RwMhZLGEJfv6" }',
	March 27th 2020, 12:36:13.640	s-rc-srv-02	rocketchat_rocketchat_2_1	name: 'BulkWriteError',
	March 27th 2020, 12:36:13.640	s-rc-srv-02	rocketchat_rocketchat_2_1	driver: true,
	March 27th 2020, 12:36:13.640	s-rc-srv-02	rocketchat_rocketchat_2_1	},
	March 27th 2020, 12:36:13.640	s-rc-srv-02	rocketchat_rocketchat_2_1	err: {
	March 27th 2020, 12:36:13.639	s-rc-srv-02	rocketchat_rocketchat_2_1	ts: 2020-03-27T09:36:13.620Z
	March 27th 2020, 12:36:13.639	s-rc-srv-02	rocketchat_rocketchat_2_1	userId: 'QFNqm28joYJqhmBCA',
	March 27th 2020, 12:36:13.639	s-rc-srv-02	rocketchat_rocketchat_2_1	messageId: 'Ewi50RwMhZLGEJfv6',
	March 27th 2020, 12:36:13.638	s-rc-srv-02	rocketchat_rocketchat_2_1	roomId: 'xnGcER23hDhCaXS6G',
	March 27th 2020, 12:36:13.634	s-rc-srv-02	rocketchat_rocketchat_2_1	op: {
	March 27th 2020, 12:36:13.634	s-rc-srv-02	rocketchat_rocketchat_2_1	code: 11000,
	March 27th 2020, 12:36:13.634	s-rc-srv-02	rocketchat_rocketchat_2_1	errmsg: 'E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key: { : "xnGcER23hDhCaXS6G", : "QFNqm28joYJqhmBCA", : "Ewi50RwMhZLGEJfv6" }',
	March 27th 2020, 12:36:13.634	s-rc-srv-02	rocketchat_rocketchat_2_1	_id: 'tz4GPFEmF2NmwHpzY',
	March 27th 2020, 12:36:13.633	s-rc-srv-02	rocketchat_rocketchat_2_1	at handler (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/topologies/replset.js:1155:22)
	March 27th 2020, 12:36:13.633	s-rc-srv-02	rocketchat_rocketchat_2_1	at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/pool.js:397:18
	March 27th 2020, 12:36:13.633	s-rc-srv-02	rocketchat_rocketchat_2_1	at processTicksAndRejections (internal/process/task_queues.js:75:11) {
	March 27th 2020, 12:36:13.633	s-rc-srv-02	rocketchat_rocketchat_2_1	index: 0,
	March 27th 2020, 12:36:13.632	s-rc-srv-02	rocketchat_rocketchat_2_1	at OrderedBulkOperation.handleWriteError (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/bulk/common.js:1112:11)
	March 27th 2020, 12:36:13.632	s-rc-srv-02	rocketchat_rocketchat_2_1	at resultHandler (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/bulk/common.js:501:23)
	March 27th 2020, 12:36:13.627	s-rc-srv-02	rocketchat_rocketchat_2_1	BulkWriteError: E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key: { : "xnGcER23hDhCaXS6G", : "QFNqm28joYJqhmBCA", : "Ewi50RwMhZLGEJfv6" }
	March 27th 2020, 12:36:13.626	s-rc-srv-02	rocketchat_rocketchat_2_1	=== UnHandledPromiseRejection ===
	March 27th 2020, 12:34:14.506	s-rc-srv-02	rocketchat_rocketchat_2_1	=================================
	March 27th 2020, 12:34:14.505	s-rc-srv-02	rocketchat_rocketchat_2_1	Future node.js versions will automatically exit the process
	March 27th 2020, 12:34:14.503	s-rc-srv-02	rocketchat_rocketchat_2_1	Setting EXIT_UNHANDLEDPROMISEREJECTION will cause the process to exit allowing your service to automatically restart the process
	March 27th 2020, 12:34:14.502	s-rc-srv-02	rocketchat_rocketchat_2_1	Errors like this can cause oplog processing errors.
	March 27th 2020, 12:34:14.502	s-rc-srv-02	rocketchat_rocketchat_2_1	---------------------------------
	March 27th 2020, 12:34:14.501	s-rc-srv-02	rocketchat_rocketchat_2_1	},
	March 27th 2020, 12:34:14.501	s-rc-srv-02	rocketchat_rocketchat_2_1	}
	March 27th 2020, 12:34:14.501	s-rc-srv-02	rocketchat_rocketchat_2_1	[Symbol(mongoErrorContextSymbol)]: {}
	March 27th 2020, 12:34:14.501	s-rc-srv-02	rocketchat_rocketchat_2_1	}
	March 27th 2020, 12:34:14.500	s-rc-srv-02	rocketchat_rocketchat_2_1	nRemoved: 0,
	March 27th 2020, 12:34:14.500	s-rc-srv-02	rocketchat_rocketchat_2_1	lastOp: [Object]
	March 27th 2020, 12:34:14.500	s-rc-srv-02	rocketchat_rocketchat_2_1	upserted: [],
	March 27th 2020, 12:34:14.498	s-rc-srv-02	rocketchat_rocketchat_2_1	nModified: 0,
	March 27th 2020, 12:34:14.495	s-rc-srv-02	rocketchat_rocketchat_2_1	nMatched: 0,
	March 27th 2020, 12:34:14.495	s-rc-srv-02	rocketchat_rocketchat_2_1	nInserted: 0,
	March 27th 2020, 12:34:14.495	s-rc-srv-02	rocketchat_rocketchat_2_1	nUpserted: 0,
	March 27th 2020, 12:34:14.494	s-rc-srv-02	rocketchat_rocketchat_2_1	writeErrors: [Array],
	March 27th 2020, 12:34:14.494	s-rc-srv-02	rocketchat_rocketchat_2_1	writeConcernErrors: [],
	March 27th 2020, 12:34:14.494	s-rc-srv-02	rocketchat_rocketchat_2_1	ok: 1,
	March 27th 2020, 12:34:14.494	s-rc-srv-02	rocketchat_rocketchat_2_1	insertedIds: [Array],
	March 27th 2020, 12:34:14.493	s-rc-srv-02	rocketchat_rocketchat_2_1	result: {
	March 27th 2020, 12:34:14.493	s-rc-srv-02	rocketchat_rocketchat_2_1	},
	March 27th 2020, 12:34:14.493	s-rc-srv-02	rocketchat_rocketchat_2_1	result: BulkWriteResult {
	March 27th 2020, 12:34:14.492	s-rc-srv-02	rocketchat_rocketchat_2_1	}
	March 27th 2020, 12:34:14.490	s-rc-srv-02	rocketchat_rocketchat_2_1	ts: 2020-03-27T09:34:14.457Z
	March 27th 2020, 12:34:14.489	s-rc-srv-02	rocketchat_rocketchat_2_1	roomId: 'A9ag6QqqTqYnKDnafsh2tfEBdr4nZoqSH5',
	March 27th 2020, 12:34:14.489	s-rc-srv-02	rocketchat_rocketchat_2_1	op: {
	March 27th 2020, 12:34:14.489	s-rc-srv-02	rocketchat_rocketchat_2_1	userId: 'A9ag6QqqTqYnKDnaf',
	March 27th 2020, 12:34:14.489	s-rc-srv-02	rocketchat_rocketchat_2_1	messageId: 'fSgY8fg4BL9zWTBxi',
	March 27th 2020, 12:34:14.489	s-rc-srv-02	rocketchat_rocketchat_2_1	_id: 'FAL7JejWpQcvCHKoK',
	March 27th 2020, 12:34:14.488	s-rc-srv-02	rocketchat_rocketchat_2_1	code: 11000,
	March 27th 2020, 12:34:14.488	s-rc-srv-02	rocketchat_rocketchat_2_1	errmsg: 'E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key: { : "A9ag6QqqTqYnKDnafsh2tfEBdr4nZoqSH5", : "A9ag6QqqTqYnKDnaf", : "fSgY8fg4BL9zWTBxi" }',
	March 27th 2020, 12:34:14.488	s-rc-srv-02	rocketchat_rocketchat_2_1	index: 0,
	March 27th 2020, 12:34:14.487	s-rc-srv-02	rocketchat_rocketchat_2_1	err: {
	March 27th 2020, 12:34:14.486	s-rc-srv-02	rocketchat_rocketchat_2_1	driver: true,
	March 27th 2020, 12:34:14.484	s-rc-srv-02	rocketchat_rocketchat_2_1	ts: 2020-03-27T09:34:14.457Z
	March 27th 2020, 12:34:14.484	s-rc-srv-02	rocketchat_rocketchat_2_1	},
	March 27th 2020, 12:34:14.484	s-rc-srv-02	rocketchat_rocketchat_2_1	name: 'BulkWriteError',
	March 27th 2020, 12:34:14.484	s-rc-srv-02	rocketchat_rocketchat_2_1	messageId: 'fSgY8fg4BL9zWTBxi',
	March 27th 2020, 12:34:14.483	s-rc-srv-02	rocketchat_rocketchat_2_1	op: {
	March 27th 2020, 12:34:14.483	s-rc-srv-02	rocketchat_rocketchat_2_1	_id: 'FAL7JejWpQcvCHKoK',
	March 27th 2020, 12:34:14.483	s-rc-srv-02	rocketchat_rocketchat_2_1	roomId: 'A9ag6QqqTqYnKDnafsh2tfEBdr4nZoqSH5',
	March 27th 2020, 12:34:14.483	s-rc-srv-02	rocketchat_rocketchat_2_1	userId: 'A9ag6QqqTqYnKDnaf',
	March 27th 2020, 12:34:14.482	s-rc-srv-02	rocketchat_rocketchat_2_1	code: 11000,
	March 27th 2020, 12:34:14.482	s-rc-srv-02	rocketchat_rocketchat_2_1	errmsg: 'E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key: { : "A9ag6QqqTqYnKDnafsh2tfEBdr4nZoqSH5", : "A9ag6QqqTqYnKDnaf", : "fSgY8fg4BL9zWTBxi" }',
	March 27th 2020, 12:34:14.481	s-rc-srv-02	rocketchat_rocketchat_2_1	index: 0,
	March 27th 2020, 12:34:14.478	s-rc-srv-02	rocketchat_rocketchat_2_1	at processTicksAndRejections (internal/process/task_queues.js:75:11) {
	March 27th 2020, 12:34:14.478	s-rc-srv-02	rocketchat_rocketchat_2_1	at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/pool.js:397:18
	March 27th 2020, 12:34:14.477	s-rc-srv-02	rocketchat_rocketchat_2_1	at resultHandler (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/bulk/common.js:501:23)
	March 27th 2020, 12:34:14.477	s-rc-srv-02	rocketchat_rocketchat_2_1	at OrderedBulkOperation.handleWriteError (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/bulk/common.js:1112:11)
	March 27th 2020, 12:34:14.477	s-rc-srv-02	rocketchat_rocketchat_2_1	at handler (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/topologies/replset.js:1155:22)
	March 27th 2020, 12:34:14.476	s-rc-srv-02	rocketchat_rocketchat_2_1	BulkWriteError: E11000 duplicate key error index: rocketchat.rocketchat_message_read_receipt.$roomId_1_userId_1_messageId_1 dup key: { : "A9ag6QqqTqYnKDnafsh2tfEBdr4nZoqSH5", : "A9ag6QqqTqYnKDnaf", : "fSgY8fg4BL9zWTBxi" }
	March 27th 2020, 12:34:14.465	s-rc-srv-02	rocketchat_rocketchat_2_1	=== UnHandledPromiseRejection ===
	March 27th 2020, 12:31:17.852	s-rc-srv-01	rocketchat_rocketchat_1_1	at handler (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/topologies/replset.js:1155:22)
	March 27th 2020, 12:31:17.852	s-rc-srv-01	rocketchat_rocketchat_1_1	at processTicksAndRejections (internal/process/task_queues.js:75:11)
	March 27th 2020, 12:31:17.852	s-rc-srv-01	rocketchat_rocketchat_1_1	at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/pool.js:397:18
	March 27th 2020, 12:31:17.849	s-rc-srv-01	rocketchat_rocketchat_1_1	at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/operations/collection_ops.js:1259:5
	March 27th 2020, 12:31:17.849	s-rc-srv-01	rocketchat_rocketchat_1_1	at handleCallback (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/utils.js:128:55)
	March 27th 2020, 12:31:17.848	s-rc-srv-01	rocketchat_rocketchat_1_1	at executeCallback (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/utils.js:395:17)
	March 27th 2020, 12:31:17.848	s-rc-srv-01	rocketchat_rocketchat_1_1	at deleteCallback (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/operations/collection_ops.js:349:17)
	March 27th 2020, 12:31:17.848	s-rc-srv-01	rocketchat_rocketchat_1_1	at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/operations/collection_ops.js:364:54
	March 27th 2020, 12:31:17.848	s-rc-srv-01	rocketchat_rocketchat_1_1	at ClientSession.endSession (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/sessions.js:134:41)
	March 27th 2020, 12:31:17.847	s-rc-srv-01	rocketchat_rocketchat_1_1	at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/gridfs-stream/index.js:188:25
	March 27th 2020, 12:31:17.847	s-rc-srv-01	rocketchat_rocketchat_1_1	at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/utils.js:410:17
	March 27th 2020, 12:31:17.847	s-rc-srv-01	rocketchat_rocketchat_1_1	at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/utils.js:398:11
	March 27th 2020, 12:31:17.846	s-rc-srv-01	rocketchat_rocketchat_1_1	Error: FileNotFound: no file with id JwNf9f6ePkM7tKM8M found
	March 27th 2020, 12:13:18.506	s-rc-srv-02	rocketchat_rocketchat_4_1	at processTicksAndRejections (internal/process/task_queues.js:75:11)
	March 27th 2020, 12:13:18.506	s-rc-srv-02	rocketchat_rocketchat_4_1	at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/operations/collection_ops.js:1259:5
	March 27th 2020, 12:13:18.506	s-rc-srv-02	rocketchat_rocketchat_4_1	at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/pool.js:397:18
	March 27th 2020, 12:13:18.506	s-rc-srv-02	rocketchat_rocketchat_4_1	at handler (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/topologies/replset.js:1155:22)
	March 27th 2020, 12:13:18.505	s-rc-srv-02	rocketchat_rocketchat_4_1	at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/operations/collection_ops.js:364:54
	March 27th 2020, 12:13:18.505	s-rc-srv-02	rocketchat_rocketchat_4_1	at handleCallback (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/utils.js:128:55)
	March 27th 2020, 12:13:18.503	s-rc-srv-02	rocketchat_rocketchat_4_1	at deleteCallback (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/operations/collection_ops.js:349:17)
	March 27th 2020, 12:13:18.503	s-rc-srv-02	rocketchat_rocketchat_4_1	at executeCallback (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/utils.js:395:17)
	March 27th 2020, 12:13:18.499	s-rc-srv-02	rocketchat_rocketchat_4_1	at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/gridfs-stream/index.js:188:25
	March 27th 2020, 12:13:18.499	s-rc-srv-02	rocketchat_rocketchat_4_1	at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/utils.js:410:17
	March 27th 2020, 12:13:18.499	s-rc-srv-02	rocketchat_rocketchat_4_1	at /app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/utils.js:398:11
	March 27th 2020, 12:13:18.499	s-rc-srv-02	rocketchat_rocketchat_4_1	at ClientSession.endSession (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/sessions.js:134:41)
	March 27th 2020, 12:13:18.498	s-rc-srv-02	rocketchat_rocketchat_4_1	Error: FileNotFound: no file with id HC2JgAwfeCAyfGEZ6 found`
@ankar84
Copy link
Author

ankar84 commented Mar 27, 2020

Temporary disabled Detailed Read Receipts.
I think it is a main issue area

@sampaiodiego
Copy link
Member

thanks for reporting @ankar84

as you mentioned, on #10061 the issue with message editings, do you happen to know what action causes the error you're seeing? even on the logs it is not clear what caused it.. if you know that will help a lot on finding a solution, thx

@ankar84
Copy link
Author

ankar84 commented Mar 28, 2020

thanks for reporting @ankar84

as you mentioned, on #10061 the issue with message editings, do you happen to know what action causes the error you're seeing? even on the logs it is not clear what caused it.. if you know that will help a lot on finding a solution, thx

Now true cause of that error is unknown. I did try to edit message after it was read by a few users and that error did not appear.
I will try to find what actually happened when that error appear in logs, because understand that it will help to handle that issue gratefully.

@emikolajczak
Copy link

emikolajczak commented Mar 30, 2020

Hi, check time synchro on all you servers. I had simillar issue when servers didn't synchronize time with NTP.

@dusatvoj
Copy link

I have same issue on 3.1.3. Is there anything I can help with? (Logs, etc.)

@sampaiodiego
Copy link
Member

@dusatvoj yes. are you able to reproduce it every time you want or it just happen some times? I still don't know what actions cause the error to happen

@dusatvoj
Copy link

@sampaiodiego I'm sorry, I don't know how to reproduce it :/ but as it's already written here I think it could be caused with message edit and detailed read receipt. I'm not a programmer but how it's handled by clients when message is edited?
I'm thinking about the case when a man write some message. I send a read receipt. After that is message edited and I read it again and send another read receipt.

@galshiff
Copy link
Contributor

galshiff commented Aug 2, 2020

Any update on this?
We have the same issue and it crashes the rocket chat instance...

@galshiff
Copy link
Contributor

galshiff commented Aug 8, 2020

Hi, our team successfully found fix for this bug!
Could someone approve my account, so ill be able to push a pull request in order to close this issue and fix this bug?

Thanks ;)

@sampaiodiego
Copy link
Member

that's good news @galshiff .. you just need to fork the repo and submit a pull request, no need to approval account.. see https://guides.github.com/activities/forking/

@ankar84
Copy link
Author

ankar84 commented Aug 18, 2020

Hi, our team successfully found fix for this bug!

Hi @galshiff
Do you know the way to reproduce that bug?

@sampaiodiego
Copy link
Member

@ankar84 pls see #18560

@sampaiodiego
Copy link
Member

Closed by #18560

@adrianovieira
Copy link

adrianovieira commented Jan 15, 2021

We also just get this error here.

What did we do so far... upgraded server from v-3.3.0 to v-3.10.4

And after the upgrade process this happened during the first start up of the rocketchat server .

logging:

 BulkWriteError: E11000 duplicate key error collection: rocketchat.rocketchat_room index: _id_ dup key: {
 	: "rocket.catsNYZbWWBmK3wDAAec"
 }
 at OrderedBulkOperation.handleWriteError(/opt/Rocket.Chat - 3.10 .4 / programs / server / npm / node_modules / meteor / npm - mongo / node_modules / mongodb / lib / bulk / common.js: 1257: 9)
 at resultHandler(/opt/Rocket.Chat - 3.10 .4 / programs / server / npm / node_modules / meteor / npm - mongo / node_modules / mongodb / lib / bulk / common.js: 521: 23)
 at / opt / Rocket.Chat - 3.10 .4 / programs / server / npm / node_modules / meteor / npm - mongo / node_modules / mongodb / lib / core / connection / pool.js: 405: 18
 at processTicksAndRejections(internal / process / task_queues.js: 79: 11) => awaited here:
 	at Promise.await(/opt/Rocket.Chat - 3.10 .4 / programs / server / npm / node_modules / meteor / promise / node_modules / meteor - promise / promise_server.js: 60: 12)
 at Server.apply(packages / ddp - server / livedata_server.js: 1638: 22)
 at Server.call(packages / ddp - server / livedata_server.js: 1607: 17)
 at DDPCommon.MethodInvocation. < anonymous > (server / lib / sendMessagesToAdmins.js: 16: 43)
 at packages / dispatch_run - as - user.js: 211: 14
 at Meteor.EnvironmentVariable.EVp.withValue(packages / meteor.js: 1234: 12)
 at Object.Meteor.runAsUser(packages / dispatch_run - as - user.js: 210: 33)
 at server / lib / sendMessagesToAdmins.js: 16: 12
 at SynchronousCursor.forEach(packages / mongo / mongo_driver.js: 1138: 16)
 at Cursor. < computed > [as forEach](packages / mongo / mongo_driver.js: 918: 44)
 at sendMessagesToAdmins(server / lib / sendMessagesToAdmins.js: 13: 33)
 at server / startup / migrations / v201.js: 41: 4
 at Function.time(/opt/Rocket.Chat - 3.10 .4 / programs / server / profile.js: 273: 30)
 at / opt / Rocket.Chat - 3.10 .4 / programs / server / boot.js: 415: 15
 at / opt / Rocket.Chat - 3.10 .4 / programs / server / boot.js: 465: 7
 at Function.run(/opt/Rocket.Chat - 3.10 .4 / programs / server / profile.js: 280: 14)
 at / opt / Rocket.Chat - 3.10 .4 / programs / server / boot.js: 463: 13 {
 	driver: true,
 	code: 11000,
 	writeErrors: [WriteError {
 		err: [Object]
 	}],
 	result: BulkWriteResult {
 		result: {
 			ok: 1,
 			writeErrors: [Array],
 			writeConcernErrors: [],
 			insertedIds: [Array],
 			nInserted: 0,
 			nUpserted: 0,
 			nMatched: 0,
 			nModified: 0,
 			nRemoved: 0,
 			upserted: [],
 			lastOp: [Object]
 		}
 	}
 }

I appears that all migrations has been applied:

 {"line":"120","file":"migrations.js","message":"Migrations: Migrating from version 191 -> 213","time":{"$date":1610733109193},"level":"info"}
 {"line":"120","file":"migrations.js","message":"Migrations: Running up() on version 192","time":{"$date":1610733109197},"level":"info"}
 {"line":"120","file":"migrations.js","message":"Migrations: Running up() on version 193","time":{"$date":1610733109738},"level":"info"}
 {"line":"120","file":"migrations.js","message":"Migrations: Running up() on version 194","time":{"$date":1610733109756},"level":"info"}
 {"line":"120","file":"migrations.js","message":"Migrations: Running up() on version 195","time":{"$date":1610733109772},"level":"info"}
 {"line":"120","file":"migrations.js","message":"Migrations: Running up() on version 196","time":{"$date":1610733109870},"level":"info"}
 {"line":"120","file":"migrations.js","message":"Migrations: Running up() on version 197","time":{"$date":1610733109876},"level":"info"}
 {"line":"120","file":"migrations.js","message":"Migrations: Running up() on version 198","time":{"$date":1610733109967},"level":"info"}
 {"line":"120","file":"migrations.js","message":"Migrations: Running up() on version 199","time":{"$date":1610733109994},"level":"info"}
 Changing schema of User records, this may take a long time ...
 Migrating 0/507
 Changing schema of User records finished.
 {"line":"120","file":"migrations.js","message":"Migrations: Running up() on version 200","time":{"$date":1610733111077},"level":"info"}
 {"line":"120","file":"migrations.js","message":"Migrations: Running up() on version 201","time":{"$date":1610733111147},"level":"info"}
 {"line":"120","file":"migrations.js","message":"Migrations: Running up() on version 202","time":{"$date":1610733111156},"level":"info"}
 {"line":"120","file":"migrations.js","message":"Migrations: Running up() on version 203","time":{"$date":1610733111175},"level":"info"}
 {"line":"120","file":"migrations.js","message":"Migrations: Running up() on version 204","time":{"$date":1610733111377},"level":"info"}
 {"line":"120","file":"migrations.js","message":"Migrations: Running up() on version 205","time":{"$date":1610733111392},"level":"info"}
 {"line":"120","file":"migrations.js","message":"Migrations: Running up() on version 206","time":{"$date":1610733111397},"level":"info"}
 {"line":"120","file":"migrations.js","message":"Migrations: Running up() on version 207","time":{"$date":1610733111422},"level":"info"}
 {"line":"120","file":"migrations.js","message":"Migrations: Running up() on version 208","time":{"$date":1610733111428},"level":"info"}
 {"line":"120","file":"migrations.js","message":"Migrations: Running up() on version 209","time":{"$date":1610733111435},"level":"info"}
 {"line":"120","file":"migrations.js","message":"Migrations: Running up() on version 210","time":{"$date":1610733111440},"level":"info"}
 {"line":"120","file":"migrations.js","message":"Migrations: Running up() on version 211","time":{"$date":1610733111479},"level":"info"}
 {"line":"120","file":"migrations.js","message":"Migrations: Running up() on version 212","time":{"$date":1610733112934},"level":"info"}
 {"line":"120","file":"migrations.js","message":"Migrations: Running up() on version 213","time":{"$date":1610733112947},"level":"info"}
 {"line":"120","file":"migrations.js","message":"Migrations: Finished migrating.","time":{"$date":1610733113066},"level":"info"}
 Not migrating, control is locked. Attempt 1/30. Trying again in 10 seconds.
 {"line":"120","file":"migrations.js","message":"Migrations: Not migrating, already at version 213","time":{"$date":1610733123079},"level":"info"}

@ankar84
Copy link
Author

ankar84 commented Jan 16, 2023

@sampaiodiego hey, Diego!
Looks like that bug reappeared in 5.4.1

@ankar84 ankar84 reopened this Jan 16, 2023
@sampaiodiego
Copy link
Member

@matheusbsilva137

@trogwarz
Copy link

It's very odd to bump this but i still got it quite often on version 6.6.3 in March 2024. Which also just kills my server for a while o people can't chat anymore.

@RAurelian
Copy link

we have the same error as well

@grzebo
Copy link

grzebo commented Mar 20, 2024

Here's a solution that fixes this bug:

#26259 (comment)

Before running the commands provided there you should launch MongoDB shell with the command mongosh and then switch to rocketchat database by running use rocketchat

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

9 participants