Any CRUD statement execution may have rare hang case

[jack-chui]

Do you want to request a feature or report a bug?
bug report

What is the current behavior?
I am using Mongoose in AWS Lambda and using the following code to connect with MongoDB

let conn = null;

const connectionDB = async (logger) => {
    logger.writeActionLog("MongoDB", "Start Connection");

    if (conn == null) {
        conn = mongoose.createConnection(uri, {
            useNewUrlParser: true,
            useFindAndModify: false,
            useUnifiedTopology: true,

            bufferCommands: false, // Disable mongoose buffering
            bufferMaxEntries: 0 // and MongoDB driver buffering
        });

        await conn;

        for (const [key, value] of Object.entries(CollectionName)) {
            conn.model(value, require(`../../schema/${value}`));
        }
    } else {
        logger.writeActionLog("MongoDB", "Used cache");
    }

    logger.writeActionLog("MongoDB", "Start Success");
}

let getCollection = async (name) => {
    return conn.model(name);
}

module.exports = {
    CollectionName,
    connectionDB,
    getCollection,
    ObjectId
}

When I execute any CRUD statement (e.g. aggregate), Mongoose can return result to in < 1s. However, it has a rare hanging case without any exception or error in the same CRUD. Even I wait it in 900s, it still hang.

If the current behavior is a bug, please provide the steps to reproduce.

1. Use the above connectionDB() function
2. getCollection and execute CRUD

What is the expected behavior?
All CRUD statement should be always work.

What are the versions of Node.js, Mongoose and MongoDB you are using? Note that "latest" is not a version.
mongoose: ^5.10.9
Node.js 12.x

[bhrigushr]

Issue may be related with #9179

[jack-chui]

Issue may be related with #9179

It seem that it is not the same issue. #9179 still can obtain the result. But I cannot obtain any result or error if CRUD statement hang.

[vkarpov15]

@SinLok just to confirm, have you tried waiting for 10 seconds? Also, try reducing heartbeatFrequencyMS to 1000 and see if that helps.

[jack-chui]

@SinLok just to confirm, have you tried waiting for 10 seconds? Also, try reducing heartbeatFrequencyMS to 1000 and see if that helps.

I tried to set AWS Lambda timeout to 900s and added 2 console.log surrounding Mongoose aggregate function. After 900s, I saw AWS print timeout to me and only the first console.log output if the aggregate hanging.

For heartbeatFrequencyMS, I will try it later.

[jack-chui]

@SinLok just to confirm, have you tried waiting for 10 seconds? Also, try reducing heartbeatFrequencyMS to 1000 and see if that helps.

Using heartbeatFrequencyMS 1000, the problem still exist.

[vkarpov15]

Try reducing serverSelectionTimeoutMS to 1000:

conn = mongoose.createConnection(uri, {
            useNewUrlParser: true,
            useFindAndModify: false,
            useUnifiedTopology: true,

            serverSelectionTimeoutMS: 1000, // <-- Add this
            heartbeatFrequencyMS: 1000,

            bufferCommands: false, // Disable mongoose buffering
            bufferMaxEntries: 0 // and MongoDB driver buffering
        });

That should help you get an error message. It might be that Mongoose isn't able to connect for some reason, and is stuck in a server selection loop.

[jack-chui]

Try reducing serverSelectionTimeoutMS to 1000:

conn = mongoose.createConnection(uri, {
            useNewUrlParser: true,
            useFindAndModify: false,
            useUnifiedTopology: true,

            serverSelectionTimeoutMS: 1000, // <-- Add this
            heartbeatFrequencyMS: 1000,

            bufferCommands: false, // Disable mongoose buffering
            bufferMaxEntries: 0 // and MongoDB driver buffering
        });

That should help you get an error message. It might be that Mongoose isn't able to connect for some reason, and is stuck in a server selection loop.

I am getting 3 types of error:

1. Task timed out after 10.01 seconds <- this comes from AWS Lambda timeout since CRUD statement hang
2. Server selection timed out after 1000 ms
3. MongooseServerSelectionError: connection <monitor> to **_MY SERVER IP_** timed out\\n at Function.Model.$wrapCallback (\/var\/task\/node_modules\/mongoose\/lib\/model.js:4856:32)\\n at \/var\/task\/node_modules\/mongoose\/lib\/query.js:4393:21\\n at \/var\/task\/node_modules\/mongoose\/lib\/helpers\/promiseOrCallback.js:31:5\\n at new Promise (<anonymous>)\\n at promiseOrCallback (\/var\/task\/node_modules\/mongoose\/lib\/helpers\/promiseOrCallback.js:30:10)\\n at model.Query.exec (\/var\/task\/node_modules\/mongoose\/lib\/query.js:4392:10)\\n at model.Query.Query.then (\/var\/task\/node_modules\/mongoose\/lib\/query.js:4473:15)\\n at processTicksAndRejections (internal\/process\/task_queues.js:97:5)

Btw, I am using MongoDB atlas as my DB server

[vkarpov15]

Try setting socketTimeoutMS to 5000:

conn = mongoose.createConnection(uri, {
            useNewUrlParser: true,
            useFindAndModify: false,
            useUnifiedTopology: true,

            serverSelectionTimeoutMS: 1000,
            heartbeatFrequencyMS: 1000,
            socketTimeoutMS: 5000, // <-- Add this

            bufferCommands: false, // Disable mongoose buffering
            bufferMaxEntries: 0 // and MongoDB driver buffering
        });

The "connection timed out" error is often indicative of a very slow query timing out the socket.

Another alternative: there is a slight race condition in your connectionDB() function. If you call connectionDB() twice in parallel, the 2nd call won't wait for the connection to be established. I don't think that is causing this issue, but, just to be safe, try putting:

    if (conn == null) {
        conn = mongoose.createConnection(uri, {
            useNewUrlParser: true,
            useFindAndModify: false,
            useUnifiedTopology: true,

            bufferCommands: false, // Disable mongoose buffering
            bufferMaxEntries: 0 // and MongoDB driver buffering
        });

        await conn;

        for (const [key, value] of Object.entries(CollectionName)) {
            conn.model(value, require(`../../schema/${value}`));
        }
    } else {
        logger.writeActionLog("MongoDB", "Used cache");

        await conn; // <-- add this
    }