Bluebird remove return, reflect, fromCallback

[maximelkin]

No description provided.

[kibertoad]

It seems to be failing on Node 12 consistently.

[maximelkin]

Hm, strange errors, I will research this

[maximelkin]

So error got passed through try-catch, really strange

Error: This socket has been ended by the other party
    at Socket.writeAfterFIN [as write] (net.js:441:14)
    at Connection.write (/home/travis/build/knex/knex/node_modules/mysql2/lib/connection.js:217:17)
    at Connection.writePacket (/home/travis/build/knex/knex/node_modules/mysql2/lib/connection.js:262:12)
    at Quit.start (/home/travis/build/knex/knex/node_modules/mysql2/lib/commands/quit.js:24:16)
    at Quit.execute (/home/travis/build/knex/knex/node_modules/mysql2/lib/commands/command.js:39:22)
    at Connection.handlePacket (/home/travis/build/knex/knex/node_modules/mysql2/lib/connection.js:408:32)
    at Connection.addCommand (/home/travis/build/knex/knex/node_modules/mysql2/lib/connection.js:430:12)
    at Connection.end (/home/travis/build/knex/knex/node_modules/mysql2/lib/connection.js:796:26)
    at internal/util.js:278:30
    at new Promise (<anonymous>)
    at bound end (internal/util.js:277:12)
    at Client_MySQL2.destroyRawConnection (/home/travis/build/knex/knex/lib/dialects/mysql/index.js:85:20)

[maximelkin]

connection.end called after last command fail, which cleared all events, so 'error' event emitted to nowhere and emitted as unhandled error

[maximelkin]

nodejs/node#24111

[maximelkin]

Before node12

1. connection.end called
2. also error emit in 1) synchroniously
3. catch
4. finally (removeAllListeners)

Was

1. connection.end called (process.nextTick error triggered because closed)
2. Bluebird.catch
3. error emitted
4. Bluebird.finally (removeAllListeners)

Now

1. connection.end (process.nextTick error triggered)
2. catch (e) {}
3. finally {} (removeAllListeners)
4. error emitted

Bluebird solution worked because of additional microtask from .catch?

[maximelkin]

@kibertoad have you any ideas to resolve this?

[kibertoad]

@maximelkin Can we add additional listener to catch the error when it is thrown asynchronously?

[maximelkin]

@kibertoad can this trigger leaks? When to unsubscribe this listener?

[maximelkin]

7069ce5 <- here removeAllListeners added

[maximelkin]

@kibertoad that do you think?

[elhigu]

Dropping that removeAllListeners call seems wrong.

[elhigu]

Is that finally removeAllListeners really unnecessary? Is there test somewhere, which fails if connection has listeners after disposal?

[maximelkin]

I run knex-stress-test for a some minutes (with process.memoryUsage().heapUsed logging)
memory usage is stable, so no leak introduced

[elhigu]

Sounds promising. Did you try it with both mysql and mysql2 drivers?

Though if that listener removal is not necessary, shouldn't this function just return end() and leave it to caller to set __knex__disposed like in every other driver 🤔

One more thing... if that Error: This socket has been ended by the other party is now not leaked, I suppose that error is still happening, but it is ignored somewhere (by existence of some event handler maybe). So when it is ignored how does that get recorded to connection.__knex__disposed or does it even matter...

Also I'm pretty confused how is it possible that error is leaking and where that leak is happening exactly (stack trace showed lib/dialects/mysql/index.js:85:20, but then it should have been handled by caller's error handler). And when it is leaked why caller's error handling is not catching it? That was an async function after all.

This is a part of really fundamental functionality of knex and may cause hard to catch failures for huge user base if it is not working properly.I would really like to understand the problem properly before trying to patch it.

[elhigu]

Can I easily reproduce the original error to investigate it a bit more? Did it happen in some very specific test or every time?

[maximelkin]

Install 12 node
Add finally block with removeAllListeners
run test "#2321 dead connections are not evicted from pool"

[maximelkin]

Dropping that removeAllListeners call seems wrong.

@elhigu why?

[elhigu]

Dropping that removeAllListeners call seems wrong.

@elhigu why?

Because its addition didn't look like an accident and there was not proper investigation done, if it can be removed or not.

[maximelkin]

As temporal solution we can add delay before removeAllListeners, so it would work like before

[kibertoad]

@maximelkin I believe https://lodash.com/docs/4.17.14#defer does exactly this?

[maximelkin]

@kibertoad yes

[maximelkin]

But i still think what we should review removeAllListeners usage, maybe in separate issue

[maximelkin]

@elhigu @kibertoad done