Skip to content
This repository has been archived by the owner on Jul 10, 2023. It is now read-only.

Handle ECONNRESET error when keepAlive connection is terminated by … #87

Merged
merged 2 commits into from
Jul 22, 2019

Conversation

kobalski
Copy link
Contributor

…the server. In a long-running lambda container keep-alive connections are reset by our collector servers due to nginx config for keepalive_timeout is 30 seconds. If a request fails due to this error, we do not log the error but we try to send the data again otherwise we miss monitoring data.

if (err.code === 'ECONNRESET') {
ThundraLogger.getInstance().debug(
'Keep Alive connection reset by server. Will send monitoring data again.');
await this.sendReports();
Copy link
Member

@serkan-ozal serkan-ozal Jul 20, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is better to try only once. Not sure whether it is possible in practice, but in theory, this might result infinite recursion by calling sendReports recursively. To prevent this:

async sendReports(dontRetry: boolean): Promise<void> {
    ...
    await Promise.all(reportPromises).catch(async (err) => {
         if (dontRetry != true && err.code === 'ECONNRESET') {
             ThundraLogger.getInstance().debug('Connection reset by server. Will send monitoring data again.');
             await this.sendReports(true);
             return;
         }
         ThundraLogger.getInstance().error(err);
    });
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I included connectionRetryCount to Reporter to make sure we try to reconnect to server only once.

@coveralls
Copy link

Pull Request Test Coverage Report for Build 561

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 24 unchanged lines in 1 file lost coverage.
  • Overall coverage decreased (-0.1%) to 69.955%

Files with Coverage Reduction New Missed Lines %
dist/Reporter.js 24 59.09%
Totals Coverage Status
Change from base Build 556: -0.1%
Covered Lines: 2242
Relevant Lines: 2977

💛 - Coveralls

@serkan-ozal serkan-ozal merged commit 41c2b5c into master Jul 22, 2019
@kobalski kobalski deleted the fix/handle-econnreset branch July 24, 2019 05:29
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants