Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: graceful shutdown timeout #2422

Merged
merged 10 commits into from
Nov 26, 2023
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,10 @@ If everything is set up correctly, you can access the healthcheck on `http://loc
For more information, [see docs](https://docs.nestjs.com/recipes/terminus).
You can find more samples in the [samples/](https://github.com/nestjs/terminus/tree/master/sample) folder of this repository.

### Graceful shutdown timeout

If your application requires postponing its shutdown process, this can be done by configuring the `gracefulShutdownTimeoutMs` in the `TerminusModule options`. This setting can prove particularly beneficial when working with an orchestrator such as Kubernetes. By setting a delay slightly longer than the readiness check interval, you can achieve zero downtime when shutting down containers.

## Contribute

In order to get started, first read through our [Contributing guidelines](https://github.com/nestjs/terminus/blob/master/CONTRIBUTING.md).
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
import { Test } from '@nestjs/testing';
import { LoggerService } from '@nestjs/common';
import {
GracefulShutdownService,
TERMINUS_GRACEFUL_SHUTDOWN_TIMEOUT,
} from './graceful-shutdown-timeout.service';
import { TERMINUS_LOGGER } from '../health-check/logger/logger.provider';
import { sleep } from '../utils';

jest.mock('../utils', () => ({
sleep: jest.fn(),
}));

const loggerMock: Partial<LoggerService> = {
log: jest.fn(),
error: jest.fn(),
warn: jest.fn(),
debug: jest.fn(),
};

describe('GracefulShutdownService', () => {
let service: GracefulShutdownService;
let logger: LoggerService;

beforeEach(async () => {
const module = await Test.createTestingModule({
providers: [
GracefulShutdownService,
{ provide: TERMINUS_LOGGER, useValue: loggerMock },
{ provide: TERMINUS_GRACEFUL_SHUTDOWN_TIMEOUT, useValue: 1000 },
],
}).compile();

logger = module.get(TERMINUS_LOGGER);
service = module.get(GracefulShutdownService);
});

it('should not trigger sleep if signal is not SIGTERM', async () => {
await service.beforeApplicationShutdown('SIGINT');
expect(sleep).not.toHaveBeenCalled();
});

it('should trigger sleep if signal is SIGTERM', async () => {
await service.beforeApplicationShutdown('SIGTERM');
expect(sleep).toHaveBeenCalledWith(1000);
});
});
42 changes: 42 additions & 0 deletions lib/graceful-shutdown-timeout/graceful-shutdown-timeout.service.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
import {
type BeforeApplicationShutdown,
ConsoleLogger,
Inject,
Injectable,
LoggerService,
} from '@nestjs/common';
import { TERMINUS_LOGGER } from '../health-check/logger/logger.provider';
import { sleep } from '../utils';

export const TERMINUS_GRACEFUL_SHUTDOWN_TIMEOUT =
'TERMINUS_GRACEFUL_SHUTDOWN_TIMEOUT';

/**
* Handles Graceful shutdown timeout useful to await
* for some time before the application shuts down.
*/
@Injectable()
export class GracefulShutdownService implements BeforeApplicationShutdown {
constructor(
@Inject(TERMINUS_LOGGER)
private readonly logger: LoggerService,
@Inject(TERMINUS_GRACEFUL_SHUTDOWN_TIMEOUT)
private readonly gracefulShutdownTimeoutMs: number,
) {
if (this.logger instanceof ConsoleLogger) {
this.logger.setContext(GracefulShutdownService.name);
}
}

async beforeApplicationShutdown(signal: string) {
this.logger.log(`Received termination signal ${signal}`);

if (signal === 'SIGTERM') {
this.logger.log(
`Awaiting ${this.gracefulShutdownTimeoutMs}ms before shutdown`,
);
await sleep(this.gracefulShutdownTimeoutMs);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BrunnerLivio you can just:

import { setTimeout } from 'node:timers/promises';

await setTimeout(this.gracefulShutdownTimeoutMs);

see: https://nodejs.org/api/timers.html#timerspromisessettimeoutdelay-value-options

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BrunnerLivio sorry, wrong mention... the author of the PR is @Lp-Francois

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIL, didn't know this API exist :)

this.logger.log(`Timeout reached, shutting down now`);
}
}
}
20 changes: 20 additions & 0 deletions lib/terminus-options.interface.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,27 @@ import { type LoggerService, type Type } from '@nestjs/common';

export type ErrorLogStyle = 'pretty' | 'json';

/**
* The Terminus module options
*
* errorLogStyle: The style of the error logger. Either 'pretty' or 'json'. Default to 'json'.
* logger: The logger to use. Either default logger or your own.
* gracefulShutdownTimeoutMs: The timeout to wait in ms before the application shuts down. Default to 0ms.
* @publicApi
*/
export interface TerminusModuleOptions {
Lp-Francois marked this conversation as resolved.
Show resolved Hide resolved
/**
* The style of the error logger
* @default 'json'
*/
errorLogStyle?: ErrorLogStyle;
Lp-Francois marked this conversation as resolved.
Show resolved Hide resolved
/**
* The logger to use. Either default logger or your own.
*/
logger?: Type<LoggerService> | boolean;
Lp-Francois marked this conversation as resolved.
Show resolved Hide resolved
/**
* The timeout to wait in ms before the application shuts down
* @default 0
*/
gracefulShutdownTimeoutMs?: number;
Lp-Francois marked this conversation as resolved.
Show resolved Hide resolved
}
11 changes: 10 additions & 1 deletion lib/terminus.module.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { type DynamicModule, Module } from '@nestjs/common';
import { TERMINUS_GRACEFUL_SHUTDOWN_TIMEOUT } from './graceful-shutdown-timeout/graceful-shutdown-timeout.service';
import { HealthCheckService } from './health-check';
import { getErrorLoggerProvider } from './health-check/error-logger/error-logger.provider';
import { ERROR_LOGGERS } from './health-check/error-logger/error-loggers.provider';
Expand Down Expand Up @@ -30,14 +31,22 @@ const exports_ = [HealthCheckService, ...HEALTH_INDICATORS];
})
export class TerminusModule {
static forRoot(options: TerminusModuleOptions = {}): DynamicModule {
const { errorLogStyle = 'json', logger = true } = options;
const {
errorLogStyle = 'json',
logger = true,
gracefulShutdownTimeoutMs = 0,
} = options;

return {
module: TerminusModule,
providers: [
...providers,
getErrorLoggerProvider(errorLogStyle),
getLoggerProvider(logger),
{
provide: TERMINUS_GRACEFUL_SHUTDOWN_TIMEOUT,
useValue: gracefulShutdownTimeoutMs,
},
],
exports: exports_,
};
Expand Down
1 change: 1 addition & 0 deletions lib/utils/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@ export * from './promise-timeout';
export * from './checkPackage.util';
export * from './types';
export * from './is-error';
export * from './sleep';
2 changes: 2 additions & 0 deletions lib/utils/sleep.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
export const sleep = (ms: number) =>
new Promise((resolve) => setTimeout(resolve, ms));