Configure terminus

[onordgren]

Hello! We're running directus in k8s and would like to be able to control the timeout period in the terminus handler. From the source code it looks like it's hard coded at 1000ms. Is there some other recommended way of doing this, was thinking of maybe doing it in the server.stop hook. Or would you consider adding a configuration option to be able to set the terminus timeout?

[rijkvanzanten]

@onordgren I think we can/should make that configurable from an environment variable. Any ideas / recommendations on a good name for that? I'd rather not tie it to terminus directly, as we might swap that out over time (eg keep the env var name a little more generic). Maybe SERVER_SHUTDOWN_TIMEOUT?

For those reading along, the values are here:

directus/api/src/server.ts

Lines 84 to 90 in 276cc47

	const terminusOptions: TerminusOptions = {
	timeout: 1000,
	signals: ['SIGINT', 'SIGTERM', 'SIGHUP'],
	beforeShutdown,
	onSignal,
	onShutdown,
	};

[onordgren]

Ah yea, we are currently using that one but from what I can see it will not return 503 during shutdown. Might be missing something though.

[onordgren]

Maybe there should be a testShutdown or similar here?

checks: merge(
  ...(await Promise.all([testDatabase(), testCache(), testRateLimiter(), testStorage(), testEmail()]))
),

[rijkvanzanten]

I'm not quite sure what the question is.. You're saying that /health should return 503 during shutdowns? I don't understand what you mean with the testShutdown example

[onordgren]

Terminus exposes a /healthcheck end point by default, see below copy paste from their read me

  healthChecks: {
    '/healthcheck': healthCheck,    // a function accepting a state and returning a promise indicating service health,
    ...
  },
  ...
  statusOk,                         // [optional = 200] status to be returned for successful healthchecks
  statusError,                      // [optional = 503] status to be returned for unsuccessful healthchecks

This endpoint will return 200 by default but once the shutdown has been triggered it will return 503 during the timeout. During the timeout the app itself can still handle regular traffic but this end point is used to tell k8s that the app is no longer healthy.

I don't think that the directus /health end point takes this into account so it cannot be used as a way to determine if the app is shutting down or not. One possible solution would be to add a handler like testShutdown to it that checks if the app is shutting down or not, kind of what terminus does on /healthcheck.

[rijkvanzanten]

Ah gotcha! Yeah that's a good thing to resolve on the /health endpoint. I'd rather not add a secondary /healthcheck endpoint. Created an issue to track here: #17241

[rijkvanzanten]

This feature request has been accepted and is queued to be implemented! You can follow along with the progress here: #17239

[onordgren]

Awesome! Thx for the swift feedback.

[rijkvanzanten]

Awesome! Thx for the swift feedback.

Super easy tweak, big QOL improvement for k8s use 🙂