fix: separate signals for passive, active, and forced shutdown

[kylecarbs]

This is from a customer request to gracefully drain instances instead of forcefully killing the jobs.

SIGTERM: Passive shutdown stopping provisioner daemons from accepting new
jobs but waiting for existing jobs to be successfully completed.

SIGINT (old existing behavior): Notify provisioner daemons to cancel in-flight jobs, wait for 5s for jobs to be exited, then force quit.

SIGKILL: Untouched from before, will force-quit.

[mafredri]

I like there being an option to let in-flight jobs complete, but I think it would be worthwhile to consolidate around one shutdown method. (I'm fine with wait for jobs being the default.)

Also, I think we should merge back "interrupt" and "stop" signal slices so that we don't break other commands.

If we still want interrupt in coderd to be "faster" (only useful for development, really?), we could use two additional slices.

var StopSignals = []os.Signal{ // new default (prev. interruptsignals renamed)
	os.Interrupt,
	syscall.SIGTERM,
	syscall.SIGHUP,
}

var StopSignalsNoInterrupt = []os.Signal{
	syscall.SIGTERM,
	syscall.SIGHUP,
}
var InterruptSignals = []os.Signal{
	os.Interrupt,
}

That's my proposal, also moved HUP since there's no inherent reason for HUP to use the fast-sequence either.

(We don't even need to take HUP as a stop signal, but it can be useful for development. If we don't use it for stop we should probably ignore it though to protect users from abrupt shutdown.)

[mafredri]

I feel both methods are graceful, so perhaps a rename here is appropriate?

Suggested change

	graceful = true
	waitForProvisionerJobs = true

[kylecarbs]

I think that's reasonable.

[mafredri]

Here we're changing the default behavior for interrupt, and since we split interrupt and stop, we're also not handling SIGTERM here anymore.

[kylecarbs]

Fixed

[mafredri]

This is a breaking change in all CLI commands that handle signals. They used to handle TERM but no longer do.

[aaronlehmann]

It would be really fantastic to get this finished and included in a release! That would make deployments much safer since they wouldn't risk interrupting the provisioner.

[mafredri]

LGTM! 🎉

[mafredri]

I love how Windows leaves you SOL. 🥲

[kylecarbs]

No doubt!