Fix error in k8s after job completes

[DrJosh9000]

Description

There's a spurious error in some k8s job logs:

🚨 Error: Error waiting for client interrupt: context canceled

This happens when a job is ending normally. The cause is the surrounding context is being closed, and the goroutine that logs it then races the closure of the log writer.

While I'm here, I noticed that cancelCh is unbuffered, but there is a mismatch between sends and receives which would lead to leaking the goroutine that logs the message. So I fixed that by using a "close-with-guard pattern" (that's what I'm calling it, anyway).

Context

Noticed a bug, and it was pretty easy to fix.

Changes

• Don't log the error if the context was cancelled
• Add some explanatory comments
• Change the cancelCh usage from sending to closing

Testing

• Tests have run locally (with go test ./...). Buildkite employees may check this if the pipeline has run automatically.
• Code is formatted (with go fmt ./...)

[zhming0]

🚀 ! 👍🏿 for all these descriptive code comments.