Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

envoy: exit if envoy exits #2240

Merged
merged 1 commit into from
May 27, 2021
Merged

Conversation

calebdoxsey
Copy link
Contributor

@calebdoxsey calebdoxsey commented May 26, 2021

Summary

If envoy exits unexpectedly, we should use log.Fatal to exit as well. This PR also fixes a bug with hot-restart. The restart-epoch was supposed to start at 1 for the first restart, not 0.

Related issues

Fixes https://github.com/pomerium/internal/issues/414

Checklist

  • reference any related issues
  • updated docs
  • updated unit tests
  • updated UPGRADING.md
  • add appropriate tag (improvement / bug / etc)
  • ready for review

@calebdoxsey calebdoxsey added the bug Something isn't working label May 26, 2021
@calebdoxsey calebdoxsey requested a review from a team as a code owner May 26, 2021 16:51
@calebdoxsey calebdoxsey requested a review from wasaga May 26, 2021 16:51
@codeclimate
Copy link

codeclimate bot commented May 26, 2021

Code Climate has analyzed commit 53c22e8 and detected 0 issues on this pull request.

View more on Code Climate.

@desimone desimone requested a review from travisgroth May 27, 2021 18:11
Copy link
Contributor

@wasaga wasaga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm;

I'd remove monitorProcessCancel from Server but it's a style preference.

} else {
args = append(args, "--use-dynamic-base-id", "--base-id-path", baseIDPath)
}
srv.restartEpoch++

cmd := exec.Command(srv.envoyPath, args...) // #nosec
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

=> exec.CommandContext for consistency

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want the background context. The context passed to run comes from the on change handler. But envoy needs to survive past the end of this function.

@@ -213,6 +218,13 @@ func (srv *Server) run(ctx context.Context) error {
if err != nil {
return fmt.Errorf("error starting envoy: %w", err)
}
// call Wait to avoid zombie processes
go func() { _ = cmd.Wait() }()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure why monitorProcessCancel has to be inside Server - you can just wait for the process completion in this goroutine, and invoke monitorProcessCancel there.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we update envoy config we start a new envoy process using hot-reload. Once hot-reload completes it will kill the previous envoy process. If we don't cancel the previous process monitor, we will immediately exit when this happens.
We have to cancel the process monitor before the process exits.

@calebdoxsey calebdoxsey merged commit ef62d9b into master May 27, 2021
@calebdoxsey calebdoxsey deleted the cdoxsey/414-exit-if-envoy-exits branch May 27, 2021 21:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants