Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logs from waitForAPI stop being output after first time.Sleep call #31

Open
stbenjam opened this issue Jul 16, 2019 · 0 comments
Open

Comments

@stbenjam
Copy link
Member

While waiting for the Ironic API to come up, there's a for loop that tries the API, and on failure sleeps for 5 seconds and then tries again until the specified timeout is reached:

for tries := 1; tries <= maxTries; tries++ {
// FIXME(stbenjam): After we sleep at the end of the loop, *something* is changing log output to be discarded, and
// when we get woken up again, our logs aren't printed anymore until the end of the loop. Very odd error, possibly
// something in terraform is doing this.
log.Printf("[DEBUG] Trying to connect to API, attempt %d of %d\n", tries, maxTries)
r, err := httpClient.Get(client.Endpoint)
if err == nil {
statusCode := r.StatusCode
r.Body.Close()
if statusCode == http.StatusOK {
log.Printf("[DEBUG] API successfully connected.")
return nil
}
}
time.Sleep(5 * time.Second)
}

The logging in that for loop is only output once while inside openshift/installer. All the output occurs when running the tests, so I think it's something that happens while terraform is being executed in the openshift/installer.

The way that time.Sleep works is that the goroutine gets blocked and the Go scheduler wakes it up after the specified duration. It appears that something is setting log output to ioutil.Discard or similar while the goroutine is asleep. Haven't figured out how to set breakpoints in the installer's invocation of terraform yet, which would probably help here. I see a few calls to log.SetOutput(ioutil.Discard) in terrraform and openshift/installer but not any obvious ways those code paths would be taken while the goroutine is asleep.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant