Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved speed of the test harness for testing net/http parsing by adding a call to runtime.Gosched() before shutting down the server #63

Merged
merged 1 commit into from Aug 12, 2020

Conversation

mattiasgrenfeldt
Copy link
Contributor

@mattiasgrenfeldt mattiasgrenfeldt commented Aug 12, 2020

Fixes #59

Added a call to runtime.Gosched() before the call to server.Shutdown().

This dramatically improved the time it took to run the tests. It turns out that server.Shutdown polls every 0.5 s if all idle connections are closed. When we call server.Shutdown, our connection isn't closed just yet, so therefore we always have to wait 0.5 s for each test. But in reality it takes a lot less time than 0.5 s for the connection to finish closing. So if we just give up execution for a moment before calling server.Shutdown then the other goroutines have the time to shut the connection down before server.Shutdown checks if everything is closed. And then we don't get punished with waiting for 0.5 s every time we try to shut down.

With this approach sometimes tests still take 0.5 s to run. This might be because the scheduler continues execution on the same goroutine faster than the time it takes to close the connection. Putting in a time.Sleep(100 * time.Microsecond) instead of the runtime.Gosched() consistently makes the tests run faster, on my machine at least. But that is something I'll leave to be investigated at another time.

Added a call to runtime.Gosched() before the call to server.Shutdown(). This drammatically improved the time it took to run the tests. It turns out that server.Shutdown polls every 0.5 s if all idle connections are closed. When we call server.Shutdown, our connection isn't closed just yet, so therefore we always have to wait 0.5 s for each test. But in reality it takes a lot less time than 0.5 s for the connection to finish closing. So if we just give up execution for a moment before calling server.Shutdown then the other goroutines have the time to shut the connection down before server.Shutdown checks if everything is closed. And then we don't get punished with waiting for 0.5 s every time we try to shut down.

With this approach sometimes tests still take 0.5 s to run. This might be because the scheduler continues execution on the same goroutine faster than the time it takes to close the connection. Putting in a time.Sleep(100 * time.Microsecond) instead of the runtime.Gosched() consistently makes the tests run faster, on my machine at least. But that is something I'll leave to be investigated at another time.
@empijei empijei merged commit 8ea7e8c into master Aug 12, 2020
@mattiasgrenfeldt mattiasgrenfeldt deleted the grenfeldt-speed-up-test-harness branch August 12, 2020 08:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

The test harness for testing net/http parsing is very slow
3 participants