New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use monotonic clocks for timing #96

Closed
ScottMansfield opened this Issue Aug 22, 2016 · 2 comments

Comments

Projects
None yet
1 participant
@ScottMansfield
Contributor

ScottMansfield commented Aug 22, 2016

Currently the timing metrics collected use a realtime clock provided by time.Now(). This is inaccurate for timing purposes where we care about the actual amount of time that has passed vs. the time that has passed in the "real" wall-clock time. Clock adjustments can provide wildly inaccurate timing for short-term actions. The answer, unfortunately, is to create a package that can provide a raw monotonic clock read by calling clock_gettime through VDSO.

This must be done in assembly to conform to the C calling conventions. This is a paved path, which is really nice for us. spacemonkeygo has a proper package to do this:

https://github.com/spacemonkeygo/monotime/blob/master/mono_linux_amd64.s

This code can be added to the rend project (along with proper license handling) to handle raw monotonic time.

@ScottMansfield

This comment has been minimized.

Contributor

ScottMansfield commented Sep 9, 2016

Turns out that the CLOCK_MONOTONIC_RAW argument was not really appropriate and didn't work. I decided to use CLOCK_MONOTONIC as the final argument. Partially, I was convinced by Java using it in lieu of the _RAW version: https://bugs.openjdk.java.net/browse/JDK-8006942

It was going to be based on the spacemonkeygo code but it ended up not being a single line of code from there. I lifted the runtime·nanotime function from the Go standard library instead: https://golang.org/src/runtime/sys_linux_amd64.s#L167

@ScottMansfield

This comment has been minimized.

Contributor

ScottMansfield commented Sep 9, 2016

What first sent me down the path of discovery was that using CLOCK_MONOTONIC_RAW was actually 3 times slower than the standard CLOCK_MONOTONIC argument. I decided to dig a bit more and found that the kernel version I was using (at least) did not support the _RAW version in the __vdso_clock_gettime and was instead giving a fallback answer. Since Rend runs on the Netflix base image, I do not have control of the kernel version and cannot even change it to something that does support _RAW (if any even do). This, combined with Java using the non-raw clock, convinced me to just go with CLOCK_MONOTONIC.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment