Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use wall-time for credential helper invalidation #18301

Conversation

illicitonion
Copy link
Contributor

Previously we were using time intervals which excluded time spent with the system sleeping, which is not appropriate for expiring tokens which expire based on wall-time duration.

Previously we were using time intervals which excluded time spent with
the system sleeping, which is not appropriate for expiring tokens which
expire based on wall-time duration.
@github-actions github-actions bot added the awaiting-review PR is awaiting review from an assigned reviewer label May 3, 2023
@illicitonion
Copy link
Contributor Author

/cc @tjgq

@sgowroji sgowroji added the team-Remote-Exec Issues and PRs for the Execution (Remote) team label May 3, 2023
@tjgq
Copy link
Contributor

tjgq commented May 3, 2023

I'm not sure I understand the problem this PR is trying to solve.

You're proposing to switch from System.nanoTime (the default according to the Caffeine documentaion) to System.currentTimeMillis. My understanding is that the former reads a monotonic clock while the latter reads the system clock. I'd argue that the system clock is not appropriate in this situation, because it would include corrections (e.g., if the system clock is set back by an hour while a cache entry is alive, its lifetime would be extended by an hour).

I'm also skeptical of the claim that System.nanoTime remains stationary while the process is asleep. On my system, this program prints values that are 10 seconds apart:

public class Clock {
  public static void main(String[] args) {
    System.out.println(System.nanoTime() / 1_000_000_000);
    try {
      Thread.sleep(10_000);
    } catch (InterruptedException e) {}
    System.out.println(System.nanoTime() / 1_000_000_000);
  }
}

Is this in relation to a particular implementation of the JVM? Do I misunderstand the Java time APIs or what Caffeine is doing?

@illicitonion
Copy link
Contributor Author

By "asleep" I mean "if you close your laptop and it suspends". See https://blogs.sap.com/2023/02/10/jfr-timestamps-and-system.nanotime/ for more details - System.nanoTime ends up calling mach_absolute_time which is documented: "this clock does not increment while the system is asleep".

I have observed that if I get a credential helper token while configuring Bazel to use with an 8 hour timeout, close my laptop over night, open it 16 hours later and do a build, Bazel attempts to re-use my stale token. This PR avoids that problem.

@tjgq
Copy link
Contributor

tjgq commented May 3, 2023

Aha, thanks for the clarification! Unfortunately, I think that means neither System.nanoTime nor System.currentTimeMillis are appropriate for this purpose: the former will misbehave if the machine is suspended, while the latter will misbehave if a (large) correction is made to the system clock. But the former is much more likely than the latter, so I agree that this is moving things in the right direction.

@tjgq tjgq added awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally and removed awaiting-review PR is awaiting review from an assigned reviewer labels May 3, 2023
@ben-manes
Copy link

I think you should file this as a JDK bug as it is surprising behavior! It sounds like they should have used mach_continuous_time on MacOS.

@copybara-service copybara-service bot closed this in 5f77aba May 4, 2023
@illicitonion
Copy link
Contributor Author

@bazel-io flag

@illicitonion illicitonion deleted the credentialhelper-millis-ticker branch May 4, 2023 08:53
@bazel-io bazel-io added the potential release blocker Flagged by community members using "@bazel-io flag". Should be added to a release blocker milestone label May 4, 2023
@sgowroji sgowroji removed the awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally label May 4, 2023
@keertk
Copy link
Member

keertk commented May 4, 2023

@bazel-io fork 6.3.0

@bazel-io bazel-io removed the potential release blocker Flagged by community members using "@bazel-io flag". Should be added to a release blocker milestone label May 4, 2023
@ben-manes
Copy link

Submitted to bugs.java.com as internal id 9075217.

@ben-manes
Copy link

Oddly they removed the link to this issue and the target OS, instead stating that it works on Windows. Responded with a clarification on the OS and system methods.

https://bugs.java.com/bugdatabase/view_bug?bug_id=JDK-8307677

iancha1992 pushed a commit to iancha1992/bazel that referenced this pull request May 15, 2023
Previously we were using time intervals which excluded time spent with the system sleeping, which is not appropriate for expiring tokens which expire based on wall-time duration.

Closes bazelbuild#18301.

PiperOrigin-RevId: 529340767
Change-Id: I15e74e7bc87284f8ba53aedace955b29bd52df8e
iancha1992 added a commit that referenced this pull request May 17, 2023
Previously we were using time intervals which excluded time spent with the system sleeping, which is not appropriate for expiring tokens which expire based on wall-time duration.

Closes #18301.

PiperOrigin-RevId: 529340767
Change-Id: I15e74e7bc87284f8ba53aedace955b29bd52df8e

Co-authored-by: Daniel Wagner-Hall <dwagnerhall@apple.com>
Co-authored-by: keertk <keerthanakumar@google.com>
fweikert pushed a commit to fweikert/bazel that referenced this pull request May 25, 2023
Previously we were using time intervals which excluded time spent with the system sleeping, which is not appropriate for expiring tokens which expire based on wall-time duration.

Closes bazelbuild#18301.

PiperOrigin-RevId: 529340767
Change-Id: I15e74e7bc87284f8ba53aedace955b29bd52df8e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
team-Remote-Exec Issues and PRs for the Execution (Remote) team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants