Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Transform] Use monotonic time in TransformScheduler #95456

Merged
merged 4 commits into from
Apr 24, 2023

Conversation

przemekwitek
Copy link
Contributor

System clock may not be monotonic, i.e.: it is possible, that the following snippet fails:

  long time1 = clock.millis();
  long time2 = clock.millis();
  assertThat(time2, is(greaterThanOrEqualTo(time1));

In the context of transforms, it can happen that after _schedule_now is called, the transform will not be processed instantly (as is the premise of _schedule_now), because the current time fetched in line (*) will be earlier than the transform's next scheduled time (which was set just a while ago):

        long currentTimeMillis = clock.millis();  // (*)

        // Check if the task is eligible for processing
        if (currentTimeMillis < scheduledTask.getNextScheduledTimeMillis()) {
            // It is too early to process this task.
            // Consequently, it is too early to process other tasks because the tasks are sorted by their next scheduled time.
            // Try again later.
            return false;
        }

This PR fixes it by ensuring that the next time is at least as big as the previous time.

Fixes #95445

@elasticsearchmachine elasticsearchmachine added the Team:ML Meta label for the ML team label Apr 21, 2023
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/ml-core (Team:ML)

@elasticsearchmachine
Copy link
Collaborator

Hi @przemekwitek, I've created a changelog YAML for you.

Copy link
Contributor

@droberts195 droberts195 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@przemekwitek przemekwitek merged commit 6b8d1ef into elastic:main Apr 24, 2023
@przemekwitek przemekwitek deleted the monotonic_scheduler_time branch April 24, 2023 07:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :ml/Transform Transform Team:ML Meta label for the ML team v8.8.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CI] TransformSchedulerTests testScheduleNowWithSystemClock failing
3 participants