Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(checkstate): ignore and abort carryover changes #415

Merged
merged 1 commit into from May 2, 2024

Conversation

flotter
Copy link
Contributor

@flotter flotter commented Apr 29, 2024

Addresses #414.

Running checks have changes that get persisted by the state-engine. This means that following a reboot, changes and tasks that are not complete (not ready) will be resumed. In the case of some managers, such as checkstate, carryover changes and tasks are an unwanted side effect.

Currently the plan manager will perform the first plan load very early during startup (before the state-engine is ready, and before StartUp hooks have been called). The result is that a PlanChanged propagation will take place at plan load, and force checkstate to inspect the running checks prematurely.

Checkstate discovers a running change from a previous boot context, and tries to load its data from cached state, which does not exist.

Gracefully ignore changes for which cached state does not exist, and use this to identify changes that should be aborted on the first ensure pass.

Test:

<reboot>
59   Hold    today at 23:29 SAST  today at 23:37 SAST  Recover exec check "internet-online"
60   Error   today at 23:37 SAST  today at 23:37 SAST  Perform exec check "internet-online"
:
66   Doing   today at 23:37 SAST  -                    Recover exec check "internet-online"
<reboot>
:
66   Hold    today at 23:37 SAST  today at 23:41 SAST  Recover exec check "internet-online"
:
69   Error   today at 23:41 SAST  today at 23:42 SAST  Perform exec check "internet-online"
:
73   Doing   today at 23:42 SAST  -                    Recover exec check "internet-online"

As the test demonstrates, following a reboot, the change (66) is aborted, and as a result the status is now Hold. A new change is created which starts as a Perform, and then fails (as expected) after a while and changes to a Recover.

Running checks have changes that gets persisted the state-engine.
This means that following a reboot, changes and tasks that are not
complete (ready) will be resumed. In the case of some managers, such
as checkstate, carryover changes and tasks are an unwanted side effect.

Currently the plan manager will perform the first plan load very early
during startup (before the state-engine is ready, and before StartUp
hooks have been called). The result is that a PlanChanged propagation
will take place at plan load, and force checkstate to inspect the
running checks prematurely.

Checkstate discovers a running change from a previous boot context,
and tries to load its data from cached state, which does not exist.

Gracefully ignore changes for which cached state does not exist,
and use this to identify changes that should be aborted on the first
ensure pass.
@flotter flotter changed the title fix(checkstate): ignore and abort carryover tasks fix(checkstate): ignore and abort carryover changes Apr 29, 2024
@flotter flotter requested a review from benhoyt April 29, 2024 21:40
@flotter flotter requested a review from hpidcock April 29, 2024 21:48
Copy link
Contributor

@benhoyt benhoyt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll work on a more holistic fix, but that might not be right away, so I'm happy for this temporary fix to go in in the meantime. Thanks!

@flotter flotter requested a review from IronCore864 May 1, 2024 11:18
@benhoyt benhoyt merged commit 02d7034 into canonical:master May 2, 2024
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants