Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update-engine sends empty requests when restarted before a pending reboot #388

Closed
pothos opened this issue Apr 27, 2021 · 2 comments · Fixed by flatcar/update_engine#9
Closed
Labels
channel/alpha Issue concerns the Alpha channel. channel/beta Issue concerns the Beta channel. channel/stable Issue concerns the Stable channel. kind/bug Something isn't working

Comments

@pothos
Copy link
Member

pothos commented Apr 27, 2021

Description

Update-engine sends empty requests to an empty URL when the update-engine service is restarted before a pending reboot. This can only be resolved by rebooting or by resetting the update-engine state.

Impact

The instance does not communicate with the update server anymore.

Environment and steps to reproduce

  1. Set-up: Run an old Flatcar version with standard settings, but mask locksmith
  2. Task: Trigger an update (update_engine_client -update), then restart update-engine.service
  3. Action(s): Restart update-engine after the update is written to the partition and update-engine waits for the reboot
  4. Error: in the journal one can see that all ping's are now done with empty values
journalctl --no-pager -u update-engine | grep -i "Posting an Omaha request to"
May 25 08:05:54 localhost update_engine[859]: I0525 08:05:54.272867   859 omaha_request_action.cc:268] Posting an Omaha request to https://public.update.flatcar-linux.net/v1/update/
May 25 08:06:01 localhost update_engine[859]: I0525 08:06:01.907414   859 omaha_request_action.cc:268] Posting an Omaha request to https://public.update.flatcar-linux.net/v1/update/
May 25 08:10:33 localhost update_engine[1011]: I0525 08:10:33.162166  1011 omaha_request_action.cc:268] Posting an Omaha request to

Expected behavior
Without restarting the service the pings have the regular values as before the update (except for the update check entry). This is also expected after restarting the service but doesn't work currently.

Additional information
When resetting the update-engine state there may be a crash of the service but it recovers

@pothos pothos added the kind/bug Something isn't working label Apr 27, 2021
@tormath1
Copy link
Contributor

tormath1 commented May 25, 2021

From an initial look, we can clearly see that even the values of the payload are missing between the restarts of the update-engine service:

<request protocol="3.0" version="update_engine-0.4.10" updaterversion="update_engine-0.4.10" installsource="ondemandupdate" ismachine="1">
    <os version="Chateau" platform="CoreOS" sp="2765.2.4_x86_64"></os>
    <app appid="{e96281a6-d1af-4bde-9a0a-97b76e56dc57}" version="2765.2.4" track="stable" bootid="{f2195e2e-7a2d-44f7-8ec7-e8f1327175aa}" oem="qemu" oemversion...>
        <ping active="1"></ping>
        <updatecheck></updatecheck>
        <event eventtype="3" eventresult="2" previousversion="0.0.0.0"></event>
    </app>
</request>

After restart:

<request protocol="3.0" version="update_engine-0.4.10" updaterversion="update_engine-0.4.10" installsource="scheduler" ismachine="1">
    <os version="Chateau" platform="CoreOS" sp=""></os>
    <app appid="{e96281a6-d1af-4bde-9a0a-97b76e56dc57}" version="" track="stable" bootid="" oem="" oemversion="" alephversion="" machineid="" machinealias="" lang="" board="" hardware_class="" delta_okay="true" >
        <ping active="1"></ping>
    </app>
</request>

This parameters (and the update_url) are computed in this lines:
https://github.com/kinvolk/update_engine/blob/312ab260fce283e84635b73d92dd6d526d1e3d96/src/update_engine/update_attempter.cc#L159-L162

This method is not called if the status is UPDATE_STATUS_UPDATED_NEED_REBOOT because we "simply" PingOmaha (https://github.com/kinvolk/update_engine/blob/312ab260fce283e84635b73d92dd6d526d1e3d96/src/update_engine/update_attempter.cc#L128-L134)

@pothos
Copy link
Member Author

pothos commented May 25, 2021

Good find, seems ok to try calling this earlier but maybe a special case is needed for the pending update state.

@t-lo t-lo added this to Ready to release in Flatcar Container Linux Releases Planning Jun 1, 2021
@t-lo t-lo added channel/alpha Issue concerns the Alpha channel. channel/beta Issue concerns the Beta channel. channel/stable Issue concerns the Stable channel. labels Jun 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
channel/alpha Issue concerns the Alpha channel. channel/beta Issue concerns the Beta channel. channel/stable Issue concerns the Stable channel. kind/bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants