Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

flatcar-postinst: Handle airgapped self-hosted nebraska instances #39

Merged
merged 1 commit into from Feb 26, 2024

Conversation

jepio
Copy link
Member

@jepio jepio commented Feb 22, 2024

This is a backport of the equivalent of flatcar/scripts#1580 to 3815, without switching to ue-rs.

Tested on the full-response from the airgapped environment (in commit message):

$ cat flatcar-postinst
set -euo pipefail
set -x
umask 0022
OEMID=vmware
sysext_download() {
    ....
}
sysext_download "oem-${OEMID}.gz" "./var/lib/update_engine/oem-${OEMID}.raw" test.xml

Output:

$ bash -x flatcar-postinst
+ set -euo pipefail
+ set -x
+ umask 0022
+ OEMID=vmware
+ sysext_download oem-vmware.gz ./var/lib/update_engine/oem-vmware.raw test2.xml
+ local name=oem-vmware.gz
+ local target=./var/lib/update_engine/oem-vmware.raw
+ local from=test2.xml
+ local base=
+ local entries=
+ local hash=
+ local size=
+ local url=
+ local ret
+ SUCCESS=false
+ set +e
+ set -e
+ '[' test2.xml = release-server ']'
+ '[' test2.xml = bincache-server ']'
++ grep -m 1 -o 'codebase="[^"]*"' test2.xml
++ cut -d '"' -f 2
+ base=https://nebraska-example.org/flatcar/
++ grep -m 1 -o '<package name="[^"]*oem-vmware.gz"[^>]*' test2.xml
+ entries='<package name="extrafile-amd64-3815.2.0-oem-vmware.gz" hash="llmt48kEE1pZxHbhE86zTczZMmY=" hash_sha256="7c4095f16579402d73ea42a63be63b2c92d98457303a080399d2a5bb33f46f88" size="1536465" required="false"/'
++ echo '<package name="extrafile-amd64-3815.2.0-oem-vmware.gz" hash="llmt48kEE1pZxHbhE86zTczZMmY=" hash_sha256="7c4095f16579402d73ea42a63be63b2c92d98457303a080399d2a5bb33f46f88" size="1536465" required="false"/'
++ grep -o 'name="[^"]*'
++ cut -d '"' -f 2
+ name=extrafile-amd64-3815.2.0-oem-vmware.gz
+ url=https://nebraska-example.org/flatcar/extrafile-amd64-3815.2.0-oem-vmware.gz
++ echo '<package name="extrafile-amd64-3815.2.0-oem-vmware.gz" hash="llmt48kEE1pZxHbhE86zTczZMmY=" hash_sha256="7c4095f16579402d73ea42a63be63b2c92d98457303a080399d2a5bb33f46f88" size="1536465" required="false"/'
++ grep -o 'size="[0-9]*'
++ cut -d '"' -f 2
+ size=1536465
++ echo '<package name="extrafile-amd64-3815.2.0-oem-vmware.gz" hash="llmt48kEE1pZxHbhE86zTczZMmY=" hash_sha256="7c4095f16579402d73ea42a63be63b2c92d98457303a080399d2a5bb33f46f88" size="1536465" required="false"/'
++ grep -o -P 'hash="[^"]*'
++ cut -d '"' -f 2
+ hash=llmt48kEE1pZxHbhE86zTczZMmY=
++ echo '<package name="extrafile-amd64-3815.2.0-oem-vmware.gz" hash="llmt48kEE1pZxHbhE86zTczZMmY=" hash_sha256="7c4095f16579402d73ea42a63be63b2c92d98457303a080399d2a5bb33f46f88" size="1536465" required="false"/'
++ grep -o -P 'hash_sha256="[^"]*'
++ cut -d '"' -f 2
+ hash_sha256=7c4095f16579402d73ea42a63be63b2c92d98457303a080399d2a5bb33f46f88
+ rm -f ./var/lib/update_engine/oem-vmware.raw.tmp
+ curl -fsSL --retry-delay 1 --retry 60 --retry-connrefused --retry-max-time 60 --connect-timeout 20 -o ./var/lib/update_engine/oem-vmware.raw.tmp https://nebraska-example.org/flatcar/extrafile-amd64-3815.2.0-oem-vmware.gz
curl: (6) Could not resolve host: nebraska-example.org
curl: (6) Could not resolve host: nebraska-example.org
....

Right now the stable update from 3602.2.3 -> 3815.2.0 fails for airgapped
environment with self-hosted nebraska instances that sync packages, because the
parsing in flatcar-postinst can't handle the names of OEM sysexts in the
nebraska response. These are called "extrafile-<BOARD>-<VERSION>-oem-<OEM>.gz".
Fix this by matching against "*oem-<OEM>.gz" and extracting the remote filename
from the response.

In main this is fixed in #36, but
that depends on switching to ue-rs.

Here is an example response for testing:

<response protocol="3.0" server="nebraska">
  <daystart elapsed_seconds="0"/>
  <app appid="{e96281a6-d1af-4bde-9a0a-97b76e56dc57}" status="ok">
    <ping status="ok"/>
    <updatecheck status="ok">
      <urls>
        <url codebase="https://nebraska-example.org/flatcar/"/>
      </urls>
      <manifest version="3815.2.0">
        <packages>
          <package name="flatcar-amd64-3815.2.0.gz" hash="JLT5o2wJzffAgHY5GesCWymJHng=" size="458309926" required="true"/>
          <package name="extrafile-amd64-3815.2.0-oem-vmware.gz" hash="llmt48kEE1pZxHbhE86zTczZMmY=" hash_sha256="7c4095f16579402d73ea42a63be63b2c92d98457303a080399d2a5bb33f46f88" size="1536465" required="false"/>
        </packages>
    </updatecheck>
  </app>
</response>

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Copy link
Member

@pothos pothos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@jepio jepio merged commit 71b7cc3 into flatcar-3815-backport Feb 26, 2024
@jepio jepio deleted the jepio/3815-fix-self-sync-nebraska branch February 26, 2024 11:06
jepio added a commit to flatcar/scripts that referenced this pull request Feb 26, 2024
Pulls in flatcar/update_engine#39 which fixes updates
on the stable channel for nodes with self-hosted packages in an environment
where nodes are not allowed access to the upstream Flatcar update server.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants