New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ignition config is not loaded when passed in by QEMU #2084

Closed
stevert351 opened this Issue Aug 1, 2017 · 10 comments

Comments

Projects
None yet
4 participants
@stevert351
Copy link

stevert351 commented Aug 1, 2017

Issue Report

Ignition config file not loaded during vm boot. Looks like this problem was raised late last year, see the bug report here for a similar situation.

Bug

Ignition config parameters are passed into VM by QEMU but Ignition does not see them nor action them.

Container Linux Version

$ cat /etc/os-release
NAME="Container Linux by CoreOS"
ID=coreos
VERSION=1409.7.0
VERSION_ID=1409.7.0
BUILD_ID=2017-07-19-0005

BUG_REPORT_URL="https://issues.coreos.com"

Environment

VM launched on Ubuntu Server 17.04 using QEMU v2.8

Expected Behavior

Action Ignition config file

Actual Behavior

Ignored config file

Reproduction Steps

Ignition config file:

{
  "ignition": {
    "version": "2.1.0",
    "config": {}
  },
  "storage": {
    "files": [{
      "filesystem": "root",
      "path": "/etc/hostname",
      "mode": 420,
      "contents": { "source": "data:,corehost1" }
    }]
  },
  "systemd": {},
  "networkd": {},
  "passwd": {}
}

Command to start VM:

./coreos_production_qemu.sh -s -i config.ign -nographic 2>&1 | tee -a vm.log

which resulted in this command being executed: (edited for readability)

qemu-system-x86_64 -name coreos_production_qemu-1409-7-0 -m 1024 \`
-net nic,vlan=0,model=virtio \
-net user,vlan=0,hostfwd=tcp::2222-:22,hostname=coreos_production_qemu-1409-7-0 \
-fw_cfg name=opt/com.coreos/config,file=config.ign \ 
-drive if=virtio,file=./coreos_production_qemu_image.img \ 
-fsdev local,id=conf,security_model=none,readonly,path=/tmp/coreos-configdrive.p4QzAGs5KK \ 
-device virtio-9p-pci,fsdev=conf,mount_tag=config-2 \ 
-machine accel=tcg -nographic

The journal file from the VM has this for Ignition:

Aug 01 11:33:25 localhost ignition[308]: Ignition v0.14.0
Aug 01 11:33:25 localhost ignition[308]: parsed url from cmdline: ""
Aug 01 11:33:25 localhost ignition[308]: no config URL provided
Aug 01 11:33:25 localhost ignition[308]: noop provider fetching empty config
Aug 01 11:33:25 localhost ignition[308]: failed to fetch config: not a config (empty)
Aug 01 11:33:25 localhost ignition[308]: not a config (empty): ignoring user-provided config
Aug 01 11:33:29 localhost ignition[367]: Ignition v0.14.0
Aug 01 11:33:29 localhost ignition[367]: parsed url from cmdline: ""
Aug 01 11:33:29 localhost ignition[367]: no config URL provided
Aug 01 11:33:29 localhost ignition[367]: noop provider fetching empty config
Aug 01 11:33:29 localhost ignition[367]: failed to fetch config: not a config (empty)
Aug 01 11:33:29 localhost ignition[367]: not a config (empty): ignoring user-provided config

System firmware:

core@coreos_production_qemu-1409-7-0 ~ $ ls -al /sys/firmware/qemu_fw_cfg/by_name/opt/com.coreos/config/
total 0
drwxr-xr-x.  2 root root    0 Aug  1 12:38 .
drwxr-xr-x. 14 root root    0 Aug  1 12:38 ..
-r--------.  1 root root 4096 Aug  1 12:38 key
-r--------.  1 root root 4096 Aug  1 12:38 name
-r--------.  1 root root    0 Aug  1 12:38 raw
-r--------.  1 root root 4096 Aug  1 12:38 size

key file: 43

name file: opt/com.coreos/config

size file: 287

raw file:

{
  "ignition": {
    "version": "2.1.0",
    "config": {}
  },
  "storage": {
    "files": [{
      "filesystem": "root",
      "path": "/etc/hostname",
      "mode": 420,
      "contents": { "source": "data:,corehost1" }
    }]
  },
  "systemd": {},
  "networkd": {},
  "passwd": {}
}

Other Information

Feature Request

Environment

What hardware/cloud provider/hypervisor is being used to run Container Linux?

Desired Feature

Other Information

@lucab

This comment has been minimized.

Copy link
Member

lucab commented Aug 1, 2017

I didn't spot anything clearly wrong in the execution, however I'm concerned by the time skew between log entries (11:33) and the file info (12:38).

Are you by chance re-running this VM image for the non-first time? If so, ignition config is only parsed and applied on first boot (where you perhaps didn't pass it, thus empty) and further config file are not taken into account.

@stevert351

This comment has been minimized.

Copy link

stevert351 commented Aug 1, 2017

I am not sure why the times are out by so much. So i have rerun the test with a fresh setup.

The times are:

core@coreos_production_qemu-1409-7-0 ~ $ date
Tue Aug 1 15:21:43 UTC 2017

core@coreos_production_qemu-1409-7-0 ~ $ journalctl --identifier=ignition --all
-- Logs begin at Tue 2017-08-01 15:15:21 UTC, end at Tue 2017-08-01 15:21:33 UTC. --
Aug 01 15:15:32 localhost ignition[322]: Ignition v0.14.0
Aug 01 15:15:32 localhost ignition[322]: parsed url from cmdline: ""
Aug 01 15:15:32 localhost ignition[322]: no config URL provided
Aug 01 15:15:32 localhost ignition[322]: noop provider fetching empty config
Aug 01 15:15:32 localhost ignition[322]: failed to fetch config: not a config (empty)
Aug 01 15:15:32 localhost ignition[322]: not a config (empty): ignoring user-provided config
Aug 01 15:15:35 localhost ignition[374]: Ignition v0.14.0
Aug 01 15:15:35 localhost ignition[374]: parsed url from cmdline: ""
Aug 01 15:15:35 localhost ignition[374]: no config URL provided
Aug 01 15:15:35 localhost ignition[374]: noop provider fetching empty config
Aug 01 15:15:35 localhost ignition[374]: failed to fetch config: not a config (empty)
Aug 01 15:15:35 localhost ignition[374]: not a config (empty): ignoring user-provided config

core@coreos_production_qemu-1409-7-0 ~ $ ls -al /sys/firmware/qemu_fw_cfg/by_name/opt/com.coreos/config/
total 0
drwxr-xr-x. 2 root root 0 Aug 1 15:17 .
drwxr-xr-x. 14 root root 0 Aug 1 15:17 ..
-r--------. 1 root root 4096 Aug 1 15:17 key
-r--------. 1 root root 4096 Aug 1 15:17 name
-r--------. 1 root root 0 Aug 1 15:17 raw
-r--------. 1 root root 4096 Aug 1 15:17 size

@crawford

This comment has been minimized.

Copy link
Member

crawford commented Aug 1, 2017

According to the logs, Ignition doesn't recognize this is QEMU:

Aug 01 15:15:35 localhost ignition[374]: noop provider fetching empty config

The noop provider is used for a number of images which don't yet support Ignition. Which image did you download?

@stevert351

This comment has been minimized.

Copy link

stevert351 commented Aug 1, 2017

I used the channel documented in the Running CoreOS Container Linux on QEMU, which is:

https://stable.release.core-os.net/amd64-usr/current/coreos_production_qemu_image.img.bz2

@crawford

This comment has been minimized.

Copy link
Member

crawford commented Aug 1, 2017

I just tried this and it worked fine:

core@corehost1 ~ $ journalctl -t ignition --no-pager
-- Logs begin at Tue 2017-08-01 17:34:04 UTC, end at Tue 2017-08-01 17:34:43 UTC. --
Aug 01 17:34:06 localhost ignition[394]: Ignition v0.14.0
Aug 01 17:34:06 localhost ignition[394]: parsed url from cmdline: ""
Aug 01 17:34:06 localhost ignition[394]: no config URL provided
Aug 01 17:34:06 localhost ignition[394]: op(1): [started]  loading QEMU firmware config module
Aug 01 17:34:06 localhost ignition[394]: op(1): executing: "modprobe" "qemu_fw_cfg"
Aug 01 17:34:06 localhost ignition[394]: op(1): [finished] loading QEMU firmware config module
Aug 01 17:34:06 localhost ignition[394]: parsing config: {
                                                 "ignition": {
                                                         "version": "2.0.0",
                                                         "config": {}
                                                 },
                                                 "storage": {
                                                         "files": [{
                                                                 "filesystem": "root",
                                                                 "path": "/etc/hostname",
                                                                 "mode": 420,
                                                                 "contents": { "source": "data:,corehost1" }
                                                         }]
                                                 },
                                                 "systemd": {},
                                                 "networkd": {},
                                                 "passwd": {}
                                         }
Aug 01 17:34:06 localhost ignition[459]: Ignition v0.14.0
Aug 01 17:34:06 localhost ignition[459]: files: createFilesystemsFiles: createFiles: op(1): [started]  writing file "/etc/hostname"
Aug 01 17:34:06 localhost ignition[459]: files: createFilesystemsFiles: createFiles: op(1): [finished] writing file "/etc/hostname"

I did have to modify your config so that it used "2.0.0" instead of "2.1.0" (Stable doesn't yet support the "2.1.0" specification, but it will in a couple weeks). Without that change, Ignition would fail.

Which version of QEMU are you using?

@stevert351

This comment has been minimized.

Copy link

stevert351 commented Aug 1, 2017

@crawford

This comment has been minimized.

Copy link
Member

crawford commented Aug 1, 2017

What I don't understand is why my original
command line did not show the error. Instead it started the VM albeit very
slowly, about 2 minutes.

Ah, the -s flag disables KVM support, so QEMU is emulating the processor instead of using hardware acceleration.

Glad you got it working.

@crawford crawford closed this Aug 1, 2017

@stevert351

This comment has been minimized.

Copy link

stevert351 commented Aug 2, 2017

I guess QEMU has an issue processing the fw_cfg option when in safe mode (-s flag), which causes the problem when passing an Ignition config through with the -i flag

Might it be worth amending the script 'coreos_production_qemu.sh ' to issue a warning if these two options are specified together?

As to why I used the safe option in the first place, I can only put it down to being a newbie on CoreOS.

@crawford

This comment has been minimized.

Copy link
Member

crawford commented Aug 2, 2017

Good idea.

@bgilbert

This comment has been minimized.

Copy link
Member

bgilbert commented Sep 26, 2018

Thank you for reporting this issue. Unfortunately, we don't think we'll end up addressing it in Container Linux.

We're now working on Fedora CoreOS, the successor to Container Linux, and we expect most major development to occur there instead. Meanwhile, Container Linux will be fully maintained into 2020 but won't see many new features. We appreciate your taking the time to report this issue and we're sorry that we won't be able to address it.

@bgilbert bgilbert closed this Sep 26, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment