Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

coreos-metadata.service: Caused by: Couldn't find '' flag in cmdline file (/proc/cmdline) #2303

jforman opened this Issue Dec 25, 2017 · 6 comments


None yet
4 participants
Copy link

jforman commented Dec 25, 2017

Issue Report


Container Linux Version

core@coreA11 ~ $ cat /etc/os-release
NAME="Container Linux by CoreOS"
PRETTY_NAME="Container Linux by CoreOS 1590.2.0 (Ladybug)"


Hardware: Bare-metal Ubuntu running Libvirt/Kvm with CoreOS as the guest VM

Expected Behavior

coreos-metadata.service agent should start

Actual Behavior

jforman@desktop1:~$ ssh core@
Container Linux by CoreOS beta (1590.2.0)
Failed Units: 1

core@coreA11 ~ $ systemctl status coreos-metadata.service
● coreos-metadata.service - CoreOS Metadata Agent
Loaded: loaded (/usr/lib/systemd/system/coreos-metadata.service; disabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Mon 2017-12-25 14:08:28 UTC; 36s ago
Process: 701 ExecStart=/usr/bin/coreos-metadata ${COREOS_METADATA_OPT_PROVIDER} --attributes=/run/metadata/coreos (code=exited, status=1/FAILURE)
Main PID: 701 (code=exited, status=1/FAILURE)

Dec 25 14:08:28 coreA11 systemd[1]: Starting CoreOS Metadata Agent...
Dec 25 14:08:28 coreA11 systemd[1]: coreos-metadata.service: Main process exited, code=exited, status=1/FAILURE
Dec 25 14:08:28 coreA11 systemd[1]: coreos-metadata.service: Failed with result 'exit-code'.
Dec 25 14:08:28 coreA11 systemd[1]: Failed to start CoreOS Metadata Agent.

core@coreA11 ~ $ cat /usr/lib/systemd/system/coreos-metadata.service
Description=CoreOS Metadata Agent

ExecStart=/usr/bin/coreos-metadata ${COREOS_METADATA_OPT_PROVIDER} --attributes=/run/metadata/coreos

core@coreA11 ~ $ /usr/bin/coreos-metadata --cmdline --attributes=/run/metadata/coreos
Error: initialization
Caused by: Couldn't find '' flag in cmdline file (/proc/cmdline)

Reproduction Steps

Reboot the VM. Any login. The service just doesn't start.

Other Information

I used the vagrant-virtualbox config transpiler platform for creating my ignition config.

I plan to try to hack on the coreos-metadata repo to add support for libvirt as a provider (vagrant-virtualbox does not work as a provider since it expects eth1 as the network interface, where as libvirt provides eth0 by default).


This comment has been minimized.

Copy link

crawford commented Jan 9, 2018

I don't expect coreos-metadata to support any platforms without a metadata service. The intention of coreos-metadata is just to consolidate all of the garbage logic needed to fetch metadata from each of the cloud providers and make it accessible to systemd services. In your scenario, a call to ip a will suffice. And, in fact, coreos-metadata can't know which interface corresponds to which role.

You can use something like this example as your service. It fundamentally the same way as coreos-metadata, it just doesn't have any fancy code. I hope that makes sense.


This comment has been minimized.

Copy link

jforman commented Jan 9, 2018

Oh interesting, I see what you mean @crawford. I just need to make sure this oneshot service starts before 'everything' else, yes?


This comment has been minimized.

Copy link

crawford commented Jan 9, 2018

Exactly. coreos-metadata.service is typically explicitly added as a dependency of (Requires=coreos-metadata.service) and ordered before (Before=coreos-metadata.service) each of the services which need its output. This ensures that everything starts lazily (if it's needed at all) and systemd will just take care of the rest.


This comment has been minimized.

Copy link

lucab commented Jan 9, 2018

I missed this initial bug-report, but I've left a similar comment when reviewing the resulting PR: coreos/coreos-metadata#81 (comment). I'd guess this ticket can be closed then.


This comment has been minimized.

Copy link

nickgrealy commented Apr 13, 2018

Hi @crawford,

Apologies up front if these questions are "stupid", I'm just getting up to speed and had a few questions:

1) Do I need to manually add a flag to the /proc/cmdline?

Manually sounds bad, how should the value be populated normally? There is currently no flag at all in my /proc/cmdline.

(I don't understand how a custom metadata agent will fix this issue.)


core@localhost ~ $ cat /proc/cmdline
rootflags=rw mount.usrflags=ro BOOT_IMAGE=/coreos/vmlinuz-a mount.usr=/dev/mapper/usr verity.usr=PARTUUID=<SOME_ID> rootflags=rw mount.usrflags=ro consoleblank=0 root=LABEL=ROOT console=ttyS0,115200n8 console=tty0 coreos.config.url=oem:///coreos-install.json verity.usrhash=<SOME_HASH>

2) Does it matter what filename the custom metadata agent writes to?

Can I call it foobar, as long as it aligns with the EnvironmentFile=/run/metadata/foobar?

3) What triggers the use of coreos-metadata?

Is it the presence of a variable in the etcd section, or in the systemd.units section, or both? (when I remove all variables from my Ignition config, I don't get this issue).


# setup shared config
  version:                     3.1.6
  name:                        "{HOSTNAME}"
  advertise_client_urls:       http://{PRIVATE_IPV4}:2375,http://{PRIVATE_IPV4}:9000
  discovery:         <token>


    # install portainer
    - name: portainer.service
      enable: true
      contents: |

        ExecStartPre=-/usr/bin/docker stop %n
        ExecStartPre=-/usr/bin/docker rm %n
        ExecStartPre=/usr/bin/docker volume create portainer_data
        ExecStartPre=/usr/bin/docker pull portainer/portainer
        ExecStart=/usr/bin/docker run --rm --name %n -p 9000:9000 -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer
        ExecStartPost=/usr/bin/etcdctl set /domains/{PRIVATE_IPV4}:9000 running
        ExecStop=/usr/bin/docker stop %n
        ExecStopPost=/usr/bin/etcdctl rm /domains/{PRIVATE_IPV4}:9000



UPDATE: I've just read this document on setting up a custom metadata provider, it's a much better resource than this article, and (if it works) answers my questions.

(If it works...) Is there any chance we can update the docs?


This comment has been minimized.

Copy link

lucab commented Apr 13, 2018

@nickgrealy trying to answer most of your questions here, but if you keep having doubts on your journey please prefer the ML for discussion.

  1. No, you don't need to set manually. if you are on a plain bare-metal node (i.e. without any external metadata or cloud-service service), it is correct for that to be missing/empty.

  2. It matters if you plan to use other tools/units of ours (e.g. ct) which expects our standard path. This seems to be your case.

  3. When processing your YAML configuration, templated variables are inspected and coreos-metadata dependencies (After+Requires) are added to the resulting units. I think you may have some confusion here, because the config you pasted is not the final ignition config. This is just the input for container-linux-config-transpiler (i.e. ct), but the final result will be a JSON config file for ignition with those additional details and dependencies.

  4. it is hard for us to strike a good balance in user doc details, so we'll gladly take any hints from your side to iron out that article. The source for that article is here, and you can just freely send a PR with your suggestions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.