Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

any way to pass raw.lxc arguments? #11

Open
juju4 opened this issue Aug 26, 2016 · 12 comments
Open

any way to pass raw.lxc arguments? #11

juju4 opened this issue Aug 26, 2016 · 12 comments

Comments

@juju4
Copy link

juju4 commented Aug 26, 2016

I have fallen on a common issue of container with httpd install on centos requiring some capabilities

a workaround is already documented
https://lists.linuxcontainers.org/pipermail/lxc-users/2014-June/007085.html

Look for /usr/share/lxc/config/fedora.common.conf (or whatever it is on
fedora, try "rpm -ql lxc"), then comment out this line

lxc.cap.drop = setfcap

It doesn't seem possible to use as an official config so need to use raw.lxc
https://github.com/lxc/lxd/issues/1982

lxc config set container-name raw.lxc=lxc.cap.drop=some-cap

any way to pass that inside kitchen? per platform as only centos but probably other case than can apply to each one.

Thanks
@bradenwright
Copy link
Owner

I do allow config to be passed so if LCD allows it, then pretty sure it
should work. I'll try to post later today/tonight

On Aug 26, 2016 7:42 AM, "Julien" notifications@github.com wrote:

I have fallen on a common issue of container with httpd install on centos
requiring some capabilities

a workaround is already documented
https://lists.linuxcontainers.org/pipermail/lxc-users/2014-
June/007085.html

Look for /usr/share/lxc/config/fedora.common.conf (or whatever it is on
fedora, try "rpm -ql lxc"), then comment out this line

lxc.cap.drop = setfcap

It doesn't seem possible to use as an official config so need to use
raw.lxc
lxc/lxd#1982 https://github.com/lxc/lxd/issues/1982

lxc config set container-name raw.lxc=lxc.cap.drop=some-cap

any way to pass that inside kitchen? per platform as only centos but
probably other case than can apply to each one.

Thanks

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#11, or mute the
thread
https://github.com/notifications/unsubscribe-auth/ABesvgR5MaVtrmRZYYcFR8tg3IQP8OK9ks5qjt8tgaJpZM4JuCgT
.

@juju4
Copy link
Author

juju4 commented Aug 30, 2016

it seems also profile definition has changed

driver:
  name: lxd_cli
  profile: unconfined

gets

-----> Creating <default-centos-7>...
       Initializing container default-centos-7-1472172303
error: Requested profile 'unconfined' doesn't exist
error: not found

I suppose it is now security.privileged=true but how to pass it to the driver?

thanks

@bradenwright
Copy link
Owner

bradenwright commented Aug 30, 2016 •
edited

I'll test the profile thing and see if I can get it to work. I'll try to find time soon b/c I don't think it will take me very long but we have some work outages that have been going on, so I haven't had much time.

For turning on security privilege it is just that config option like you mention and you can set it like:

cat .kitchen.lxd.yml 
---
driver:
  name: lxd_cli
  config:
    security.privileged: true
    limits.memory: 2GB
    limits.cpu: 2

Which you can see in the debug info:

 kit create -l debug
-----> Starting Kitchen (v1.8.0)
D      Berksfile found at /home/bwright/chef-repo/cookbooks/olab_haproxy/Berksfile, loading Berkshelf
D      Berkshelf 4.3.3 library loaded
-----> Creating <default-ubuntu-1404>...
D      Container default-ubuntu-1404-1472439710 doesn't exist
D      Publish Image Name is kitchen-default-ubuntu-1404
D      Image Name is olab_base-ubuntu-14.04-static-ip
D      Image olab_base-ubuntu-14.04-static-ip exists
D      Config Args:  -c security.privileged=true -c limits.memory=2GB -c limits.cpu=2
       Initializing container default-ubuntu-1404-1472439710
D      run_local_command ran: lxc init olab_base-ubuntu-14.04-static-ip default-ubuntu-1404-1472439710   -c security.privileged=true -c limits.memory=2GB -c limits.cpu=2

and on the container info:

lxc config show default-ubuntu-1404-1472439710 
name: default-ubuntu-1404-1472439710
profiles:
- default
config:
  limits.cpu: "2"
  limits.memory: 2GB
  raw.lxc: |+
    lxc.network.0.ipv4 = 10.100.0.10/24
    lxc.network.0.ipv4.gateway = 10.100.0.1

  security.privileged: "true"

So config options seem to be working properly.

@bradenwright
Copy link
Owner

Profiles seem to work properly as well. That error says the profile doesn't exist.... do you see it when you run
lxc profile list

B/c just like config when profile both string and array work.

cat .kitchen.lxd.yml 
---
driver:
  name: lxd_cli
  profile: [default, docker]

~$ lxc profile list
default
docker

But when I try a profile that doesn't exist on the system like I get the same error

 kit creat
-----> Starting Kitchen (v1.8.0)
-----> Creating <default-ubuntu-1404>...
       Initializing container default-ubuntu-1404-1472599256
error: Requested profile 'defaul' doesn't exist
error: not found

@bradenwright
Copy link
Owner

You can pass lxc.raw arguments as long as you can on the cli. If lxc launch ubuntu:16.04 u1 -c blah works then it should work in kitchen.

The only thing I can think of that could cause a bug would be if you are using an ipv4 config option I set raw options, can't say for sure if they overwrite or not. I know create raw.lxc arguments are blocked by lxd.r

Hopefully that give you enough to go on, otherwise maybe search the lxd github for stuff about raw.lxc arguments. Let me know how things go, and I'll help more if its quick/I can otherwise I'll take more of a look but don't know how much time I'll have this week.

@juju4
Copy link
Author

juju4 commented Sep 11, 2016

so for now, security.privileged: "true" globally is working fine.
Ideally, I want to restrict to centos7 or corresponding platforms when I had a config to platforms entry

for docker.io install, I had to set both privileged and profile which is not consistent with
https://www.stgraber.org/2016/04/13/lxd-2-0-docker-in-lxd-712/
case: https://github.com/juju4/ansible-remnux

for httpd/centos7, I would prefer to remove privileged and just add cap config in kitchen but for now not possible it seems.

@juju4
Copy link
Author

juju4 commented Oct 12, 2016

Hello Brad,

Had you time to take a look on capability config?

Thanks

@bradenwright
Copy link
Owner

bradenwright commented Oct 16, 2016 •
edited

I'll try to mess with it this week. For a workaround you can probably make a profile with the config and pass the profile to kitchen there an option for that.

@bradenwright
Copy link
Owner

I'll try to spend sometime tonight/this week.

@bradenwright
Copy link
Owner

So fyi at least this works:

cat .kitchen.lxd.yml 
platforms:
- name: ubuntu-14.04

suites:
- name: sensu_server
  driver_config:
    config: 
      raw.lxc: "lxc.cap.drop=setfcap"
  run_list:
  - recipe[olab_sensu_server]

lxc config show sensu-server-ubuntu-1404-1473825082 
name: sensu-server-ubuntu-1404-1473825082
profiles:
- default
config:
  limits.cpu: "1"
  limits.memory: 2GB
  raw.lxc: lxc.cap.drop=setfcap
  volatile.base_image: 63fbed3b9b2b82bd896150c593ab85360474f411c31b91fa4ffd2da5370f458a
  volatile.eth0.hwaddr: 00:16:3e:e9:d7:f0
  volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":165536,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":165536,"Nsid":0,"Maprange":65536}]'
devices:
  root:
    path: /
    type: disk
ephemeral: false

@bradenwright
Copy link
Owner

It does look if you try to set something like an ip or a gateway that you aren't able to set a raw config setting. I incorrectly overwrite it. So I'll leave this open to try and fix that.

@juju4
Copy link
Author

juju4 commented Oct 16, 2016

Thanks a lot @bradenwright
I gave a try to

raw.lxc: lxc.cap.drop=setfcap

but sadly, I need it in the reverse way, drop all but setfcap (https://bugzilla.redhat.com/attachment.cgi?id=804061&action=diff) and it seems spaces may be an issue here.

## .kitchen.yml
#    raw.lxc: "lxc.cap.drop='audit_control audit_write mac_admin mac_override mknod setpcap sys_admin sys_boot sys_module sys_nice sys_pacct sys_rawio sys_resource sys_time sys_tty_config'"
$ kitchen test default-centos-7 -l debug
-----> Starting Kitchen (v1.13.2)
-----> Cleaning up any prior instances of <default-centos-7>
-----> Destroying <default-centos-7>...
D      Container default-centos-7-1476600655 exists
D      Container default-centos-7-1476600655 is running
       Stopping container default-centos-7-1476600655
D      run_local_command ran: lxc stop default-centos-7-1476600655 --force
D      Command finished: pid 7009 exit 0
       Deleting container default-centos-7-1476600655
D      run_local_command ran: lxc delete default-centos-7-1476600655 --force
D      Command finished: pid 7089 exit 0
       Finished destroying <default-centos-7> (0m1.29s).
-----> Testing <default-centos-7>
-----> Creating <default-centos-7>...
D      Container default-centos-7-1476600655 doesn't exist
D      Publish Image Name is kitchen-default-centos-7
D      Image Name is centos-7
D      Image centos-7 exists
D      Config Args:  -c raw.lxc=lxc.cap.drop='audit_control audit_write mac_admin mac_override mknod setpcap sys_admin sys_boot sys_module sys_nice sys_pacct sys_rawio sys_resource sys_time sys_tty_config'
       Initializing container default-centos-7-1476600655
D      run_local_command ran: lxc init centos-7 default-centos-7-1476600655   -c raw.lxc=lxc.cap.drop='audit_control audit_write mac_admin mac_override mknod setpcap sys_admin sys_boot sys_module sys_nice sys_pacct sys_rawio sys_resource sys_time sys_tty_config'
D      Command finished: pid 7106 exit 0
D      Container default-centos-7-1476600655 isn't running
       Starting container default-centos-7-1476600655
D      run_local_command ran: lxc start default-centos-7-1476600655 
D      Command finished: pid 7121 exit 0
D      Setting up /etc/hosts
D      Waiting for /etc/hosts to become available...
D      run_local_command ran: lxc exec default-centos-7-1476600655 -- ls /etc/hosts > /dev/null 2>&1
D      Command finished: pid 7234 exit 0
D      Found /etc/hosts
       Waiting for network to become ready
D      Still waiting for IP Address...
D      Still waiting for IP Address...
D      Still waiting for IP Address...
[stalling here...]

## .kitchen.yml
#    raw.lxc: "lxc.cap.drop=audit_control audit_write mac_admin mac_override mknod setpcap sys_admin sys_boot sys_module sys_nice sys_pacct sys_rawio sys_resource sys_time sys_tty_config"
$ kitchen test default-centos-7 -l debug
-----> Starting Kitchen (v1.13.2)
-----> Cleaning up any prior instances of <default-centos-7>
-----> Destroying <default-centos-7>...
D      Container default-centos-7-1476600655 exists
D      Container default-centos-7-1476600655 is running
       Stopping container default-centos-7-1476600655
D      run_local_command ran: lxc stop default-centos-7-1476600655 --force
D      Command finished: pid 7422 exit 0
       Deleting container default-centos-7-1476600655
D      run_local_command ran: lxc delete default-centos-7-1476600655 --force
D      Command finished: pid 7503 exit 0
       Finished destroying <default-centos-7> (0m1.46s).
-----> Testing <default-centos-7>
-----> Creating <default-centos-7>...
D      Container default-centos-7-1476600655 doesn't exist
D      Publish Image Name is kitchen-default-centos-7
D      Image Name is centos-7
D      Image centos-7 exists
D      Config Args:  -c raw.lxc=lxc.cap.drop=audit_control audit_write mac_admin mac_override mknod setpcap sys_admin sys_boot sys_module sys_nice sys_pacct sys_rawio sys_resource sys_time sys_tty_config
       Initializing container default-centos-7-1476600655
D      run_local_command ran: lxc init centos-7 default-centos-7-1476600655   -c raw.lxc=lxc.cap.drop=audit_control audit_write mac_admin mac_override mknod setpcap sys_admin sys_boot sys_module sys_nice sys_pacct sys_rawio sys_resource sys_time sys_tty_config
Initialize a container from a particular image.

lxc init [remote:]<image> [remote:][<name>] [--ephemeral|-e] [--profile|-p <profile>...] [--config|-c <key=value>...]

Initializes a container using the specified image and name.

Not specifying -p will result in the default profile.   
Specifying "-p" with no argument will result in no profile.

Example:
lxc init ubuntu u1

error: wrong number of subcommand arguments
D      Command finished: pid 7520 exit 1
error: not found

first case should be correct but for some reason, network is lost while in use.
no issue if privileged, or with drop=setfcap (but fails to install httpd)
tried to restart my lxd-bridge without improvements.

If I do without any special config, initialization is correct
and with manual settings

$ lxc config show default-centos-7-1476600655
name: default-centos-7-1476600655
profiles:
- default
config:
  volatile.base_image: 5ca7349e918b0e10bb845b42b75574f17ac5fa593b75391ecc7ee5c3c6f25711
  volatile.eth0.hwaddr: 00:16:3e:51:ba:6b
  volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":100000,"Nsid":0,"Maprange":65536}]'
devices:
  root:
    path: /
    type: disk
ephemeral: false
$ lxc config set default-centos-7-1476600655 raw.lxc "lxc.cap.drop='audit_control audit_write mac_admin mac_override mknod setpcap sys_admin sys_boot sys_module sys_nice sys_pacct sys_rawio sys_resource sys_time sys_tty_config"
$ lxc config show default-centos-7-1476600655
name: default-centos-7-1476600655
profiles:
- default
config:
  raw.lxc: lxc.cap.drop='audit_control audit_write mac_admin mac_override mknod setpcap
    sys_admin sys_boot sys_module sys_nice sys_pacct sys_rawio sys_resource sys_time
    sys_tty_config
  volatile.base_image: 5ca7349e918b0e10bb845b42b75574f17ac5fa593b75391ecc7ee5c3c6f25711
  volatile.eth0.hwaddr: 00:16:3e:51:ba:6b
  volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":100000,"Nsid":0,"Maprange":65536}]'
devices:
  root:
    path: /
    type: disk
ephemeral: false

strangely, I still got at httpd install

Installing : httpd-tools-2.4.6-40.el7.centos.4.x86_64                  36/107 \n  Installing :
 httpd-2.4.6-40.el7.centos.4.x86_64                        37/107 \nerror: unpacking of archive failed on file /usr/sbin/suexec: cpio: cap_set_file

same result with dropping all capabilities

[container already created]
$ lxc config set default-centos-7-1476600655 raw.lxc "lxc.cap.drop="
$ kitchen verify default-centos-7
[...]
Installing : httpd-2.4.6-40.el7.centos.4.x86_64                           1/2 \nerror: unpacking of archive failed on file /usr/sbin/suexec: cpio: cap_set_file\n  Installing : fping-3.10-4.el7.x86_64                                      2/2 \nerror: httpd-2.4.6-40.el7.centos.4.x86_64: install failed\nerror: unpacking of archive failed on file /usr/sbin/fping: cpio: cap_set_file\n  Verifying  : fping-3.10-4.el7.x86_64                                      1/2 \n  Verifying  : httpd-2.4.6-40.el7.centos.4.x86_64                           2/2 \n\nFailed:\n  fping.x86_64 0:3.10-4.el7         httpd.x86_64 0:2.4.6-40.el7.centos.4

rechecking through lxc issue it seems there is no alternative to privileged true unlike redhat bug thread was suggesting :(
https://github.com/lxc/lxd/issues/1245

Only tradeoff, I moved the config privileged from global to just centos platform.

to summarize, outside of a possible space issue, option is available and not much more to do currently.

Thanks a lot Braden!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@juju4 @bradenwright