Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docker container fails to build examples #154

Closed
QuickJack opened this issue Apr 26, 2018 · 14 comments
Closed

docker container fails to build examples #154

QuickJack opened this issue Apr 26, 2018 · 14 comments
Labels

Comments

@QuickJack
Copy link
Contributor

Within the docker container (branch release/v2.9.9), it is not possible to build the elbe examples:

root@11f083760dfc:/home/elbe/elbe# ./elbe initvm create ./examples/armhf-ti-beaglebone-black.xml
Traceback (most recent call last):
File "./elbe", line 62, in
mod = import(modname)
File "/home/elbe/elbe/elbepack/commands/initvm.py", line 25, in
from elbepack.initvmaction import InitVMAction, InitVMError
File "/home/elbe/elbe/elbepack/initvmaction.py", line 35, in
import libvirt
ImportError: No module named libvirt

@QuickJack
Copy link
Contributor Author

And after installing the missing python-libvirt:

root@11f083760dfc:/home/elbe/elbe# apt-get install python-libvirt
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
python-libvirt
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 1431 kB of archives.
After this operation, 2229 kB of additional disk space will be used.
Get:1 http://security.debian.org/ wheezy/updates/main python-libvirt amd64 0.9.12.3-1+deb7u3 [1431 kB]
Fetched 1431 kB in 0s (5874 kB/s)
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LANG = "de_DE.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package python-libvirt.
(Reading database ... 24714 files and directories currently installed.)
Unpacking python-libvirt (from .../python-libvirt_0.9.12.3-1+deb7u3_amd64.deb) ...
Setting up python-libvirt (0.9.12.3-1+deb7u3) ...
root@11f083760dfc:/home/elbe/elbe# ./elbe initvm create ./examples/armhf-ti-beaglebone-black.xml
libvir: RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
Traceback (most recent call last):
File "./elbe", line 65, in
cmdmod.run_command(sys.argv[2:])
File "/home/elbe/elbe/elbepack/commands/initvm.py", line 93, in run_command
action = InitVMAction(args[0])
File "/home/elbe/elbe/elbepack/initvmaction.py", line 221, in init
InitVMAction.init(self, node, initvmNeeded=False)
File "/home/elbe/elbe/elbepack/initvmaction.py", line 83, in init
self.conn = libvirt.open("qemu:///session")
File "/usr/lib/python2.7/dist-packages/libvirt.py", line 236, in open
if ret is None:raise libvirtError('virConnectOpen() failed')
libvirt.libvirtError: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory

@manut
Copy link
Contributor

manut commented Apr 26, 2018

both problems are fixed by: e6cdc7e and will be in the next elbe-testing release.

@manut manut closed this as completed Apr 26, 2018
@QuickJack
Copy link
Contributor Author

After rebuilding the docker container (v2.9.9) I got this:

elbe@7104bf6e1806:~/elbe$ ./elbe initvm create ./examples/armhf-ti-beaglebone-black.xml
libvirt: XML-RPC error : Failed to connect socket to '/home/elbe/.cache/libvirt/libvirt-sock': No such file or directory
Traceback (most recent call last):
  File "./elbe", line 65, in <module>
    cmdmod.run_command(sys.argv[2:])
  File "/home/elbe/elbe/elbepack/commands/initvm.py", line 93, in run_command
    action = InitVMAction(args[0])
  File "/home/elbe/elbe/elbepack/initvmaction.py", line 221, in __init__
    InitVMAction.__init__(self, node, initvmNeeded=False)
  File "/home/elbe/elbe/elbepack/initvmaction.py", line 83, in __init__
    self.conn = libvirt.open("qemu:///session")
  File "/usr/lib/python2.7/dist-packages/libvirt.py", line 255, in open
    if ret is None:raise libvirtError('virConnectOpen() failed')
libvirt.libvirtError: Failed to connect socket to '/home/elbe/.cache/libvirt/libvirt-sock': No such file or directory
elbe@7104bf6e1806:~/elbe$ /etc/init.d/libvirtd restart
[....] Restarting libvirt management daemon: /usr/sbin/libvirtd/usr/sbin/libvirtd: error while loading shared libraries: libvirt-admin.so.0: cannot open shared object file: Permission denied
 failed!

We can see that libvirtd is not running. I used ssh to login to the container as user "elbe". Logging in as "root" does not work.

@manut
Copy link
Contributor

manut commented Apr 26, 2018

seems that the user is not in the libvirt group

@manut manut reopened this Apr 26, 2018
@QuickJack
Copy link
Contributor Author

I am using Ubuntu as the host OS. The libvirt apparmor profile needs to be disabled on the host as follows:

sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.libvirtd

@QuickJack
Copy link
Contributor Author

QuickJack commented Apr 26, 2018

And here is the next problem:

elbe@7104bf6e1806:~/elbe$ ./elbe initvm create ./examples/armhf-ti-beaglebone-black.xml
libvirt: QEMU Driver error : Domain not found: no domain with matching name 'initvm'
gpg: keybox '/tmp/tmpXHJrxW/etc/apt/trusted.gpg' created
gpg: key 36AA35FF22BB8F84: public key "ELBE Devel (Linutronix Elbe developers) <elbe-devel@linutronix.de>" imported
gpg: Total number processed: 1
gpg:               imported: 1
Import debian-archive-jessie-security-automatic.gpg: 
gpg: key 9D6D8F6BC857C906: public key "Debian Security Archive Automatic Signing Key (8/jessie) <ftpmaster@debian.org>" imported
gpg: Total number processed: 1
gpg:               imported: 1
Import debian-archive-wheezy-stable.gpg: 
gpg: key 6FB2A1C265FFB764: public key "Wheezy Stable Release Key <debian-release@lists.debian.org>" imported
gpg: Total number processed: 1
gpg:               imported: 1
Import debian-archive-stretch-security-automatic.gpg: 
gpg: key EDA0D2388AE22BA9: public key "Debian Security Archive Automatic Signing Key (9/stretch) <ftpmaster@debian.org>" imported
gpg: Total number processed: 1
gpg:               imported: 1
Import debian-archive-jessie-automatic.gpg: 
gpg: key 7638D0442B90D010: public key "Debian Archive Automatic Signing Key (8/jessie) <ftpmaster@debian.org>" imported
gpg: Total number processed: 1
gpg:               imported: 1
Import debian-archive-jessie-stable.gpg: 
gpg: key CBF8D6FD518E17E1: public key "Jessie Stable Release Key <debian-release@lists.debian.org>" imported
gpg: Total number processed: 1
gpg:               imported: 1
Import debian-archive-stretch-automatic.gpg: 
gpg: key E0B11894F66AEC98: public key "Debian Archive Automatic Signing Key (9/stretch) <ftpmaster@debian.org>" imported
gpg: Total number processed: 1
gpg:               imported: 1
Import debian-archive-stretch-stable.gpg: 
gpg: key EF0F382A1A7B6500: public key "Debian Stable Release Key (9/stretch) <debian-release@lists.debian.org>" imported
gpg: Total number processed: 1
gpg:               imported: 1
Import debian-archive-wheezy-automatic.gpg: 
gpg: key 8B48AD6246925553: public key "Debian Archive Automatic Signing Key (7.0/wheezy) <ftpmaster@debian.org>" imported
gpg: Total number processed: 1
gpg:               imported: 1
--2018-04-26 09:21:25--  http://debian.linutronix.de/elbe-testing/elbe-repo.pub
Connecting to 172.18.0.100:8080... connected.
Proxy request sent, awaiting response... 200 OK
Length: 2708 (2.6K)
Saving to: '/tmp/tmpXHJrxW/tmpkey.gpg'

/tmp/tmpXHJrxW/tmpkey.gpg           100%[================================================================>]   2.64K  --.-KB/s    in 0s      

2018-04-26 09:21:25 (355 MB/s) - '/tmp/tmpXHJrxW/tmpkey.gpg' saved [2708/2708]

gpg: key 36AA35FF22BB8F84: "ELBE Devel (Linutronix Elbe developers) <elbe-devel@linutronix.de>" 4 new signatures
gpg: Total number processed: 1
gpg:         new signatures: 4
--2018-04-26 09:21:26--  http://debian.linutronix.de/elbe-common/elbe-repo.pub
Connecting to 172.18.0.100:8080... connected.
Proxy request sent, awaiting response... 200 OK
Length: 2708 (2.6K)
Saving to: '/tmp/tmpXHJrxW/tmpkey.gpg'

/tmp/tmpXHJrxW/tmpkey.gpg           100%[================================================================>]   2.64K  --.-KB/s    in 0s      

2018-04-26 09:21:26 (339 MB/s) - '/tmp/tmpXHJrxW/tmpkey.gpg' saved [2708/2708]

gpg: key 36AA35FF22BB8F84: "ELBE Devel (Linutronix Elbe developers) <elbe-devel@linutronix.de>" not changed
gpg: Total number processed: 1
gpg:              unchanged: 1
Reading package lists... Done
Building dependency tree... Done
rm: cannot remove '/var/cache/apt/archives/partial/*.deb': Permission denied
Reading package lists... Done
Building dependency tree... Done
rm: cannot remove '/var/cache/apt/archives/partial/*.deb': Permission denied
--2018-04-26 09:21:38--  http://debian.linutronix.de/elbe-common/pool/main/e/elbe-bootstrap/elbe-bootstrap_20150422+deb8u5+jessie141_amd64.deb
Connecting to 172.18.0.100:8080... connected.
Proxy request sent, awaiting response... 200 OK
Length: 24585876 (23M) [application/x-debian-package]
Saving to: '/tmp/tmpTtWtCZ/elbe-bootstrap.deb'

/tmp/tmpTtWtCZ/elbe-bootstrap.deb   100%[================================================================>]  23.45M  3.88MB/s    in 6.2s    

2018-04-26 09:21:44 (3.78 MB/s) - '/tmp/tmpTtWtCZ/elbe-bootstrap.deb' saved [24585876/24585876]

make: lsb_release: Command not found
rm -rf tmp-tree
mkdir tmp-tree
cp .elbe-in/*.cfg tmp-tree/
cp .elbe-in/apt.conf tmp-tree/
mkdir -p tmp-tree/etc/apt
cp .elbe-in/apt.conf tmp-tree/etc/apt
mkdir -p tmp-tree/usr/lib/post-base-installer.d
cp .elbe-in/init-elbe.sh tmp-tree/
cp .elbe-in/source.xml tmp-tree/
mkdir -p tmp-tree/usr/share/keyrings
cp .elbe-in/*.gpg tmp-tree/usr/share/keyrings
cp: cannot stat '.elbe-in/*.gpg': No such file or directory
Makefile:27: recipe for target '.elbe-gen/initrd-preseeded.gz' failed
make: [.elbe-gen/initrd-preseeded.gz] Error 1 (ignored)
mkdir -p .elbe-gen
gzip -cd .elbe-in/initrd.gz >.elbe-gen/initrd-preseeded
cd tmp-tree && find . | cpio -H newc -o --append -F ../.elbe-gen/initrd-preseeded
17 blocks
gzip -9f .elbe-gen/initrd-preseeded
rm -rf tmp-tree
qemu-img create -f qcow2 buildenv.img 80G
Formatting 'buildenv.img', fmt=qcow2 size=85899345920 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16
mkdir -p .stamps
touch .stamps/stamp-create-buildenv-img
/usr/bin/kvm
Could not access KVM kernel module: Permission denied
failed to initialize KVM: Permission denied

------------------------------------------------------------------
kvm failed to start
This is most likely the case, because /dev/kvm is not available.
To use KVM inside a VMWARE or other VM instance,
nested KVM needs to be supported
------------------------------------------------------------------

Makefile:50: recipe for target '.stamps/stamp-install-initial-image' failed
make: *** [.stamps/stamp-install-initial-image] Error 1
Building the initvm Failed
Giving up
elbe@7104bf6e1806:~/elbe$ ls /dev/kvm
/dev/kvm

User "elbe" does not have the correct permissions to use KVM.

I resolved it by accessing the container directly using "docker exec -it <container id> bash". It seems to be working now.

It would be nice to have a step by step documentation to build the examples using docker on an Ubuntu host machine.

@QuickJack
Copy link
Contributor Author

Unfortunately, there are even more problems when the virtual machine gets restarted:

(...)
Installing the base system  ... 17%... 20%... 30%... 40%... 50%... 60%... 70%... 83%... 91%... 100%
Configuring apt  ... 14%... 21%... 35%... 40%... 50%... 64%... 71%... 85%... 92%... 100%
Select and install software  ... 10%... 20%... 30%... 41%... 50%... 60%... 70%... 81%... 90%... 100%
Installing GRUB boot loader  ... 16%... 33%... 50%... 66%... 83%... 100%
The system is going down NOW!.. 13%... 22%... 31%... 40%... 50%... 63%... 72%... 81%... 90%
Sent SIGTERM to all processes
Sent SIGKILL to all processes
Requesting system reboot
[ 1064.769560] reboot: Restarting system
mkdir -p .stamps
touch .stamps/stamp-install-initial-image
libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/virtlogd-sock': No such file or directory
Traceback (most recent call last):
  File "/home/elbe/elbe/elbe", line 65, in <module>
    cmdmod.run_command(sys.argv[2:])
  File "/home/elbe/elbe/elbepack/commands/initvm.py", line 100, in run_command
    action.execute(directory, opt, args[1:])
  File "/home/elbe/elbe/elbepack/initvmaction.py", line 109, in execute
    self.initvm.create()
  File "/usr/lib/python2.7/dist-packages/libvirt.py", line 1035, in create
    if ret == -1: raise libvirtError ('virDomainCreate() failed', dom=self)
libvirt.libvirtError: Failed to connect socket to '/var/run/libvirt/virtlogd-sock': No such file or directory
Starting the initvm Failed
Giving up

@QuickJack
Copy link
Contributor Author

The problem "libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/virtlogd-sock': No such file or directory
Traceback (most recent call last)" is because virtlogd is not running:

root@7104bf6e1806:/home/elbe/elbe# /etc/init.d/virtlogd status
[FAIL] Checking status of libvirt logging daemon: virtlogd not running failed!
root@7104bf6e1806:/home/elbe/elbe# /etc/init.d/virtlogd restart
[ ok ] Restarting libvirt logging daemon: /usr/sbin/virtlogd.
root@7104bf6e1806:/home/elbe/elbe# /etc/init.d/virtlogd status
[ ok ] Checking status of libvirt logging daemon: virtlogd running.

After that, the build still fails as follows:


(...)
Installing the base system  ... 17%... 20%... 30%... 40%... 50%... 60%... 70%... 83%... 91%... 100%
Configuring apt  ... 14%... 21%... 35%... 40%... 50%... 64%... 71%... 85%... 92%... 100%
Select and install software  ... 10%... 20%... 30%... 41%... 50%... 60%... 70%... 81%... 90%... 100%
Installing GRUB boot loader  ... 16%... 33%... 50%... 66%... 83%... 100%
The system is going down NOW!.. 13%... 22%... 31%... 40%... 50%... 63%... 72%... 81%... 90%
Sent SIGTERM to all processes
Sent SIGKILL to all processes
Requesting system reboot
[ 1002.969116] reboot: Restarting system
mkdir -p .stamps
touch .stamps/stamp-install-initial-image
libvirt: QEMU Driver error : internal error: qemu unexpectedly closed the monitor: qemu-system-x86_64: -redir tcp:5022::22: The -redir option is deprecated. Please use '-netdev user,hostfwd=...' instead.
qemu-system-x86_64: -redir tcp:7587::7588: The -redir option is deprecated. Please use '-netdev user,hostfwd=...' instead.
Could not access KVM kernel module: Permission denied
failed to initialize KVM: Permission denied
Traceback (most recent call last):
  File "/home/elbe/elbe/elbe", line 65, in <module>
    cmdmod.run_command(sys.argv[2:])
  File "/home/elbe/elbe/elbepack/commands/initvm.py", line 100, in run_command
    action.execute(directory, opt, args[1:])
  File "/home/elbe/elbe/elbepack/initvmaction.py", line 109, in execute
    self.initvm.create()
  File "/usr/lib/python2.7/dist-packages/libvirt.py", line 1035, in create
    if ret == -1: raise libvirtError ('virDomainCreate() failed', dom=self)
libvirt.libvirtError: internal error: qemu unexpectedly closed the monitor: qemu-system-x86_64: -redir tcp:5022::22: The -redir option is deprecated. Please use '-netdev user,hostfwd=...' instead.
qemu-system-x86_64: -redir tcp:7587::7588: The -redir option is deprecated. Please use '-netdev user,hostfwd=...' instead.
Could not access KVM kernel module: Permission denied
failed to initialize KVM: Permission denied
Starting the initvm Failed
Giving up

@QuickJack
Copy link
Contributor Author

QuickJack commented Apr 30, 2018

I also resolved the above problem with:

root@7104bf6e1806:/home/elbe/elbe# chgrp kvm /dev/kvm

But unfortunately, it's still not working yet:

(...)
Installing the base system  ... 17%... 20%... 30%... 40%... 50%... 60%... 70%... 83%... 91%... 100%
Configuring apt  ... 14%... 21%... 35%... 40%... 50%... 64%... 71%... 85%... 92%... 100%
Select and install software  ... 10%... 20%... 30%... 41%... 50%... 60%... 70%... 81%... 90%... 100%
Installing GRUB boot loader  ... 16%... 33%... 50%... 66%... 83%... 100%
The system is going down NOW!.. 13%... 22%... 31%... 40%... 50%... 63%... 72%... 81%... 90%
Sent SIGTERM to all processes
Sent SIGKILL to all processes
Requesting system reboot
[  740.736290] reboot: Restarting system
mkdir -p .stamps
touch .stamps/stamp-install-initial-image
*****
Build started, waiting till it finishes

Build finished !

Server returned error:

OSError: [Errno 2] No such file or directory: '/var/cache/elbe/5e53fb9b-9d92-485c-9650-d55ea8508b0d/validation.txt'
Project failed to generate validation.txt
Getting log.txt
running cmd +reprepro --basedir "/var/cache/elbe/5e53fb9b-9d92-485c-9650-d55ea8508b0d/repo" export stretch+
------------------------------------------------------------------------------
------------------------------------------------------------------------------

Enqueueing project for build
Build started

ELBE Report for Project beaglebone-black
========================================

report timestamp: 20180430-113420
Build failed
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/elbepack/asyncworker.py", line 140, in execute
    skip_pbuild=self.skip_pbuilder)
  File "/usr/lib/python2.7/dist-packages/elbepack/elbeproject.py", line 354, in build
    self.xml.validate_apt_sources(m, self.arch)
  File "/usr/lib/python2.7/dist-packages/elbepack/elbexml.py", line 259, in validate_apt_sources
    ["Repository %s can not be validated" % r["url"]])
ValidationError: Repository http://ftp.de.debian.org//debian/dists/stretch/ can not be validated



Exception:
Repository http://ftp.de.debian.org//debian/dists/stretch/ can not be validated

I am using a corporate proxy. But http_proxy and https_proxy are set accordingly. It seems that the "ValidationError" in the last line is generated here:

fp = urllib2.urlopen(u, None, 10)

I guess that the proxy environment is not set up correctly within the vm. Access to http://ftp.de.debian.org//debian/dists/stretch/ works perfectly well from my browser on the host system.

@QuickJack
Copy link
Contributor Author

QuickJack commented May 2, 2018

This is a proxy related issue which can be fixed by modifying the example .xml file. A <primary_proxy>myproxy:myproxy_port<primary_proxy> section must be put right below the <primary_proto><primary_proto> section.

Putting all of my findings together, the build within a proxied environment on an Ubuntu host works as follows:

  1. Disable apparmor from blocking access to libvirtd by executing "sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.libvirtd" on the host
  2. Build the docker image according to the docs and add "--build-args <myproxy:myproxy_port>
  3. Start the docker container according to the docs
  4. Login to the docker container using "sudo docker exec -it <container id> bash
  5. Manually add /dev/kvm to the kvm group by entering "chgrp kvm /dev/kvm"
  6. Set the http_proxy and https_proxy environment variables accordingly
  7. Modify the example .xml file by adding the proxy settings as described above
  8. cd to /elbe and start the build according to the docs

Please update your docs. The build should work on Ubuntu. Also note, that most companies do have a corporate proxy. There should be an extra section which explains all the necessary steps.

@manut
Copy link
Contributor

manut commented May 3, 2018

Thanks for your efforts and good description! You're also welcome to send pathes against docs/quickstart.txt that is shown if you click on 'Start using elbe' on elbe-rfs.org.

We will keep your findings in mind, because we're planning to providue ubuntu binary packages. Therefore i'd liko to keep this issue open.

Because of http_proxy environment variables. I like the idea! Feel free to open a new issue for this, or send a patch to elbe-devel@linutronix.de

@QuickJack
Copy link
Contributor Author

The proxy related problems (2 and 6) are fixed by 8a39dd2. However, step 7 is still required!

To build the image behind a corporate proxy use:
sudo -E make build

To create/run the container behind a corporate proxy use:
sudo -E make run

@szigetiszabolcs80
Copy link

mkdir -p .stamps
touch .stamps/stamp-create-buildenv-img
/usr/bin/kvm
Could not access KVM kernel module: Permission denied
failed to initialize KVM: Permission denied


kvm failed to start
This is most likely the case, because /dev/kvm is not available.
To use KVM inside a VMWARE or other VM instance,
nested KVM needs to be supported

You must enable the hardware virtualization in the bios.
You can check if your CPU support it with this:
egrep -c '(vmx|svm)' /proc/cpuinfo
will give you back a number which is bigger as 0.

The cpu-checker is very useful too. You must download this from the Ubuntu webpage ( packages.ubuntu.com ). I was able to setup it on Debian with:
sudo apt install ./ cpu-checker_0.7-1.1_amd64.deb

Then just write:
sudo kvm-ok
If the cpu virtualization is disabled in the bios it will write that. Otherwise if everything is ok then you can see this:

INFO: /dev/kvm exists
KVM acceleration can be used

Then the Elbe can work too.

@bene42
Copy link
Contributor

bene42 commented Oct 25, 2023

Cannot reproduce. More info needed. Please reopen if nessesary.

@bene42 bene42 closed this as completed Oct 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants