Linux kernel is not determined correctly

[myhro]

I noticed that when building a virtualbox image with the docker_daemon plugin, the linux-image package was installed from backports, as expected, but linux-headers don't. This makes the Guest Additions modules built during the process to not work with the backported kernel. Looking at the AddGuestAdditionsPackages task, I see that the detected kernel version is actually the one from the host which is running bootstrap-vz, as chroot doesn't load a new kernel.

Any suggestions on how we can detect the correct version of the kernel that is going to be installed?

[andsens]

Oh boy. I had no clue that uname returns the currently running kernel. That's why Guest Additions doesn't work when bootstrapping jessie from wheezy. In hindsight it makes sense of course. uname would have no idea what to return when there is more than one kernel installed.

I think this is where @rybaktomasz's idea about handling kernel installation a bit more centralized comes in.

[myhro]

That's why Guest Additions doesn't work when bootstrapping jessie from wheezy.

Yup.

I think this is where @rybaktomasz's idea about handling kernel installation a bit more centralized comes in.

How that would work?

I've created a task to identify the kernel that is going to be installed parsing the output of apt-cache depends linux-image-* and that solved the problem of installing the correct linux-headers version. The problem now is that I got an The headers for the current running kernel were not found. error, because the Guest Additions installation is still running on a chroot under the host kernel... And I'm not really sure if there's a workaround for that. :-(

[andsens]

Nice going Tiago. This is maybe where python-apt could have been helpful. Parsing output lines can be a bit... unstable.

I'll try and look through the install script for the guest additions, maybe we can monkey patch it, so it doesn't fail. In any case, this should be a bug report to the VirtualBox team.

[myhro]

This is maybe where python-apt could have been helpful. Parsing output lines can be a bit... unstable.

I've tried to use python-apt to do the same thing (albeit using a dirty ugly hack to import it without clashing with bootstrapvz.common.tools.apt, which is located in the same directory). The problem is that the candidate it chooses diverges from the one shown by apt-cache. So, even detecting both versions, it doesn't return the backport one.

ipdb> log_check_call(['chroot', info.root, 'apt-cache', 'policy', info.kernel_package])
Executing: chroot /target/e51cd130/root apt-cache policy linux-image-amd64
linux-image-amd64:
Installed: (none)
Candidate: 3.14+57~bpo70+1
Package pin: 3.14+57~bpo70+1
Version table:
3.14+57~bpo70+1 800
100 http://gce_debian_mirror.storage.googleapis.com/ wheezy-backports/main amd64 Packages
3.2+46 800
500 http://gce_debian_mirror.storage.googleapis.com/ wheezy/main amd64 Packages
['linux-image-amd64:', 'Installed: (none)', 'Candidate: 3.14+57~bpo70+1', 'Package pin: 3.14+57~bpo70+1', 'Version table:', '3.14+57~bpo70+1 800', '100 http://gce_debian_mirror.storage.googleapis.com/ wheezy-backports/main amd64 Packages', '3.2+46 800', '500 http://gce_debian_mirror.storage.googleapis.com/ wheezy/main amd64 Packages']
ipdb> kernel_package_cache.candidate
<Version: package:'linux-image-amd64' version:'3.2+46'>
ipdb> kernel_package_cache.versions
<VersionList: ['3.14+57~bpo70+1', '3.2+46']>

I'll try and look through the install script for the guest additions, maybe we can monkey patch it, so it doesn't fail.

Oh... That awesome shell script full of binary data. I was really hoping that we don't touch that again. :-(
Anyway, this approach will be interesting if it works, as we can apply it to the existing versions of the installer.

In any case, this should be a bug report to the VirtualBox team.

Would be nice if we could just pass the kernel version as a parameter, environment variable or something like that. Do you mind of reporting it at their bug tracker? I guess you are already registered there.

[rybaktomasz]

Should I try to provide some working python-apt implementation, or is it to early now?

[andsens]

I don't think we'll switch to python-apt before releasing 1.0, the change is simply too big. A proof of concept would be nice though, an amuse-bouche if you will :-)
The pre-condition of course being that it will indeed be easier to maintain and interface with than what we have now.

[olberger]

May I suggest to rename this issue ? I'm not sure this is specific to backported kernels. Also stands for generation of stable targe images on testing host, for instance.

[olberger]

FWIW, when playing with vagrant provisionning scripts, I experienced this dkms dependencies module building issue, which I solved in the following way: building images with vagrant up && vagrant reload --provisioning.

The 2 runs will execute the same provisionning script which does something like this:

installed=$(apt-cache policy linux-image-amd64 | grep 'Installed:' | sed 's/.*Installed: //g')
candidate=$(apt-cache policy linux-image-amd64 | grep 'Candidate:' | sed 's/.*Candidate: //g')
echo "installed: $installed"
echo "candidate: $candidate"

if [ "$installed" != "$candidate" ]; then
    apt-get install --yes virtualbox-guest-utils linux-headers-amd64 linux-image-amd64 
else
...

This way, first we update the kernel, and then do the rest only after reboot with this kernel, so in principle, /vagrant is always mountable.

Hope this helps, even though unrelated, as bootstrap-vz doesn't run vagrant.

[andsens]

Renamed the issue. I'm not sure how to fix this right now. I think a temporary stop-gap measure would be to deny building images on hosts where the kernel version differs from the one on the guest (only when installing guest additions of course).

[myhro]

Although this wouldn't be a very clean approach... Maybe we could just postpone the Guest Additions installation script to run on the first boot of the box?

[andsens]

Maybe we could just postpone the Guest Additions installation script to run on the first boot of the box?

... effectively doubling the vagrant up time when a machine has not been created yet? Both approaches are kind of ugly. Thing is, we need to merge into master soon if we want to make it before the feature freeze in jessie (god, google semantic search is awesome).

[myhro]

Thing is, we need to merge into master soon if we want to make it before the feature freeze in jessie

In this case, I guess your suggestion of blocking the image build process is the way to go.

(god, google semantic search is awesome)

They beat WolframAlpha on this one. :-)

[myhro]

Based on the suggestion that @olberger made on #130 (which didn't worked for wheezy) and his comment here about the provision script, I've fixed this by adding the corresponding virtual linux-headers-* package, instead of trying to detect the kernel version.

I'll tag this issue as fixed in dev, but we have a bigger problem to deal: the Guest Additions installation when the target has a different kernel. Tracking it on another issue would be better.

[andsens]

Agreed. Nice job ;-)

[myhro]

Thank you, Anders. :-)