eudev doesn't work, /dev not populated

[fluxer]

I've made a PKGBUILD for Arch Linux, builded the sources and installed it. After rebuilding my intramfs image and rebooting it seems that udev doesn't populate nodes for my storage devices and I'm unable to boot. I've tried doing it manually from the intramfs with:

udevd --daemon
udevadm trigger
udevadm settle

but that didn't helped. Since I was using the udev-fork of braindamaged and it worked for me I tried to use the rules from it and copied them over those that eudev provided, that didn't helped either.

After executing:

udevd --daemon --debug

I saw that there are a few warnings about missing groups - storage, tty and more. I can not provide the full output tough, I can write it all by hand from the screen to a text file but I can not provide screenshot nor video.

Source for my PKGBUILD at http://dl.dropbox.com/u/54183088/eudev-git-20121119-1.src.tar.gz

[ryao]

The problem is most likely that we are missing rules that are needed for proper module loading without kmod. A solution is in the kmod branch. It will be merged after proper review.

[ryao]

@fluxer This issue should be fixed in HEAD. Please let me know if you are still having this problem.

[fluxer]

Still no joy. Executing from the initramfs

udevd --daemon --debug

tells me that /etc/udev/hwdb.bin is missing.

[blueness]

Thanks for the report! What's going on here is that the hwdb files are being installed into

 udevhwdbdir = $(udevlibexecdir)/hwdb.d

where this is substituted at configure time

udevlibexecdir=@udevlibexecdir@

but, udevadm-hwdb.c is hard coded to look in /etc/udev/hwdb.bin. systemd upstream has already fixed this by adding another AM_CPPFLAGS -DHWDB_BIN=hwdb_bin where the later is still hard set to hwdb_bin=/etc/udev/hwdb.bin, only now in their Makefile.am rather than their c code.

I don't completely like this solution since it disconnects hwdb_bin from a configurable directory which is substituted at configure time. I will half adopt their solution.

[lu-zero]

On Fri, Nov 23, 2012 at 5:38 PM, blueness notifications@github.com wrote:

Thanks for the report! What's going on here is that the hwdb files are being installed into

udevhwdbdir = $(udevlibexecdir)/hwdb.d

where

rootprefix=@rootprefix@
udevlibexecdir=@udevlibexecdir@

but, udevadm-hwdb.c is hard coded to look in /etc/udev/hwdb.bin. systemd upstream has already fixed this by adding another AM_CPPFLAGS -DHWDB_BIN=hwdb_bin where the later is still hard set to hwdb_bin=/etc/udev/hwdb.bin, only now in their Makefile.am rather than their c code.

I wonder why he isn't defining it in config.h though, exporting it
using AC_DEFINE_UNQUOTED...

lu

[blueness]

Luca, look at my commit in

https://github.com/gentoo/eudev/commit/e5cc2b881fd1db175b0d80bcccf21ae70857a532

My next step will be to add -DHWDB_BIN=@udevhwdbdir@ and replace the hard coded patch in the code. I think my approach will make it a lot easier to control paths to various components in the future via substitution variables.

[lu-zero]

On Fri, Nov 23, 2012 at 7:59 PM, blueness notifications@github.com wrote:

Luca, look at my commit in

e5cc2b8

From what I can see you should have the make vars already defined by
autoconf no need to redo that.

My next step will be to add -DHWDB_BIN=@udevhwdbdir@ and replace the hard coded patch in the code. I think my approach will make it a lot easier to control paths to various components in the future via substitution variables.

Use AC_DEFINE_UNQUOTED instead.

lu

[blueness]

Yeah I can use AC_DEFINE_UNQUOTED, thanks.

[blueness]

I was not able to use AC_DEFINE* because AC_ARG_WITH does not set up a template in config.h.in like AC_CHECK_* does, so I reverted to using AM_CPPFLAGS as we've been doing up til now --- upstream and ryao did it this way.

@fluxer, /sbin/udevadm hwdb --update should now create /etc/udev/hwdb.bin, or wherever you configure it. It'll always be in the same directory as udev.conf. You may still bump up against issue #18, but at least udevd --daemon --debug should find the file.

Thanks! and please be patient since the pieces are just now starting to come together.

[ryao]

@fluxer I appear to have had some old rules on my system that messed up my testing. 2718feb should fix your issue.

[fluxer]

I gave it a whirl just now, it still doesn't work and still leaves me in initramfs. I changed the initramfs hook to include /etc/udev/hwdb.bin but that doesn't seem to help. And again, I did tried this from the initramfs:

udevd --daemon --debug
udevadm hwdb --update
udevadm trigger
udevadm settle

where the trigger trow a lot of output this time, something that didn't happened before and makes me thing you've made some progress. I will give it another try later on, just let me know.

[ryao]

@fluxer Is there any possibility that Arch Linux uses custom udev rules? It might be worthwhile doing a diff between the rules provided by eudev and the rules provided by Arch Linux's systemd package.

[fluxer]

I've tried that before, look at my first comment:

Since I was using the udev-fork of braindamaged and it worked for me I tried to use the rules from it and copied them over those that eudev provided, that didn't helped either.

[axs-gentoo]

Just to do a bit of a run-down on the install process, based on what's done in gentoo:

1. The usual ./configure [options] && make && make install
2. if the *.hwdb files are not installed/not part of the checkout (and we're going to remove them soon if they haven't been already), get them from https://github.com/gentoo/hwids (or from systemd) (gentoo packages them externally)
3. use 'udevadm --trigger' to create the /etc/udev/hwdb.bin file from the *.hwdb files
4. If using eudev inside an initramfs, rebuild the initramfs (including the hwdb.bin)
5. restart udevd and/or reboot

So if the above matches what you're doing, and it's failing to boot, then if you could provide us with some of that debug output we'd appreciate it -- I'm curious to see what's going on.

[fluxer]

1. check!
2. check! I use hwids package which paths I'm specifying in the configure step - /lib/hwdata/{pci,usb}.ids
3. blueness already pointed me to the right direction about that - udevadm hwdb --update, '--trigger' is not valid :)
4. check!
5. check! After I'm left in initramfs I tried all of the above and it still doesn't makes the nodes.

Here are screenshots of the output of 'udevd --daemon --debug':
http://img834.imageshack.us/img834/1125/udevd1.png
http://img542.imageshack.us/img542/5936/udevd2.png

Compiled today from master, using untouched rules provided by eudev, if the 'udevadm trigger' is important too I will have to make a few more screenshots or if you tell me how to make the nodes manually for my disks (IDE) I can dump it on it and upload it.

[TZ86]

Hello, I have the same problem with default kernel. My logs (catched by bootlogd):

eudev-git 20130121: http://pastebin.com/mk9a3VGu
eudev-git 20130121 (with included /etc/udev/hwdb.bin): http://pastebin.com/qze3qQfh
systemd 196: http://pastebin.com/Sq080jtf

BTW if look into last log, you'll see that systemd-udev misses hwdb file too but works as expected. Maybe the problem is not here?

[TZ86]

@fluxer , try rebuild eudev with --enable-libkmod.

[fluxer]

Nope, didn't worked either. I noticed that

--with-usb-ids-path=/lib/hwdata/usb.ids 
--with-pci-ids-path=/lib/hwdata/pci.ids 
--enable-rule_generator

are not valid configure options, could it be that there is something missing in the configure script?

I'm using the following commands to compile it:

./autogen.sh
./configure --prefix=/usr \
            --sysconfdir=/etc \
            --bindir=/sbin \
            --libexecdir=/lib \
            --with-firmware-path=/usr/lib/firmware/updates:/lib/firmware/updates:/usr/lib/firmware:/lib/firmware \
            --with-usb-ids-path=/lib/hwdata/usb.ids \
            --with-pci-ids-path=/lib/hwdata/pci.ids \
            --enable-rule_generator \
            --disable-introspection \
            --disable-gudev \
            --disable-manpages \
            --enable-libkmod
make

Edit: Two more things

1. I'm no longer running and testing this on Arch Linux, rather something ala LFS with Pacman. I will change the issue title
2. I'm using hwids dating from 20121226. I will update my package and see what happens.

[axs-gentoo]

That is correct. The build system doesn't actually do anything with the *.ids files. You can just copy them directly to /etc/udev/hwdb.d/ or /lib/udev/hwdb.d (or alternatively, make /lib/udev/hwdb.d be a symlink to where they are actually installed), and then run udevadm hwdb --update to generate the "hwdb.bin" file.

[axs-gentoo]

also, --enable-rule-generator should have no effect on this, as all it does is install the helper scripts and the 75-persistent-*-generator.rules as per udev-171 and older. (also, you need to checkout the rule-generator branch to obtain the code for this)

[axs-gentoo]

Just to confirm -- your kernels do have CONFIG_DEVTMPFS and CONFIG_DEVTMPFS_MOUNT enabled, correct?

[fluxer]

Tried to link the hwids files to /lib/udev/hwdb.d and still the same, hwdb.bin updated and included in the initramfs.

And about CONFIG_DEVTMPFS and CONFIG_DEVTMPFS_MOUNT - yes, both are built-in into the kernel. The kernel is 3.4.23 from Zen sources.

[ssuominengentoo]

If you have /dev line in /etc/fstab, make sure it says devtmpfs as filesystem and nothing else -- this would prevent the /dev population if it's wrongly set.

[fluxer]

Pseudo filesystems mounted as follows:

mount -t proc proc /proc -o nosuid,noexec,nodev
mount -t sysfs sys /sys -o nosuid,noexec,nodev
mount -t devtmpfs dev /dev -o mode=0755,nosuid
mount -t tmpfs run /run -o nosuid,nodev,mode=0755

After discussion with Ian he came to the conclusion that this a problem with the combination of kmod and eudev which I'm using. It seems that the necessary modules to access the real filesystems are not loaded.

[TZ86]

this a problem with the combination of kmod and eudev which I'm using

Are you sure that have modules loading support enabled (--enable-modules)? I just want to notice you that it's disabled by default.

[f0ff886f]

I was running into big issues with eudev on Archlinux myself and I just got it working with runit-musl+ignite instead of systemd.

Thanks the TZ86's comment I added --enable-modules and it works now! This is my configure line (prefix is for testing):

./configure --prefix=/usr/local/eudev --enable-modules --enable-libkmod

However, there's a big problem with ./configure --help output:

--disable-modules Disable loadable module support [default=enabled]

That is CLEARLY not the case! The default is disabled, not enabled.

So far so good. I will put up a AUR build so that more Arch users can try this out tomorrow.

[fluxer]

I tried to compile it with '--enable-modules' and nothing changed. Here is what ldd says:

ldd /sbin/udevadm 
linux-vdso.so.1 (0x00007fffbb7ff000)
libudev.so.1 => /lib/libudev.so.1 (0x00007fc581e71000)
libblkid.so.1 => /lib/libblkid.so.1 (0x00007fc581c3d000)
libkmod.so.2 => /lib/libkmod.so.2 (0x00007fc581a28000)
librt.so.1 => /lib/librt.so.1 (0x00007fc581820000)
libpthread.so.0 => /lib/libpthread.so.0 (0x00007fc581604000)
libc.so.6 => /lib/libc.so.6 (0x00007fc581260000)
libm.so.6 => /lib/libm.so.6 (0x00007fc580f66000)
/lib64/ld-linux-x86-64.so.2 (0x00007fc5820ac000)
libuuid.so.1 => /lib/libuuid.so.1 (0x00007fc580d61000)
liblzma.so.5 => /lib/liblzma.so.5 (0x00007fc580b3e000)
libz.so.1 => /lib/libz.so.1 (0x00007fc580928000)

ldd /sbin/udevd   
linux-vdso.so.1 (0x00007fff1adff000)
libudev.so.1 => /lib/libudev.so.1 (0x00007fa1709fc000)
libblkid.so.1 => /lib/libblkid.so.1 (0x00007fa1707c8000)
libkmod.so.2 => /lib/libkmod.so.2 (0x00007fa1705b3000)
libpthread.so.0 => /lib/libpthread.so.0 (0x00007fa170397000)
libc.so.6 => /lib/libc.so.6 (0x00007fa16fff3000)
libm.so.6 => /lib/libm.so.6 (0x00007fa16fcf9000)
librt.so.1 => /lib/librt.so.1 (0x00007fa16faf1000)
/lib64/ld-linux-x86-64.so.2 (0x00007fa170c37000)
libuuid.so.1 => /lib/libuuid.so.1 (0x00007fa16f8ec000)
liblzma.so.5 => /lib/liblzma.so.5 (0x00007fa16f6c9000)
libz.so.1 => /lib/libz.so.1 (0x00007fa16f4b3000)

[TZ86]

I tried to compile it with '--enable-modules' and nothing changed

Strange, it works for me. Do you update initramfs image?

[fluxer]

I did, I'm rebuilding it everytime.

It seems that it works now after adding all of the eudev rules into the initramfs, eudev compiled with '--enable-modules' and '--enable-libkmod'. I've tried that before except that eudev was not compiled with that options and it didn't worked. I guess that those configure options should be set to enabled by default?

[axs-gentoo]

This is probably a documentation issue, more than anything. At this time, --enable-modules is required for -all- module load support (that is, --disable-modules drops all module loading support). --enable-libkmod , on the other hand, enables build-time internal libkmod-based module loading.

We'll make sure to properly document the flags in the future and probably we'll also swap the default so that "modules" is enabled.

Are we at a point now where this issue can be closed?

(oops - didn't mean to actually close this issue, yet)

[fluxer]

I think the issue should stay open until the proper defaults are set in the configure script. Other than that I'm trough it.

[f0ff886f]

Or at least make sure to open a new documentation bug and fix the configure outputs :) But its good for me too.

[blueness]

I agree with fluxer and vostok4. Not ready to be closed, but getting there!

[axs-gentoo]

so --enable-modules was supposed to be on by default, but the AC_ARG_ENABLE command for it in configure was incomplete. Fixed in commit 0407037

(this should bring an eudev default build a lot closer to expected behaviour, if not resolve the issue completely)

[TZ86]

this should bring an eudev default build a lot closer to expected behaviour, if not resolve the issue completely

Yep, now it works perfectly even without --enable-libkmod and --enable-modules. Thanks!

@fluxer, I think the issue may be closed now.

[fluxer]

@TZ86: Good.

@ALL: Thank you for your time.

[axs-gentoo]

as per comments above, the now properly-implemented defaults seem to have sorted this issue. Closing again