Rear cann't mount /tmp/rear.xxxxxxxxx/outputfs when rescue media created in CentOS x64 #202

altring · 2013-02-22T12:11:39Z

Here is the situation.
When I run:
$ rear format /dev/sdc

in Centos 6.3 i386 kernel 2.6.32-279.22.1.el6.i386.
Then I boot in Centos 6.3 x86_64 kernel 2.6.32-279.22.1.el6.x86_64 and
rear mkbackup

Backup and restore of Centos 6.3 x86_64 has done without errors.

But when I run

$ rear format /dev/sdc
$ rear mkbackup

both in Centos 6.3 x86_64 I have the following lines in /var/log/rear/rear-hostname.log after executing "rear recover" command.

$ cat /var/log/rear/rear-hostname.log
2013-02-22 08:56:42 Relax-and-Recover 1.14 / Git
2013-02-22 08:56:42 Command line options: /bin/rear recover
2013-02-22 08:56:42 Using log file: /var/log/rear/rear-localhost.log
2013-02-22 08:56:42 Including /etc/rear/os.conf
2013-02-22 08:56:42 Including conf/Linux-i386.conf
2013-02-22 08:56:42 Including conf/GNU/Linux.conf
2013-02-22 08:56:42 Including /etc/rear/local.conf
2013-02-22 08:56:42 Including /etc/rear/rescue.conf
2013-02-22 08:56:42 Using build area '/tmp/rear.1aS3iekWHblvBCn'
mkdir: created directory `/tmp/rear.1aS3iekWHblvBCn/rootfs'
mkdir: created directory `/tmp/rear.1aS3iekWHblvBCn/tmp'
2013-02-22 08:56:42 Running recover workflow
2013-02-22 08:56:42 Running 'setup' stage
2013-02-22 08:56:42 Including setup/default/01_pre_recovery_script.sh
2013-02-22 08:56:42 Finished running 'setup' stage in 0 seconds
2013-02-22 08:56:42 Running 'verify' stage
2013-02-22 08:56:42 Including verify/default/02_cciss_scsi_engage.sh
2013-02-22 08:56:42 Including verify/default/02_translate_url.sh
2013-02-22 08:56:42 Including verify/default/03_translate_tape.sh
2013-02-22 08:56:42 Including verify/NETFS/default/05_check_NETFS_requirements.sh
2013-02-22 08:56:42 Skipping ping test
2013-02-22 08:56:42 Including verify/GNU/Linux/05_sane_recovery_check.sh
2013-02-22 08:56:42 Including verify/NETFS/default/07_set_backup_archive.sh
2013-02-22 08:56:42 Including verify/NETFS/default/08_start_required_daemons.sh
2013-02-22 08:56:42 Including verify/NETFS/default/09_set_readonly_options.sh
2013-02-22 08:56:42 Including verify/NETFS/default/10_mount_NETFS_path.sh
mkdir: created directory `/tmp/rear.1aS3iekWHblvBCn/outputfs'
2013-02-22 08:56:42 Mounting with 'mount -v -o ro /dev/disk/by-label/REAR-000 /tmp/rear.1aS3iekWHblvBCn/outputfs'
mount: unknown filesystem type 'ext3'
mount: you didn't specify a filesystem type for /dev/sdc1
       I will try type ext3
2013-02-22 08:56:43 ERROR: Mount command 'mount -v -o ro /dev/disk/by-label/REAR-000 /tmp/rear.1aS3iekWHblvBCn/outputfs' failed.
=== Stack trace ===
Trace 0: /bin/rear:248 main
Trace 1: /usr/share/rear/lib/recover-workflow.sh:27 WORKFLOW_recover
Trace 2: /usr/share/rear/lib/framework-functions.sh:79 SourceStage
Trace 3: /usr/share/rear/lib/framework-functions.sh:40 Source
Trace 4: /usr/share/rear/verify/NETFS/default/10_mount_NETFS_path.sh:11 source
Trace 5: /usr/share/rear/lib/global-functions.sh:124 mount_url
Trace 6: /usr/share/rear/lib/_input-output-functions.sh:131 StopIfError
Message: Mount command 'mount -v -o ro /dev/disk/by-label/REAR-000 /tmp/rear.1aS3iekWHblvBCn/outputfs' failed.
===================
2013-02-22 08:56:43 Running exit tasks.
rmdir: removing directory, `/tmp/rear.1aS3iekWHblvBCn/outputfs'
2013-02-22 08:56:43 Finished in 1 seconds
2013-02-22 08:56:43 Removing build area /tmp/rear.1aS3iekWHblvBCn
rmdir: removing directory, `/tmp/rear.1aS3iekWHblvBCn'
2013-02-22 08:56:43 End of program reached

$ cat  /etc/rear/local.conf
OUTPUT=USB
BACKUP=NETFS
BACKUP_URL=usb:///dev/disk/by-label/REAR-000

The text was updated successfully, but these errors were encountered:

gdha · 2013-02-25T07:16:53Z

Are you sure that rear format /dev/sdc worked correctly? The script /usr/share/rear/format/USB/default/30_format_usb_disk.sh is responsible for the formatting of the USB stick with an ext3 filesystem. Also, check if the label on the USB stick was created correctly:

$ e2label /dev/sdc1

The output should be REAR-000

altring · 2013-02-25T13:15:02Z

everything is ok.
Here is how output of mount command looks like:

[root@localhost ~]# mount
...
...
/dev/sdc1 on /media/REAR-000 type ext3 (rw,nosuid,nodev,uhelper=udisks)

I've managed to create media and was able to boot from it.
But

rear recover

fails

gdha · 2013-02-25T14:20:54Z

What do you see in the log-file of rear as error?

altring · 2013-02-26T07:19:10Z

Here is what I have:

...
...
2013-02-22 08:56:42 Including verify/NETFS/default/10_mount_NETFS_path.sh
mkdir: created directory `/tmp/rear.1aS3iekWHblvBCn/outputfs'
2013-02-22 08:56:42 Mounting with 'mount -v -o ro /dev/disk/by-label/REAR-000 /tmp/rear.1aS3iekWHblvBCn/outputfs'
mount: unknown filesystem type 'ext3'
mount: you didn't specify a filesystem type for /dev/sdc1
       I will try type ext3
2013-02-22 08:56:43 ERROR: Mount command 'mount -v -o ro /dev/disk/by-label/REAR-000 /tmp/rear.1aS3iekWHblvBCn/outputfs' failed.
=== Stack trace ===
Trace 0: /bin/rear:248 main
Trace 1: /usr/share/rear/lib/recover-workflow.sh:27 WORKFLOW_recover
Trace 2: /usr/share/rear/lib/framework-functions.sh:79 SourceStage
Trace 3: /usr/share/rear/lib/framework-functions.sh:40 Source
Trace 4: /usr/share/rear/verify/NETFS/default/10_mount_NETFS_path.sh:11 source
Trace 5: /usr/share/rear/lib/global-functions.sh:124 mount_url
Trace 6: /usr/share/rear/lib/_input-output-functions.sh:131 StopIfError
Message: Mount command 'mount -v -o ro /dev/disk/by-label/REAR-000 /tmp/rear.1aS3iekWHblvBCn/outputfs' failed.
===================
2013-02-22 08:56:43 Running exit tasks.
rmdir: removing directory, `/tmp/rear.1aS3iekWHblvBCn/outputfs'
2013-02-22 08:56:43 Finished in 1 seconds
2013-02-22 08:56:43 Removing build area /tmp/rear.1aS3iekWHblvBCn
rmdir: removing directory, `/tmp/rear.1aS3iekWHblvBCn'
2013-02-22 08:56:43 End of program reached

gdha · 2013-02-26T08:30:17Z

can you do the following? # grep EXT3 /boot/config-*

altring · 2013-02-26T11:03:11Z

[root@localhost ~]# grep EXT3 /boot/config-*
/boot/config-2.6.32-279.22.1.el6.x86_64:CONFIG_EXT3_FS=m
/boot/config-2.6.32-279.22.1.el6.x86_64:CONFIG_EXT3_DEFAULTS_TO_ORDERED=y
/boot/config-2.6.32-279.22.1.el6.x86_64:CONFIG_EXT3_FS_XATTR=y
/boot/config-2.6.32-279.22.1.el6.x86_64:CONFIG_EXT3_FS_POSIX_ACL=y
/boot/config-2.6.32-279.22.1.el6.x86_64:CONFIG_EXT3_FS_SECURITY=y
/boot/config-2.6.32-279.el6.x86_64:CONFIG_EXT3_FS=m
/boot/config-2.6.32-279.el6.x86_64:CONFIG_EXT3_DEFAULTS_TO_ORDERED=y
/boot/config-2.6.32-279.el6.x86_64:CONFIG_EXT3_FS_XATTR=y
/boot/config-2.6.32-279.el6.x86_64:CONFIG_EXT3_FS_POSIX_ACL=y
/boot/config-2.6.32-279.el6.x86_64:CONFIG_EXT3_FS_SECURITY=y

gdha · 2013-02-26T12:18:37Z

Please check the content of cat /etc/modules. Please paste it in here.
Try modprobe ext3 before running rear -v recover

altring · 2013-02-26T12:23:04Z

In rescue system

RESCUE localhost:~ # cat /etc/modules
r8169
r8169
r8169
usb_storage
RESCUE localhost:~ # modprobe ext3
FATAL: Module ext3 not found.

gdha · 2013-02-26T15:52:10Z

Ok, we found the problem, right? Module ext3 was not copied to the image.

altring · 2013-02-27T07:06:05Z

Yes, we did it :)

gdha · 2013-02-27T11:30:02Z

Can you proof this via the rear.log? Try grep ext /var/log/rear/rear-$(hostname).log
I find this rather strange. Was there a mix up with another kernel tree perhaps?
Try a find /lib/modules -name "ext*"

altring · 2013-02-27T12:23:16Z

grep ext /var/log/rear/rear-$(hostname).log gives huuuuuge output.
grep ext3 /var/log/rear/rear-$(hostname).log gives:

2013-02-26 12:08:39 Binaries being copied: /bin/awk /bin/basename /bin/bash /bin/cat /bin/chmod /bin/chown /bin/cp /bin/cpio /bin/cut /bin/date /bin/dd /bin/df /bin/dmesg /bin/dumpkeys /bin/egrep /bin/fgrep /bin/find /bin/grep /bin/gzip /bin/hostname /bin/ipcalc /bin/kbd_mode /bin/ln /bin/loadkeys /bin/ls /bin/mkdir /bin/mknod /bin/mktemp /bin/mount /bin/mv /bin/netstat /bin/ping /bin/ps /bin/readlink /bin/rm /bin/rmdir /bin/sed /bin/sleep /bin/sort /bin/stty /bin/sync /bin/tar /bin/touch /bin/true /bin/umount /bin/uname /bin/usleep /bin/vi /lib/udev/ata_id /lib/udev/cdrom_id /lib/udev/edd_id /lib/udev/path_id /lib/udev/usb_id /root/rear/usr/sbin/rear /sbin/agetty /sbin/arping /sbin/blkid /sbin/cfdisk /sbin/dhclient /sbin/ethtool /sbin/fdisk /sbin/fsck /sbin/fsck.ext2 /sbin/fsck.ext3 /sbin/fsck.ext4 /sbin/fsck.ext4dev /sbin/grub /sbin/grub-install /sbin/halt /sbin/ifconfig /sbin/ifenslave /sbin/init /sbin/initctl /sbin/insmod /sbin/ip /sbin/killall5 /sbin/lsmod /sbin/mdadm /sbin/mingetty /sbin/mkfs /sbin/mkfs.ext2 /sbin/mkfs.ext3 /sbin/mkfs.ext4 /sbin/mkfs.ext4dev /sbin/mkswap /sbin/modprobe /sbin/nameif /sbin/parted /sbin/partprobe /sbin/pidof /sbin/poweroff /sbin/reboot /sbin/rmmod /sbin/route /sbin/rsyslogd /sbin/scsi_id /sbin/sfdisk /sbin/shutdown /sbin/swapoff /sbin/swapon /sbin/telinit /sbin/tune2fs /sbin/udevadm /sbin/udevd /usr/bin/attr /usr/bin/bzip2 /usr/bin/chacl /usr/bin/diff /usr/bin/dirname /usr/bin/du /usr/bin/expand /usr/bin/expr /usr/bin/file /usr/bin/free /usr/bin/getent /usr/bin/getfacl /usr/bin/getfattr /usr/bin/getopt /usr/bin/head /usr/bin/id /usr/bin/join /usr/bin/killall /usr/bin/ldd /usr/bin/less /usr/bin/logger /usr/bin/md5sum /usr/bin/mkfifo /usr/bin/openvt /usr/bin/rsync /usr/bin/scp /usr/bin/seq /usr/bin/setfacl /usr/bin/setfattr /usr/bin/sftp /usr/bin/ssh /usr/bin/stat /usr/bin/strings /usr/bin/tac /usr/bin/tail /usr/bin/tee /usr/bin/top /usr/bin/tr /usr/bin/tty /usr/bin/uniq /usr/bin/wc /usr/bin/which /usr/bin/xargs /usr/bin/xz /usr/libexec/openssh/sftp-server /usr/sbin/chroot /usr/sbin/sshd
`/sbin/fsck.ext3' -> `/tmp/rear.BYBzaZFHTUzPCcs/rootfs/bin/fsck.ext3'
`/sbin/mkfs.ext3' -> `/tmp/rear.BYBzaZFHTUzPCcs/rootfs/bin/mkfs.ext3'
       I will try type ext3
/dev/sdc1 on /tmp/rear.BYBzaZFHTUzPCcs/outputfs type ext3 (rw,noatime)
       I will try type ext3
/dev/sdc1 on /tmp/rear.BYBzaZFHTUzPCcs/outputfs type ext3 (rw,noatime)
block 287118: /usr/include/asm/sigcontext32.h
block 764385: /usr/share/man/man8/mkfs.ext3.8.gz
block 764669: /usr/share/man/man8/fsck.ext3.8.gz
block 1504679: /lib/modules/2.6.32-279.22.1.el6.x86_64/kernel/fs/ext3/
block 1504680: /lib/modules/2.6.32-279.22.1.el6.x86_64/kernel/fs/ext3/ext3.ko
block 1692696: /lib/modules/2.6.32-279.el6.x86_64/kernel/fs/ext3/
block 1692697: /lib/modules/2.6.32-279.el6.x86_64/kernel/fs/ext3/ext3.ko
block 2230653: /sbin/fsck.ext3
block 2230654: /sbin/mkfs.ext3

Mix with another kernel possible as there is an update to 2.6.32-279.22.1.el6.x86_64 kernel!
Rescue media was built under latest kernel!
Here is

find /lib/modules -name "ext*"
/lib/modules/2.6.32-279.22.1.el6.x86_64/kernel/fs/ext4
/lib/modules/2.6.32-279.22.1.el6.x86_64/kernel/fs/ext4/ext4.ko
/lib/modules/2.6.32-279.22.1.el6.x86_64/kernel/fs/ext3
/lib/modules/2.6.32-279.22.1.el6.x86_64/kernel/fs/ext3/ext3.ko
/lib/modules/2.6.32-279.22.1.el6.x86_64/kernel/fs/ext2
/lib/modules/2.6.32-279.22.1.el6.x86_64/kernel/fs/ext2/ext2.ko
/lib/modules/2.6.32-279.22.1.el6.x86_64/extra
/lib/modules/2.6.32-279.el6.x86_64/kernel/fs/ext4
/lib/modules/2.6.32-279.el6.x86_64/kernel/fs/ext4/ext4.ko
/lib/modules/2.6.32-279.el6.x86_64/kernel/fs/ext3
/lib/modules/2.6.32-279.el6.x86_64/kernel/fs/ext3/ext3.ko
/lib/modules/2.6.32-279.el6.x86_64/kernel/fs/ext2
/lib/modules/2.6.32-279.el6.x86_64/kernel/fs/ext2/ext2.ko
/lib/modules/2.6.32-279.el6.x86_64/extra

for issue #202

gdha · 2013-03-04T15:40:23Z

@altring could you give it another try with latest github repo?

altring · 2013-03-28T14:01:45Z

It works!
Now we have in rescue system:

RESCUE localhost:~ # cat /etc/modules
e100
ext3
usb_storage

And restore works.
I think problem resolved.

gdha · 2013-03-31T15:13:40Z

Thanks for testing it out...and glad it is resolved.

TimDowker · 2013-04-10T14:21:54Z

Trying to do a recovery to dis-similar hardware and keep getting "Mount command failed" error. I have confirmed the ext3 module is loaded and I've tried re-running the backup just to be sure. I'm running the latest GIT - synced yesterday, with an external USB drive. Any thoughts?

Here's the output of my backup command - the format of the USB device also gave no errors.

sudo usr/sbin/rear -v mkbackup
Relax-and-Recover 1.14 / Git
Using log file: /home/tdowker/rear/var/log/rear/rear-MiddleSrv2.log
Creating disk layout
Creating root filesystem layout
WARNING: Could not determine network driver for 'bond0'. Please make
WARNING: sure that it loads automatically (e.g. via udev) or add
WARNING: it to MODULES_LOAD in /home/tdowker/rear/etc/rear/{local,site}.conf!
TIP: To login as root via ssh you need to set up /root/.ssh/authorized_keys
Copying files and directories
/home/tdowker/rear/etc/rear/local.conf' ->/tmp/rear.QdaY97YhodaPrae/rootfs/etc/rear/local.conf'
Copying binaries and libraries
Copying kernel modules
Creating initramfs
Writing MBR to /dev/sdc
Copying resulting files to usb location
Encrypting disabled
Creating tar archive '/tmp/rear.QdaY97YhodaPrae/outputfs/rear/MiddleSrv2/20130409.1009/backup.tar.gz'
Archived 150873 MiB [avg 16548 KiB/sec]OK
Archived 150873 MiB in 9337 seconds [avg 16546 KiB/sec]

ghost assigned gdha Feb 25, 2013

gdha added a commit that referenced this issue Feb 27, 2013

adding script to push module of USB device to MODULES/MODULES_LOAD

7d4769d

for issue #202

gdha closed this as completed Mar 31, 2013

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rear cann't mount /tmp/rear.xxxxxxxxx/outputfs when rescue media created in CentOS x64 #202

Rear cann't mount /tmp/rear.xxxxxxxxx/outputfs when rescue media created in CentOS x64 #202

altring commented Feb 22, 2013

gdha commented Feb 25, 2013

altring commented Feb 25, 2013

gdha commented Feb 25, 2013

altring commented Feb 26, 2013

gdha commented Feb 26, 2013

altring commented Feb 26, 2013

gdha commented Feb 26, 2013

altring commented Feb 26, 2013

gdha commented Feb 26, 2013

altring commented Feb 27, 2013

gdha commented Feb 27, 2013

altring commented Feb 27, 2013

gdha commented Mar 4, 2013

altring commented Mar 28, 2013

gdha commented Mar 31, 2013

TimDowker commented Apr 10, 2013

Rear cann't mount /tmp/rear.xxxxxxxxx/outputfs when rescue media created in CentOS x64 #202

Rear cann't mount /tmp/rear.xxxxxxxxx/outputfs when rescue media created in CentOS x64 #202

Comments

altring commented Feb 22, 2013

gdha commented Feb 25, 2013

altring commented Feb 25, 2013

gdha commented Feb 25, 2013

altring commented Feb 26, 2013

gdha commented Feb 26, 2013

altring commented Feb 26, 2013

gdha commented Feb 26, 2013

altring commented Feb 26, 2013

gdha commented Feb 26, 2013

altring commented Feb 27, 2013

gdha commented Feb 27, 2013

altring commented Feb 27, 2013

gdha commented Mar 4, 2013

altring commented Mar 28, 2013

gdha commented Mar 31, 2013

TimDowker commented Apr 10, 2013