OUTPUT=USB: REAL_USB_DEVICE mounted both below /tmp/rear... and /var/tmp/rear... #2924

frilogg · 2023-02-13T09:23:09Z

Relax-and-Recover (ReaR) Issue Template

Fill in the following items before submitting a new issue
(quick response is not guaranteed with free support):

ReaR version ("/usr/sbin/rear -V"):
Relax-and-Recover 2.4.16 / Git
OS version ("cat /etc/os-release" or "lsb_release -a" or "cat /etc/rear/os.conf"):
NAME="Red Hat Enterprise Linux Server"
VERSION="7.9 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.9"
PRETTY_NAME="Red Hat Enterprise Linux Server 7.9 (Maipo)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.9:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):
OUTPUT=USB
BACKUP=NETFS
BACKUP_URL="usb:///dev/disk/by-label/REAR-000"
BACKUP_PROG_EXCLUDE=('/home/rear' '/var/log/audit')
USB_UEFI_PART_SIZE="1648"
HOSTNAME="sp7-ui"
USB_DEVICE_FILESYSTEM="ext4"
export TMPDIR="/home/rear"
Hardware vendor/product (PC or PowerNV BareMetal or ARM) or VM (KVM guest or PowerVM LPAR):
PC Dell Precision 3450
System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device):
uname -a
Linux solx8ht 3.10.0-1160.66.1.el7.x86_64 #1 SMP Wed Apr 27 20:34:34 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or ELILO or Petitboot):
Grub2
Storage (local disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe):
Storage layout ("lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,LABEL,SIZE,MOUNTPOINT"):

NAME                       KNAME          PKNAME         TRAN TYPE FSTYPE      LABEL      SIZE MOUNTPOINT
/dev/sda                   /dev/sda                      usb  disk                       57.8G
|-/dev/sda1                /dev/sda1      /dev/sda            part vfat        REAR-EFI   1.6G
`-/dev/sda2                /dev/sda2      /dev/sda            part ext4        REAR-000  56.1G
/dev/nvme0n1               /dev/nvme0n1                       disk                      238.5G
|-/dev/nvme0n1p1           /dev/nvme0n1p1 /dev/nvme0n1        part vfat                   500M /boot/efi
|-/dev/nvme0n1p2           /dev/nvme0n1p2 /dev/nvme0n1        part ext4                     1G /boot
`-/dev/nvme0n1p3           /dev/nvme0n1p3 /dev/nvme0n1        part LVM2_member            237G
  |-/dev/mapper/rhel_-root /dev/dm-0      /dev/nvme0n1p3      lvm  ext4                    50G /
  |-/dev/mapper/rhel_-swap /dev/dm-1      /dev/nvme0n1p3      lvm  swap                   7.6G [SWAP]
  `-/dev/mapper/rhel_-home /dev/dm-2      /dev/nvme0n1p3      lvm  ext4                   150G /home

Description of the issue (ideally so that others can reproduce it):
We cant restore backup. After "enter" on relax and recover option it doesnt "fire up" loading files.
Workaround, if any:
Haven't found any
Attachments, as applicable ("rear -D mkrescue/mkbackup/recover" debug log files):

rear-solx8ht.log

The text was updated successfully, but these errors were encountered:

schlomo · 2023-02-13T18:41:02Z

Dear @frilogg, please install ReaR version 2.7, e.g. from http://download.opensuse.org/repositories/Archiving:/Backup:/Rear/RHEL_7/ and try again

In the rescue system you'll have to type rear recover, of course:

frilogg · 2023-02-13T21:07:32Z

Hi @schlomo , thank you for feedback, but .. had hoped to figure it out with this version. I have used rear for many of my customers, and some of the problem is that they "only" want to use the version that is with the official redhat repo. I also noticed that with redhat 9.1 version 2.6 is default also.. So for this cause no good quick tips or you notice anything from the logs provided ?

schlomo · 2023-02-13T22:10:39Z

Well, line 11 of your log says:

/etc/rear/local.conf: line 17: export: =/home/rear': not a valid identifier`

(don't understand why you would export something in a ReaR config anyway as everything is one large shell script)

Besides that you didn't provide anything about the actual problem which is during recovery, right?

frilogg · 2023-02-14T06:38:56Z

Hi and good morning @schlomo , this was corrected but I still had the problem.

I followed your advice and installed Rear 2.7., so thats the version Im on now.
When trying to create a backup now it exit after few seconds. When I do lsblk -p this is not mounted. Please advice.

+ source /usr/share/rear/prep/USB/Linux-i386/350_check_usb_disk.sh
++ test /dev/disk/by-label/REAR-000
+++ readlink -f /dev/disk/by-label/REAR-000
++ REAL_USB_DEVICE=/dev/sda2
++ test -b /dev/sda2
++ res=($( grep -v $BUILD_DIR /proc/mounts | grep "^$REAL_USB_DEVICE" ))
+++ grep -v /var/tmp/rear.upjUYcgOpdZ2LjO /proc/mounts
+++ grep '^/dev/sda2'
++ Error 'USB or disk device '\''/dev/sda2'\'' is already mounted on '\''/tmp/rear.upjUYcgOpdZ2LjO/outputfs'\'''
++ test -s /var/log/rear/rear-solx8ht.log
++ PrintError 'ERROR: USB or disk device '\''/dev/sda2'\'' is already mounted on '\''/tmp/rear.upjUYcgOpdZ2LjO/outputfs'\'''

frilogg · 2023-02-14T06:44:34Z

The reason we want to export and use another tmpdir or else it would not be enough space on device. I ran now export TMPDIR=/home/rear - and now the backup is created. In rear 2.4 we could have this string in the local.conf but it seems that its not working anymore?!

frilogg · 2023-02-14T07:22:35Z

I have now the same problem as #2920

schlomo · 2023-02-14T07:38:00Z

Sorry @frilogg, we need to somewhat change the mode of our conversation here:

please use correct markdown to make it simpler to read
please provide much more context, log files, screenshot etc. and not just a fragment.

With the information you shared I don't even know what to look at.

If you do this for a customer then maybe a commercial support option might be better suited for you where one of us will join you in a remote screen sharing session to look at your situation.

About your TMPDIR question, in 2.7 we changed that to be more standards compliant and the best place to find out about ReaR variabled is always default.conf, where you can find everything about TMPDIR:

# TMPDIR
#
# Relax-and-Recover needs a (temporary) working area where it builds in particular
# the rescue/recovery system ISO image (and perhaps even stores the backup archive).
# The directory name of the working area is created in /usr/sbin/rear by calling
#   mktemp -d -t rear.XXXXXXXXXXXXXXX
# which usually results /tmp/rear.XXXXXXXXXXXXXXX or $TMPDIR/rear.XXXXXXXXXXXXXXX
# the latter when the canonical Linux/Unix environment variable TMPDIR
# is set in the environment where /usr/sbin/rear is called.
# To have a specific working area directory prefix for Relax-and-Recover call
#   export TMPDIR="/prefix/for/rear/working/directory"
# before calling 'rear' (/prefix/for/rear/working/directory must already exist).
# This is useful for example when there is not sufficient free space
# in /var/tmp or $TMPDIR for the ISO image or even the backup archive.
# TMPDIR cannot be set to a default value here unconditionally but only
# if it is not set before calling the program, otherwise /usr/sbin/rear
# would not work in compliance with the Linux/Unix standards regarding TMPDIR
# see https://github.com/rear/rear/issues/968
# The default is /var/tmp instead of the more usual /tmp (the system default),
# because /tmp is not intended for such large amounts of data that ReaR usually
# produces when creating the image (see file-hierarchy(7)). In particular,
# /tmp can be a tmpfs, and thus restricted by the available RAM/swap.
export TMPDIR="${TMPDIR-/var/tmp}"
####

As it is exported there you don't need to (and shouldn't) export it again

gdha · 2023-02-14T09:07:25Z

@frilogg Concerning your customers who want to use the official rear version delivered with their disribution - they have the right to submit an incident at RH and RH support will pick it up and try to help them or provide a fix (which will be merged in our master repo). RH and Suse are activily committing enhancemants and bug fixes in our ReaR master repo.

frilogg · 2023-02-16T10:46:04Z

Thank you for quick feedback, and sorry for late response from my side.
Here you can see how far I got, and also its the attached logs with Rear 2.7.

rear27-solx8ht.log

jsmeix · 2023-02-16T11:15:20Z

Only as a side note FYI
(I know it does not actually help):

Sigh!
Just the usual user experience when booting via UEFI fails.
In general, not limited to ReaR, it always behaves is way:
Not any (error) message, neither from the UEFI firmware
nor from the bootloader - so there is nothing helpful.
I have no idea how to debug such UEFI booting issues.

jsmeix · 2023-02-16T12:10:32Z

@frilogg
in your
#2924 (comment)
the Grub menu text "Boot original system"
does not appear in ReaR 2.7 code
because that was changed via
25f86d4
and #2662

Your
https://github.com/rear/rear/files/10755054/rear27-solx8ht.log
does not contain any

+ source /usr/share/rear/output/USB/...

because it ends with

+ source /usr/share/rear/prep/USB/Linux-i386/350_check_usb_disk.sh

where it still errors out with

++ Error 'USB or disk device '\''/dev/sda2'\'' is already mounted on '\''/tmp/rear.upjUYcgOpdZ2LjO/outputfs'\'''

as in your
#2924 (comment)

jsmeix · 2023-02-16T12:17:57Z

@frilogg
your
https://github.com/rear/rear/files/10755054/rear27-solx8ht.log
contains (excerpts)

mount: /dev/sda2 mounted on /var/tmp/rear.upjUYcgOpdZ2LjO/outputfs.
...
++ Error 'USB or disk device '\''/dev/sda2'\'' is already mounted on '\''/tmp/rear.upjUYcgOpdZ2LjO/outputfs'\'''
...
umount: /var/tmp/rear.upjUYcgOpdZ2LjO/outputfs (/dev/sda2) unmounted

so it seems some 'umount' is somehow somewhere missing
before prep/USB/Linux-i386/350_check_usb_disk.sh
is run or the test in that script is wrong...

jsmeix · 2023-02-16T12:30:11Z

Now I see it:

/dev/sda2 mounted on /var/tmp/rear.upjUYcgOpdZ2LjO/outputfs

versus

/dev/sda2 already mounted on /tmp/rear.upjUYcgOpdZ2LjO/outputfs

i.e. /var/tmp/ versus /tmp/

jsmeix · 2023-02-16T12:39:52Z

@frilogg
in your
usr/share/rear/prep/USB/Linux-i386/350_check_usb_disk.sh
add at its beginning a line

cat /proc/mounts

and run "rear -D mkbackup" again and
attach its whole debug log file here.

frilogg · 2023-02-16T13:15:54Z

The reason we want to export and use another tmpdir or else it would not be enough space on device. I ran now export TMPDIR=/home/rear - and now the backup is created. In rear 2.4 we could have this string in the local.conf but it seems that its not working anymore?!

@jsmeix this error was gone when I exported the TMPDIR, so the backup ran. When I did not have this I could not make a complete backup at all. And it is this backup that I am trying to restore. You still want me to add this?

cat /proc/mounts

jsmeix · 2023-02-16T13:46:06Z

@frilogg
but then your
https://github.com/rear/rear/files/10755054/rear27-solx8ht.log
does not match your screenshots in
#2924 (comment)

To get a clean state without possibly broken
leftovers from what you did with ReaR 2.4
I suggest to completely redo things
with ReaR 2.7 from scratch i.e.:

First redo "rear -D format" for your USB disk
and save its debug log file - perhaps we need it.
Note that since ReaR 2.7 "rear format" behaves different,
see our release notes about "rear format":
https://github.com/rear/rear/blob/rear-2.7/doc/rear-release-notes.txt#L297

Then run "rear -D mkbackup" again and also
save its debug log file - perhaps we need it.
This will create the ReaR recovery system plus
its bootloader things and the backup.tar.gz
anew from scratch on your USB disk.

Then try to boot this new ReaR recovery system
from your USB disk and report here whether or not
that works for you with ReaR 2.7 - i.e. only
whether or not the ReaR recovery system boots and
if you can log in there as 'root' (no password needed).

Provided this works you can try to recreate your system
by calling "rear -D recover" from inside the running
ReaR recovery system.

When you run "rear recover" on some hardware or VM
it will completely overwrite what there already is
on the harddisk(s) or virtual disk(s) on that hardware or VM
so never try out "rear recover" on your original system
where you had run "rear mkbackup" before,
see the initial sections in
https://en.opensuse.org/SDB:Disaster_Recovery

frilogg · 2023-02-16T16:02:54Z

Hi @jsmeix , I have now created the usb again with
rear format -D

But I still get the same error when running
rear -D mkbackup

Relax-and-Recover 2.7 / 2022-07-13
Running rear mkbackup (PID 23959 date 2023-02-16 16:48:13)
Command line options: /sbin/rear -D mkbackup
Using log file: /var/log/rear/rear-solx8ht.log
Using build area: /var/tmp/rear.oVfd2oO0gXvooTT
Running 'init' stage ======================
Running workflow mkbackup on the normal/original system
Running 'prep' stage ======================
Using backup archive '/var/tmp/rear.oVfd2oO0gXvooTT/outputfs/rear/sp7-ui/20230216.1648/backup.tar.gz'
Found EFI system partition /dev/nvme0n1p1 on /boot/efi type vfat
Using UEFI Boot Loader for Linux (USING_UEFI_BOOTLOADER=1)
ERROR: USB or disk device '/dev/sda3' is already mounted on '/tmp/rear.oVfd2oO0gXvooTT/outputfs'
Some latest log messages since the last called script 350_check_usb_disk.sh:

rear-solx8ht.log

frilogg · 2023-02-18T07:34:54Z

Highly appreciate some more feedback @jsmeix or @schlomo as the backup media itself is not creating with Rear 2.7.

frilogg · 2023-02-23T11:53:23Z

Done some more research and with a help from a colleague we made following changes to

/usr/share/rear/prep/USB/Linux-i386/350_check_usb_disk.sh

# Check if the ReaR USB device is not accidentally mounted on other than $BUILD_DIR location: vBUILD_DIR=$BUILD_DIR BUILD_DIR=$(echo $BUILD_DIR | sed -r 's/\/var//g') if res=( $( grep -v $BUILD_DIR /proc/mounts | grep "^$REAL_USB_DEVICE" ) ) ; then Error "USB or disk device '$REAL_USB_DEVICE' is already mounted on '${res[1]}'" fi BUILD_DIR=$vBUILD_DIR

This made the script go pass the error

ERROR: USB or disk device '/dev/sda3' is already mounted on

The backup starting to create but stops and "freeze" when trying to finish.

> 2023-02-23 12:39:36.047271490 tar --warning=no-xdev --sparse --block-number --totals --verbose --no-wildcards-match-slash --one-file-system --ignore-failed-read --anchored --xattrs --xattrs-include=security.capability --xattrs-include=security.selinux --acls --gzip -X /home/rear/rear.whvq6zfr2uszZYN/tmp/backup-exclude.txt -C / -c -f - /home / /boot/efi /boot /var/log/rear/rear-solx8ht.log | dd of=/home/rear/rear.whvq6zfr2uszZYN/outputfs/rear/sp7-ui/20230223.1237/backup.tar.gz bs=1M
++ test 1
++ case "$( basename $BACKUP_PROG )" in
+++ basename tar
++ sleep 1
++ kill -0 30009
+++ awk '{s+=$1} END {print s}'
+++ stat -c %s /home/rear/rear.whvq6zfr2uszZYN/outputfs/rear/sp7-ui/20230223.1237/backup.tar.gz
++ size=16515072
++ ProgressInfo 'Archived 15 MiB [avg 8064 KiB/sec] '
++ echo -en '\e[2K\rArchived 15 MiB [avg 8064 KiB/sec] \e7'
++ sleep 1

Any suggestion to this error when its freezing?

rear-solx8ht.log

jsmeix · 2023-02-23T12:22:53Z

@frilogg
sorry - no idea - and currently no time for it (too many other things).
I will have to test things myself (but I use openSUSE Leap 15.4)
to get a better understanding what actually is going on
before I can make some reasonable comment here.

frilogg · 2023-02-23T14:14:37Z

@schlomo and @jsmeix

I have now got it to work but I had to do the following

Make the changes to the script 350_check_usb_disk.sh , and hopefully this can be something you can use.
Install kernel-devel
Use another TMPDIR location as this got to small for /var/tmp - and created exclude for this folder.
Define use of theese seeings
BOOTLOADER="GRUB2-EFI"
USING_UEFI_BOOTLOADER=1
Then use secure boot at startup. No secure is not working. I have also tried checking bios settings if there are any settings that could have impact here but secure boot is actually off, and also settings regarding uefi paths is off.

Its working for now, but if I get the time I will try doing more research but the workaround in the 350 script worked.

jsmeix · 2023-02-23T14:38:43Z

@frilogg
thank you for your feedback that
for now you could make it work for your case
and what you did to make it work for your case.

I will have a look when time permits.

github-actions · 2023-04-25T02:21:05Z

Stale issue message

jsmeix added support / question waiting for info labels Feb 14, 2023

jsmeix changed the title ~~Cant restore backup. Recovery mode not starting after trying to boot from USB,.~~ OUTPUT=USB on UEFI: Recovery system boot stuck at 'Loading initial ramdisk ...' Feb 16, 2023

jsmeix removed the waiting for info label Feb 16, 2023

jsmeix added the waiting for info label Feb 16, 2023

jsmeix changed the title ~~OUTPUT=USB on UEFI: Recovery system boot stuck at 'Loading initial ramdisk ...'~~ OUTPUT=USB: REAL_USB_DEVICE mounted both below /tmp/rear... and /var/tmp/rear... Feb 16, 2023

github-actions bot added the no-issue-activity label Apr 25, 2023

github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale May 3, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OUTPUT=USB: REAL_USB_DEVICE mounted both below /tmp/rear... and /var/tmp/rear... #2924

OUTPUT=USB: REAL_USB_DEVICE mounted both below /tmp/rear... and /var/tmp/rear... #2924

frilogg commented Feb 13, 2023 •

edited by schlomo

schlomo commented Feb 13, 2023 •

edited

frilogg commented Feb 13, 2023

schlomo commented Feb 13, 2023

frilogg commented Feb 14, 2023 •

edited by schlomo

frilogg commented Feb 14, 2023

frilogg commented Feb 14, 2023

schlomo commented Feb 14, 2023 •

edited

gdha commented Feb 14, 2023

frilogg commented Feb 16, 2023 •

edited

jsmeix commented Feb 16, 2023

jsmeix commented Feb 16, 2023 •

edited

jsmeix commented Feb 16, 2023

jsmeix commented Feb 16, 2023

jsmeix commented Feb 16, 2023

frilogg commented Feb 16, 2023

jsmeix commented Feb 16, 2023 •

edited

frilogg commented Feb 16, 2023 •

edited

frilogg commented Feb 18, 2023 •

edited

frilogg commented Feb 23, 2023

jsmeix commented Feb 23, 2023

frilogg commented Feb 23, 2023

jsmeix commented Feb 23, 2023

github-actions bot commented Apr 25, 2023

OUTPUT=USB: REAL_USB_DEVICE mounted both below /tmp/rear... and /var/tmp/rear... #2924

OUTPUT=USB: REAL_USB_DEVICE mounted both below /tmp/rear... and /var/tmp/rear... #2924

Comments

frilogg commented Feb 13, 2023 • edited by schlomo

Relax-and-Recover (ReaR) Issue Template

schlomo commented Feb 13, 2023 • edited

frilogg commented Feb 13, 2023

schlomo commented Feb 13, 2023

frilogg commented Feb 14, 2023 • edited by schlomo

frilogg commented Feb 14, 2023

frilogg commented Feb 14, 2023

schlomo commented Feb 14, 2023 • edited

gdha commented Feb 14, 2023

frilogg commented Feb 16, 2023 • edited

jsmeix commented Feb 16, 2023

jsmeix commented Feb 16, 2023 • edited

jsmeix commented Feb 16, 2023

jsmeix commented Feb 16, 2023

jsmeix commented Feb 16, 2023

frilogg commented Feb 16, 2023

jsmeix commented Feb 16, 2023 • edited

frilogg commented Feb 16, 2023 • edited

frilogg commented Feb 18, 2023 • edited

frilogg commented Feb 23, 2023

jsmeix commented Feb 23, 2023

frilogg commented Feb 23, 2023

jsmeix commented Feb 23, 2023

github-actions bot commented Apr 25, 2023

frilogg commented Feb 13, 2023 •

edited by schlomo

schlomo commented Feb 13, 2023 •

edited

frilogg commented Feb 14, 2023 •

edited by schlomo

schlomo commented Feb 14, 2023 •

edited

frilogg commented Feb 16, 2023 •

edited

jsmeix commented Feb 16, 2023 •

edited

jsmeix commented Feb 16, 2023 •

edited

frilogg commented Feb 16, 2023 •

edited

frilogg commented Feb 18, 2023 •

edited