ReaR Restore Error : rpcbind unavailable

[Ronjr21]

Fill in the following items before submitting a new issue
(quick response is not guaranteed with free support):

• ReaR version ("/usr/sbin/rear -V"): Relax-and-Recover 2.4

• OS version ("cat /etc/rear/os.conf" or "lsb_release -a" or "cat /etc/os-release"): Debian 10

• ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):
  OUTPUT=ISO
  BACKUP=NETFS
  BACKUP_URL=nfs://192.168.1.37/test34

• Hardware (PC or PowerNV BareMetal or ARM) or virtual machine (KVM guest or PoverVM LPAR): KVM

• System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device): x86

• Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or ELILO or Petitboot): BIOS and GRUB

• Storage (local disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe): local disk

• Description of the issue (ideally so that others can reproduce it):
  Similar to Rescue fails ERROR: Starting RPC portmapper rpcbind failed #1575. Error upon running rear -vd restore, below is the error log /var/log/rear-prx6.log. Created folder /run/rpcbind but didn't solve the issue.

• Workaround, if any:

• Attachments, as applicable ("rear -D mkrescue/mkbackup/recover" debug log files):

To paste verbatim text like command output or file content,
include it between a leading and a closing line of three backticks like

```
verbatim content
```

[jsmeix]

@Ronjr21
please attach both a rear -D mkrescue/mkkbackup
and a rear -D recover debug log file,
in particular regarding the latter see
"Debugging issues with Relax-and-Recover" at
https://en.opensuse.org/SDB:Disaster_Recovery

See also "Testing current ReaR upstream GitHub master code" at
https://en.opensuse.org/SDB:Disaster_Recovery

[Ronjr21]

Hi, attached debug log file

backup log
rear-prx6.log

restore log
rear-prx6.log

[jsmeix]

The code that fails in the ReaR recovery system is
https://github.com/rear/rear/blob/master/usr/share/rear/verify/NETFS/default/050_start_required_nfs_daemons.sh#L50

# check that RPC portmapper service is available and wait for it as needed
# on some systems portmap/rpcbind can take some time to be accessible
# hence 5 attempts each second to check that RPC portmapper service is available
for attempt in $( seq 5 ) ; do
    # on SLES11 and on openSUSE Leap 42.1 'rpcinfo -p' lists the RPC portmapper as
    #   program vers proto   port  service
    #    100000    2   udp    111  portmapper
    #    100000    4   tcp    111  portmapper
    rpcinfo -p 2>/dev/null | grep -q 'portmapper' && { attempt="ok" ; break ; }
    sleep 1
done
test "ok" = $attempt || Error "RPC portmapper '$portmapper_program' unavailable."

@Ronjr21
therefore I would like to know what on your original system
the following command outputs:

rpcinfo -p

Perhaps the above code that checks if RPC portmapper service is available
does no longer work on Debian 10 and needs to be adapted.
I do not use Debian so I cannot try out or verify things on Debian.

You could also try out if "rear recover" works when you disable the error exit
in that above code that checks if RPC portmapper service is available
by replacing in your
usr/share/rear/verify/NETFS/default/050_start_required_nfs_daemons.sh
the line

test "ok" = $attempt || Error "RPC portmapper '$portmapper_program' unavailable."

by

test "ok" = $attempt || LogPrint "RPC portmapper '$portmapper_program' unavailable."

so that it does no longer error out here.

[Ronjr21]

Changes made to 050_start_required_nfs_daemons.sh as suggested, but there is error on creating disk layout on LVM now, attached debug log.
rear-prx6-lvm.log

[pcahyna]

I think the LVM problem may be related to #2222 (you are using a thin pool).

[jsmeix]

@Ronjr21
thank you for your prompt reply and
@pcahyna
thank you for having a look regarding LVM
(I am basically a LVM noob).

@Ronjr21
regarding LVM thin pool something is already fixed
in current ReaR GitHub master code.
Perhaps those fixes are already sufficient in your case
so that you should try out if things work for you
with current ReaR GitHub master code,
see "Testing current ReaR upstream GitHub master code" at
https://en.opensuse.org/SDB:Disaster_Recovery

[jsmeix]

Curently I have no idea why

rpcinfo -p 2>/dev/null | grep -q 'portmapper'

does not succeed with zero exit code
but the same seems to work on the original system
according to the rpcinfo -p output there
#2250 (comment)

@Ronjr21
could you additionally change in your
usr/share/rear/verify/NETFS/default/050_start_required_nfs_daemons.sh
the line

    rpcinfo -p 2>/dev/null | grep -q 'portmapper' && { attempt="ok" ; break ; }

to

    rpcinfo -p | tee -a $RUNTIME_LOGFILE | grep -q 'portmapper' && { attempt="ok" ; break ; }

and re-run rear -D recover and attach its new debug log file here.

[Ronjr21]

As attached, restore succeeded.
rear-prx6-done.log

[jsmeix]

@Ronjr21
thanks for your prompt https://github.com/rear/rear/files/3717145/rear-prx6-done.log

Therein is now (excerpts)

+ source /usr/share/rear/verify/NETFS/default/050_start_required_nfs_daemons.sh
...
++ for attempt in $( seq 5 )
++ rpcinfo -p
++ tee -a /var/log/rear/rear-prx6.log
++ grep -q portmapper
rpcinfo: can't contact portmapper: RPC: Remote system error - Connection refused
++ sleep 1

which nicely shows why in general 2>/dev/null is unhelpful
cf. #2142 (comment)
because it needlessly suppresses error messages in the log
that would be helpful to see that something fails.

Currently I do not know why in this particular case
rpcinfo fails in the ReaR recovery system with

rpcinfo: can't contact portmapper: RPC: Remote system error - Connection refused

[jsmeix]

Via
af14e15
all kind of '2>/dev/null' (i.e. also '&>/dev/null') were removed
(so that '&>/dev/null' is replaced by '1>/dev/null')
so that now we get error messages in the log,
cf. #1395 therein in particular
#1395 (comment)

[Ronjr21]

Thanks @jsmeix and @pcahyna for helping out, kudos!

[jsmeix]

@Ronjr21
thank you for the feedback that now things work for you.

Could you describe in more detail what you actually changed
to make it work for you?
I would be interested to see if we could improve things in ReaR
so that such issues could be avoided in general in the future.

What did you do regarding the LVM thin pool issue?

Did you perhaps find out why in your particular case
rpcinfo fails in the ReaR recovery system or do you
just ignore that?

[Ronjr21]

@jsmeix LVM thin partition is removed from source server as we do not intend to use it. FYI we are trying ReaR bare metal backup on Proxmox 6. As for rpcinfo I just ignore it. Server is able to obtain backup files from NFS during recovery.