Unable to boot from 1Tb external USB drive

[deonlinux]

Relax-and-Recover (ReaR) Issue Template

Fill in the following items before submitting a new issue
(quick response is not guaranteed with free support):

• rear version (/usr/sbin/rear -V):Relax-and-Recover 2.2 / Git

• OS version (cat /etc/rear/os.conf or lsb_release -a):
  LSB Version: :base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch
  Distributor ID: CentOS
  Description: CentOS release 6.6 (Final)
  Release: 6.6
  Codename: Final

• rear configuration files (cat /etc/rear/site.conf or cat /etc/rear/local.conf):

[root@localhost rear-new]# cat etc/rear/local.conf 
# Default is to create Relax-and-Recover rescue media as ISO image
# set OUTPUT to change that
# set BACKUP to activate an automated (backup and) restore of your data
# Possible configuration values can be found in /usr/share/rear/conf/default.conf
#
# This file (local.conf) is intended for manual configuration. For configuration
# through packages and other automated means we recommend creating a new
# file named site.conf next to this file and to leave the local.conf as it is. 
# Our packages will never ship with a site.conf.
#
### write the rescue initramfs to USB and update the USB bootloader
OUTPUT=USB
#OUTPUT_URL=usb:///dev/sdb
#OUTPUT_URL=usb:///dev/disk/by-label/REAR-000

#OUTPUT=ISO
#OUTPUT_URL=file:///var/lib/rear/output/

### create a backup using the internal NETFS method, using 'tar'
BACKUP=NETFS

### write both rescue image and backup to the device labeled REAR-000
BACKUP_URL=usb:///dev/disk/by-label/REAR-000

• Are you using legacy BIOS or UEFI boot? Legacy
• Brief description of the issue:
  Using external Manhattan SATA 3.5 " enclosure, USB3 interface connected to Intel Server USB2 interface. Intel S1200BC server motherboard. Unable to boot from external after creating Recovery & Backup.
  Note: Smaller Sata disk (250Gb tried) works fine. Suspect Grub limitation (large number of cyl)? log has no errors.
  Yet Centos can create bootdisk successfully on 1TB. Creates 2 partitions though (LVM).
• Work-around, if any:Use CDROM rescue media and backup to USB device. inconvenient but usable.
  Included files from /tmp folder during backup creation for 1TB disk:

fdisk_output.txt
parted.txt
partitions.txt
partitions-data.txt
partitions_unsorted.txt

[gozora]

Hi @deonlinux,

Before we go any further; can you please try with configuration similar to following:

# USB testing
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000
BACKUP_URL=usb:///dev/disk/by-label/REAR-000

V.

[gozora]

@deonlinux
Did you format your disk with rear format .... command ?

V.

[deonlinux]

Yes, drive was formatted with rear format cmd
I have also used the suggestions specified previously (USB_DEVICE=/dev/disk/by-label/REAR-000 etc)
Sadly still does not work.

[gozora]

You might have really hit some limits, so I can only guess from now on.
ReaR options I'd experiment with would be following:

USB_DEVICE_PARTED_LABEL=gpt #default is msdos

or maybe try to create smaller partition on your disk using:

USB_DEVICE_FILESYSTEM_PERCENTAGE=20 #default is 100

Full list can be find in default.conf

V.

[deonlinux]

The suggestion of specifying gpt worked!!!
Percentage did not.
Great thanks gozora!
We can close this now I guess

[schlomo]

If this is really a size limit then we should warn the user that it won't work.

Maybe we can default to GPT if this is an EFI system?

[gozora]

@deonlinux, you are always welcome.

Hi @schlomo we are already doing this. Excerpt from default.conf comment:

# When UEFI is used the format workflow will create a GUID partition table in any case.

@jsmeix if I did not mention it earlier, thanks for your great comments!!!

V.

[schlomo]

@gozora I am not sure about that. As far as I can understand 300_format_usb_disk.sh there is no auto detect mechanism for USB_DEVICE_PARTED_LABEL. If EFI is set then it forces EFI mode.

I think that the code actually has two paths:

• EFI (set by command line) creates an EFI System Partition and of course uses gpt
• Otherwise it uses gpt or msdos to create a BIOS boot medium

So to make @deonlinux happy we probably should have two types of auto detect here:

• automatically use EFI mode for EFI systems (currently requires manual setting)
• automatically use gpt, either on modern Linux distros or at least for large media.

[jsmeix]

@gozora
the exact condition is in format/USB/default/300_format_usb_disk.sh

if is_true "$EFI" ; then

so that it depends on when "$EFI" has a 'true' value.
It has a 'true' value at least when the user specifies it
when calling 'rear format' e.g. as in

rear -v format -- --efi --force /dev/sdX

Note the mandatory '--' before the format workflow specific
options ('-v' is the generic rear option).
Without that the format workflow specific options won't work.
For details read the getopt documentation.

[jsmeix]

In general regarding using modern things
more or less automatically, cf.
#1390

[deonlinux]

My apologies - used the wrong disk enclosure (250Gb).
The 1TB drive still does not boot even after the gpt setting :-(
Again sorry for giving false hope.
What else can we try?

[schlomo]

@deonlinux I think you should try to narrow down the issue that you see.

• Can you boot the 1TB device on any other system? (try the mdos, gpt and EFI if you have any)
• Can you create a bootable disk without ReaR and without using LVM, preferably with a similar disk layout at ReaR chooses?
• Does your BIOS offer different ways of accessing the disk (USB hard disk, USB floppy), does any of that work?
• Would the ReaR boot disk boot over another connection like SATA? That will also give you a much higher performance.
• Is there a BIOS update available? Does it help?
• Do you have an USB2 harddisk enclosure to try? Google seems to show a lot of hits on "USB3 boot issues"

Just some ideas of what I would be trying. BTW, restoring 1TB over USB2 will take a while...

If this is urgent and you want somebody to replicate your environment or work with your hands-on then please consider commercial support

[deonlinux]

Hi Schlomo. I did some further tests:

1. Using an enclosure from another manufacturer (Coltech, USB3) I get the same results with the 1TB drive. Tried gpt as well, same result.
2. I am trying to locate a USB2 enclosure but it seems they are quite scarce here now.
3. I tried installing Mint 8.1 on the drive. Mint complains with the following message: "The partition /dev/sda1 assigned to /boot starts at an offset of 3584 bytes from the minimum alignment on the disk which will lead to very poor performance." It suggested I go back and create a new one which will correct this. Unfortunately it does not and whatever I try I cannot move beyond this.
4. I tried with Ubuntu server 16.04. It does not even detect the USB drive as a target to install to. Perhaps it does not allow it, never tried Ubuntu in this fashion.
5. Gparted interestingly says the first 8 Mb is not allocated (sectors 0 to 16383).
6. Placing the 1TB drive in the system using the SATA interface the device boots.

So, unless I locate an USB2 enclosure I cannot conclude that it is USB3 related. Despite all the technical docs on these enclosures saying the will work with up to 3TB drives!
All is not lost. We can swop the drives for 500Gb ones and continue. Most sites data would fit on it.

I will report back if I have some more news.

[schlomo]

If it works with a SATA connection then I would suspect the BIOS in your server. Can you use SATA instead of USB for your production use case? If the server has an eSATA connector then I would even recommend that (although you might have to invest some effort into ensuring that the recovery happens to the internal disk and not the external one) for performance reasons.

The alignment errors are likely the result of different tools (parted, sfdisk ...) used by different distros.

Finally, could it be that you happen to have a 4k drive?

[deonlinux]

Yes indeed the Seagate Baracuda 1TB does have 4k sectors.
I will attempt to reformat the drive manually and see if that fixes it. We can use SATA directly but that would entail sending a techie out to the customer site opening the servers and involve downtime of at least 8 hours. Something I was hoping to avoid.
Today is a public holiday for us so will only test this on Thursday.

[schlomo]

Wow, first time ever that I hear about someone with such a drive. Would be interesting to see if the problem goes away if you find a 1TB drive with 512b sectors.

If it is really the 4k issue then it might indeed be an issue at the BIOS level. Does that server also support UEFI? You might have more success using UEFI boot in such a case.

[deonlinux]

I have finally managed to obtain an enclosure with a USB2 interface. .The issue persists so it is definitely related to the 4k issue. Tested the default and gpt. I am not that familiar with setting up boot structures on drives. My attempt to format the drive with gparted and then just copied the different files across did not work. Attempting to use syslinux to install to the disk fails - it says it needs a FAT partition. Perhaps there are other methods/cmds I can try? Trying to understand the rear scripts is mindblowing! Much more complex that I am used to!

[deonlinux]

One more thing. The bios does support UEFI but does not offer it for the drive. I expect it does not detect the drive as being UEFI capable? The BIOS was updated but the last time Intel released a bios update for this motherboard was in 2011.

[schlomo]

You can try http://blog.schlomo.schapiro.org/2013/11/magic-iso-image-booting-with-gnu-grub-2.html or other similar guides.

For UEFI look at guides like https://askubuntu.com/questions/395879/how-to-create-uefi-only-bootable-usb-live-media.

[gdha]

@deonlinux Are your questions answered?

[jsmeix]

Because there are no further comments
I assume this isssue is sufficiently answered
so that I can close it hereby.