New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
use better compression for initrd which must be less than 32MB when booting on ppc64 via yaboot #1142
Comments
@abbahag Regarding switching from cpio/gzip to cpio/xz-lzma Next Monday I will have a closer look. For now: |
Not for the right now upcoming ReaR 2.0 but after that. |
Could it be specified via a VARIABLE OPTION in /etc/local.conf ? (INITRD_CPIO_COMPRESSION) |
I think your suggestion is the more elegant and more flexible. It should
be implemented via the local.conf.
|
@schabrolles In general: I like it very much when "all and everything" The less is hardcoded in the actual code and When "all and everything" can be adjusted This means that usually the user should not need For "all and everything" there should be a reasonable Furthermore for "all and everything" there should be a reasonable In this particular case we may even be able # 900_create_initramfs.sh # # create initramfs for Relax-and-Recover # # This file is part of Relax-and-Recover, licensed under the GNU General # Public License. Refer to the included COPYING for full text of license. LogPrint "Creating initramfs" pushd "$ROOTFS_DIR" >/dev/null # First try to create initrd.xz with the newer xz-lzma compression, # see https://github.com/rear/rear/issues/1142 if find . ! -name "*~" | cpio -H newc --create --quiet | xz --format=lzma --compress --stdout > "$TMP_DIR/initrd.xz" ; then LogPrint "Created initrd.xz with xz-lzma compression ($( stat -c%s $TMP_DIR/initrd.xz ) bytes)" else # If it fails to create initrd.xz with xz-lzma compression # fall back to the traditional way and create initrd.cgz with gzip compression, # cf. "Dirty hacks welcome" at https://github.com/rear/rear/wiki/Coding-Style if find . ! -name "*~" | cpio -H newc --create --quiet | gzip > "$TMP_DIR/initrd.cgz" ; then LogPrint "Created initrd.cgz with gzip compression ($( stat -c%s $TMP_DIR/initrd.cgz ) bytes)" else Error "Failed to create initramfs" fi fi popd >/dev/null As a consequence all scripts where initrd.cgz is useed # In 900_create_initramfs.sh the Relax-and-Recover initrd is created # either as initrd.xz with the newer xz-lzma compression # or as initrd.cgz with gzip compression # see https://github.com/rear/rear/issues/1142 test -s "$TMP_DIR/initrd.cgz" && initrd_filename="initrd.cgz" test -s "$TMP_DIR/initrd.xz" && initrd_filename="initrd.xz" test "$initrd_filename" || Error "Neither initrd.cgz nor initrd.xz found" and then using $initrd_filename everywhere. I will do a pull request to make it more clear |
I fear my "automatically do the right thing" attempt In other words: |
The solution with xz-lzma is necessary for me to boot under IBM Power8
SLES 11 with a HANA installation.
Unfortunately yaboot only has a maximum size of 32 MB. I have the initrd
with cpio / gz not get below the limit of 32 MB.
I found the idea good and flexible with an option in the local.conf to
determine whether one would want a different compression than the
standard.
|
I am currently testing I noticed a big disadvantage of xz-lzma: 2017-01-26 14:34:42.321885598 Creating recovery/rescue system initramfs/initrd ... 2017-01-26 14:36:37.937206502 Created initrd.xz with xz-lzma compression (91401572 bytes) I.e. 116 seconds (almost two minutes) In contrast with 'gzip': 2017-01-26 14:51:32.230285525 Creating recovery/rescue system initramfs/initrd ... 2017-01-26 14:51:48.339397663 Created initrd.cgz with gzip compression (149199959 bytes) I.e. 16 seconds (more than 7 times faster as xz-lzma) to create This matters because "rear mkbackup" is meant to run Accordingly I will change my current implementation This way it is even safe against regerssions because |
@abbahag |
In #1182 # REAR_INITRD_COMPRESSION # # What compression to use when creating the initramfs/initrd for Relax-and-Recover. # By default and also as fallback an initrd.cgz with gzip default compression is created. # The default is known to work well in practice. # The ReaR initrd is loaded by possibly various bootloaders (normally syslinux). # Usually any bootloader should be able to load any initrd file but then # the initrd must be usable by the kernel which means a specially compressed initrd # may not always work, in particular not with older kernels. # With REAR_INITRD_COMPRESSION="fast" # an initrd.cgz with gzip --fast compression is created (fast speed but less compression). # With REAR_INITRD_COMPRESSION="best" # an initrd.cgz with gzip --best compression is created (best compression but slow speed). # With REAR_INITRD_COMPRESSION="lzma" # an initrd.xz with xz using the lzma compression is created # (see https://github.com/rear/rear/issues/1142). REAR_INITRD_COMPRESSION="" Results on my SLES12 test system "rear mkrescue" with the default: Created initrd.cgz with gzip default compression (149198234 bytes) about 142 MiB. "rear mkrescue" with REAR_INITRD_COMPRESSION="fast" Created initrd.cgz with gzip fast compression (159204253 bytes) about 152 MiB. "rear mkrescue" with REAR_INITRD_COMPRESSION="best" Created initrd.cgz with gzip best compression (148502717 bytes) about 142 MiB. "rear mkrescue" with REAR_INITRD_COMPRESSION="lzma" Created initrd.xz with xz lzma compression (91405354 bytes) about 87 MiB. I found an advantage of using a high compression: This matters also because system recovery time is very ReaR is known to be fast so that this way we could |
…th_lzma_but_use_gzip_as_fallback_issue1142 New REAR_INITRD_COMPRESSION config variable to specify the compression of the recovery system initrd, see #1142
With #1182 @abbahag REAR_INITRD_COMPRESSION="lzma" and provide feedback whether or not How to test the currently newest rear GitHub master code Basically "git clone" it into a directory and then # git clone https://github.com/rear/rear.git # cd rear # vi etc/rear/local.conf # usr/sbin/rear -d -D mkbackup (note the relative paths "etc/rear/" and "usr/sbin/"). |
@jsmeix Option is working fine in my environment. |
@abbahag |
I have a general question about ReaR's initrd size: On my x86_64 system (using BIOS) the ReaR initrd.xz In contrast I think independent of the architecture the ReaR initrd should Therefore I wonder why the ReaR initrd.xz on ppc64 Can someone explain to me what the reason FYI: e205:/tmp/rear.BBbYtTeC5sJIVFt/rootfs # du -shc * 37M bin 7.5M boot 132K dev 9.2M etc 0 init 196M lib 9.5M lib64 20K mnt 8.0K proc 5.2M root 28K run 0 sbin 8.0K selinux 8.0K sys 1.4M tmp 72M usr 136K var 338M total e205:/tmp/rear.BBbYtTeC5sJIVFt/rootfs # du -shc lib/* 143M lib/firmware 54M lib/modules 0 lib/udev 196M total It seems more than the half of ReaR's initrd Perhaps on ppc64 there is much less firmware? |
Hi @jsmeix,
|
@schabrolles I assume on ppc64le in BareMetal Mode (PowerNV) |
Sharing my results using Arch Linux 4.9.11-1 x86_64: Now it's time to clean up |
@ProBackup-nl |
For the bare minimum Arch Linux system installers, cleaning up After removing /usr/lib/firmware contents except for /usb/lib/firmware/rtl_nic (
After removing a load of /usr/lib/modules contents, like bluetooth, pata, pcmcia, infiniband, parport, Wi-Fi, snd, wimax, etcetera, 83M /usr/lib/modules shrank to 48M, resulting in:
PS The beauty of Arch Linux is that you can prevent /usr/lib/firmware re-creation upon firmware package upgrades by adding a line to
|
@jsmeix Is it really necessary to create an initrd file when the kernel of Arch Linux runs with a 6M/11M initramfs (instead of 35M/150M generated initrd)?
|
@ProBackup-nl |
@ProBackup-nl |
@schabrolles I assume on ppc64le in BareMetal Mode (PowerNV) it does not work with yaboot. see in my new enhancement idea in particular |
Yes, you are right.
In BareMetal mode (PowerNV), only recent Linux distro are supported (rhel7.2+ and ubuntu16.04)
But we can have older distro in PowerVM or PowerKVM (sles11, rhel6) => They still uses yaboot/lilo
|
@jsmeix After the modules and firmware cleanup in my x86-64 UEFI booted Arch Linux, these (manual) folder traverse size sorted listings of ROOTFS are potential suspects for further size reduction. Additionally, for the cases where you don't need networking to recover (as is the goal of my USB setup), removing all network related stuff (sshd, ssh, ip, rsync, kernel/net, kernel/drivers/net) could save a considerable amount of space and time to create the initrd.
|
Relax-and-Recover (rear) Issue Template
Please fill in the following items before submitting a new issue (quick response is not guaranteed with free support):
cat /etc/rear/local.conf
Brief description of the issue
If you want to boot SLES11 SP4 ppc64 to an lpar on IBM 9119-MME (P8 880) with firmware> = 840 via a TFTP server (NIM), the initrd must be less than or equal to 32 MB (with yaboot). Unfortunately, a standard HANA on Power (with SLES HA ) installation is larger than 32 MB.
Work-around, if any
The solution to the problem of switching from cpio/gzip to cpio/xz-lzma. This reduces the generated initrd by approximately 13 MB. With gzip, the initrd is 34.1 MB with xz and lzma 21.4 MB. For this purpose the file 900_create_initramfs.sh in /usr/share/rear/pack and the File 800_copy_to_tftp.sh in /usr/share/rear/output/ must be adapted.
900_create_initramfs.sh:
800_copy_to_tftp.sh:
Change .cgz with .xz
The text was updated successfully, but these errors were encountered: