Still output in /var/lib/rear when using BACKUP=TSM #634

Closed
eholtz opened this Issue Aug 12, 2015 · 14 comments

Comments

Projects
None yet
3 participants
@eholtz

eholtz commented Aug 12, 2015

Hi,

I updated to rear-1.17.1 and read about the OUTPUT_URL=null option. I thought that would work out great for me and activated it, but I still see two disk images in my environment.

I have default config files for local.conf and os.conf and:

[@machine ~]# find / -iname "*.iso"
[@machine ~]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.1 (Maipo)
[@machine ~]# rpm -qa | grep rear
rear-1.17.1-1.el7.noarch
[@machine ~]# cat /etc/rear/site.conf
# configuration file for relax and recover
# don't upload to tsm directly as this is scheduled somewhere else
BACKUP=TSM
TSM_RESULT_SAVE=n
OUTPUT_URL=null
[@machine ~]# /usr/sbin/rear mkrescue
[@machine ~]# find / -iname "*.iso"
/opt/tivoli/tsm/rear/rear-machine.iso
/var/lib/rear/output/rear-machine.iso
[@machine ~]#

@gdha gdha added the bug label Aug 12, 2015

@gdha gdha self-assigned this Aug 12, 2015

@gdha

This comment has been minimized.

Show comment
Hide comment
@gdha

gdha Aug 12, 2015

Member

could you run rear -vD mkrescue to see where it goes wrong? Is linked to issue #501

Member

gdha commented Aug 12, 2015

could you run rear -vD mkrescue to see where it goes wrong? Is linked to issue #501

@eholtz

This comment has been minimized.

Show comment
Hide comment
@eholtz

eholtz Aug 13, 2015

Sure, no problem. Unfortunately I'm not seeing any option to upload the logfile, therefore I put it here:
http://eholtz.de/rear-machine.log.xz

[@machine ~]# find / -iname "*.iso" -exec rm {} \;
[@machine ~]# /usr/sbin/rear -vD mkrescue
Relax-and-Recover 1.17.1 / Git
Using log file: /var/log/rear/rear-machine.log
Creating disk layout
Creating root filesystem layout
Copying files and directories
Copying binaries and libraries
Copying kernel modules
Creating initramfs
Making ISO image
Wrote ISO image: /var/lib/rear/output/rear-machine.iso (201M)
Saving result files with TSM
[@machine ~]# find / -iname "*.iso"
/opt/tivoli/tsm/rear/rear-machine.iso
/var/lib/rear/output/rear-machine.iso
[@machine ~]#

eholtz commented Aug 13, 2015

Sure, no problem. Unfortunately I'm not seeing any option to upload the logfile, therefore I put it here:
http://eholtz.de/rear-machine.log.xz

[@machine ~]# find / -iname "*.iso" -exec rm {} \;
[@machine ~]# /usr/sbin/rear -vD mkrescue
Relax-and-Recover 1.17.1 / Git
Using log file: /var/log/rear/rear-machine.log
Creating disk layout
Creating root filesystem layout
Copying files and directories
Copying binaries and libraries
Copying kernel modules
Creating initramfs
Making ISO image
Wrote ISO image: /var/lib/rear/output/rear-machine.iso (201M)
Saving result files with TSM
[@machine ~]# find / -iname "*.iso"
/opt/tivoli/tsm/rear/rear-machine.iso
/var/lib/rear/output/rear-machine.iso
[@machine ~]#
@gdha

This comment has been minimized.

Show comment
Hide comment
@gdha

gdha Aug 13, 2015

Member

@eholtz thank you for the log. Apparently the OUTPUT_URL=null was designed to work well with BACKUP=NTEFS scheme and was not implemented for other backup methods. This is not a bug, but a missing feature of the patch kindly donated by an user.
We need to enhance the code to make this happen in a generic way.

Member

gdha commented Aug 13, 2015

@eholtz thank you for the log. Apparently the OUTPUT_URL=null was designed to work well with BACKUP=NTEFS scheme and was not implemented for other backup methods. This is not a bug, but a missing feature of the patch kindly donated by an user.
We need to enhance the code to make this happen in a generic way.

@gdha

This comment has been minimized.

Show comment
Hide comment
@gdha

gdha Aug 13, 2015

Member

@eholtz if you define TSM_RM_ISOFILE=1 in /etc/rear/site.conf file then /opt/tivoli/tsm/rear/rear-machine.iso will be removed.
PS: I rather use the generic code for OUTPUT_URL=null everywhere

Member

gdha commented Aug 13, 2015

@eholtz if you define TSM_RM_ISOFILE=1 in /etc/rear/site.conf file then /opt/tivoli/tsm/rear/rear-machine.iso will be removed.
PS: I rather use the generic code for OUTPUT_URL=null everywhere

@gdha

This comment has been minimized.

Show comment
Hide comment
@gdha

gdha Aug 13, 2015

Member

@eholtz One last remark: by defining TSM_RESULT_SAVE=n you will not make a backup of the iso file towards TSM. Is this what you want? As you want all ISOs to be removed on the local system? I don't think so...

Member

gdha commented Aug 13, 2015

@eholtz One last remark: by defining TSM_RESULT_SAVE=n you will not make a backup of the iso file towards TSM. Is this what you want? As you want all ISOs to be removed on the local system? I don't think so...

@eholtz

This comment has been minimized.

Show comment
Hide comment
@eholtz

eholtz Aug 13, 2015

@gdha Thank you for your replies. The TSM-Backup is scheduled elsewhere, so as long as I have one copy left the iso will be saved once a day to the server. I don't want to upload it directly to TSM after the cronjob, because I would produce a heavy load on the server when multiple VMs are starting their jobs at the same time.
I will try if the TSM_RM_ISOFILE=1 works out for me and let you know.

eholtz commented Aug 13, 2015

@gdha Thank you for your replies. The TSM-Backup is scheduled elsewhere, so as long as I have one copy left the iso will be saved once a day to the server. I don't want to upload it directly to TSM after the cronjob, because I would produce a heavy load on the server when multiple VMs are starting their jobs at the same time.
I will try if the TSM_RM_ISOFILE=1 works out for me and let you know.

@eholtz

This comment has been minimized.

Show comment
Hide comment
@eholtz

eholtz Aug 14, 2015

@gdha I tried using

# configuration file for relax and recover
# don't upload to tsm directly as this is scheduled somewhere else
BACKUP=TSM
TSM_RESULT_SAVE=n
TSM_RM_ISOFILE=1

But I'm still left with two iso files:

[root@lxf101p001 ~]# find / -iname "*.iso" -exec rm {} \;
[root@lxf101p001 ~]# rear mkrescue
[root@lxf101p001 ~]# find / -iname "*.iso"
/opt/tivoli/tsm/rear/rear-lxf101p001.iso
/var/lib/rear/output/rear-lxf101p001.iso

Apparently the iso gets only removed when it has been uploaded to TSM (which I believe is kind of useful for standard environments)

eholtz commented Aug 14, 2015

@gdha I tried using

# configuration file for relax and recover
# don't upload to tsm directly as this is scheduled somewhere else
BACKUP=TSM
TSM_RESULT_SAVE=n
TSM_RM_ISOFILE=1

But I'm still left with two iso files:

[root@lxf101p001 ~]# find / -iname "*.iso" -exec rm {} \;
[root@lxf101p001 ~]# rear mkrescue
[root@lxf101p001 ~]# find / -iname "*.iso"
/opt/tivoli/tsm/rear/rear-lxf101p001.iso
/var/lib/rear/output/rear-lxf101p001.iso

Apparently the iso gets only removed when it has been uploaded to TSM (which I believe is kind of useful for standard environments)

@gdha

This comment has been minimized.

Show comment
Hide comment
@gdha

gdha Aug 19, 2015

Member

@eholtz Is your question answered? The OUTPUT_URL is only honored with BACKUP=NETFS scheme. If you would like it different then I would prefer another issue - to keep 2 discussions apart.

Member

gdha commented Aug 19, 2015

@eholtz Is your question answered? The OUTPUT_URL is only honored with BACKUP=NETFS scheme. If you would like it different then I would prefer another issue - to keep 2 discussions apart.

@eholtz

This comment has been minimized.

Show comment
Hide comment
@eholtz

eholtz Aug 20, 2015

@gdha Thanks for your reply. I do understand that OUTPUT_URL is only honored with BACKUP=NETFS scheme. The documentation should be updated to reflect that, maybe I'm not the only one who misunderstood that.
Nevertheless I think there should be an option to do a BACKUP=TSM in combination with TSM_RESULT_SAVE=n where I'm not left with two ISO images. I would like to contribute this myself, but I need a couple of hours to look into the code so I won't be able to do that in the next days.

eholtz commented Aug 20, 2015

@gdha Thanks for your reply. I do understand that OUTPUT_URL is only honored with BACKUP=NETFS scheme. The documentation should be updated to reflect that, maybe I'm not the only one who misunderstood that.
Nevertheless I think there should be an option to do a BACKUP=TSM in combination with TSM_RESULT_SAVE=n where I'm not left with two ISO images. I would like to contribute this myself, but I need a couple of hours to look into the code so I won't be able to do that in the next days.

@gdha

This comment has been minimized.

Show comment
Hide comment
@gdha

gdha Aug 20, 2015

Member

@eholtz @schlomo Well the OUTPUT_URL was never a concern (in most cases with external backup), but I agree you can use it to have second copy.
Wonder what Schlomo idea is about the scope of OUTPUT_URL=null: shall we stick to NETFS mechanism or go broader?

Member

gdha commented Aug 20, 2015

@eholtz @schlomo Well the OUTPUT_URL was never a concern (in most cases with external backup), but I agree you can use it to have second copy.
Wonder what Schlomo idea is about the scope of OUTPUT_URL=null: shall we stick to NETFS mechanism or go broader?

@schlomo

This comment has been minimized.

Show comment
Hide comment
@schlomo

schlomo Aug 20, 2015

Member

@gdha grepping through our code makes me think that the actual challange is that there is no single tool to handle all types of OUTPUT_URL which means that we need lots of code to handle the different types of URLs.

Another questions: Why do we need OUTPUT_URL=null if the default of OUTPUT_URL is empty? So far most ReaR variables can be set to empty or unset to disable a feature.

Member

schlomo commented Aug 20, 2015

@gdha grepping through our code makes me think that the actual challange is that there is no single tool to handle all types of OUTPUT_URL which means that we need lots of code to handle the different types of URLs.

Another questions: Why do we need OUTPUT_URL=null if the default of OUTPUT_URL is empty? So far most ReaR variables can be set to empty or unset to disable a feature.

@gdha

This comment has been minimized.

Show comment
Hide comment
@gdha

gdha Oct 2, 2015

Member

@schlomo Sorry for the delay - if OUTPUT_URL is empty it will get the same value as BACKUP_URL (which is mandatory) and it makes sense as you want your iso image saved (most likely on the same location as the tar archive)

Member

gdha commented Oct 2, 2015

@schlomo Sorry for the delay - if OUTPUT_URL is empty it will get the same value as BACKUP_URL (which is mandatory) and it makes sense as you want your iso image saved (most likely on the same location as the tar archive)

@gdha

This comment has been minimized.

Show comment
Hide comment
@gdha

gdha Feb 5, 2016

Member

Re-open this issue when we found a sponsor to implement this missing feature

Member

gdha commented Feb 5, 2016

Re-open this issue when we found a sponsor to implement this missing feature

@gdha gdha closed this Feb 5, 2016

@gdha

This comment has been minimized.

Show comment
Hide comment
@gdha

gdha Mar 8, 2017

Member

The solution is described in issue #802

Member

gdha commented Mar 8, 2017

The solution is described in issue #802

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment