New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tar --test-label is not supported on Centos 5 who have tar version 1.15. #1014

Closed
philippebolduc opened this Issue Sep 28, 2016 · 15 comments

Comments

Projects
None yet
3 participants
@philippebolduc

philippebolduc commented Sep 28, 2016

Relax-and-Recover (rear) Issue Template

Please fill in the following items before submitting a new issue:

  • rear version (/usr/sbin/rear -V):
  • OS version (cat /etc/rear/os.conf or lsb_release -a):
  • rear configuration files (cat /etc/rear/site.conf or cat /etc/rear/local.conf):
  • Brief description of the issue
  • Work-around, if any
@philippebolduc

This comment has been minimized.

Show comment
Hide comment
@philippebolduc

philippebolduc Sep 28, 2016

Relax-and-Recover 1.18 / Git
cat /etc/rear/os.conf
OS_VENDOR=RedHatEnterpriseServer
OS_VERSION=5
cat /etc/rear/local.conf

# sample local configuration
# Create Rear rescue media as ISO image
OUTPUT=ISO
USE_DHCLIENT=yes 
# optionally define (non-default) backup software, e.g. TSM, NBU, DP, BACULA
# BACKUP=TSM
BACKUP_TYPE=incremental
FULLBACKUPDAY="Sun"
BACKUP=NETFS
BACKUP_URL=nfs://10.2.1.40/backup
# the following is required on older VMware VMs
MODULES_LOAD=( vmxnet )
# to see boot messages on the serial console (uncomment next line)
# KERNEL_CMDLINE="console=tty0 console=ttyS1

Restore do not work

Change line 53
From
BASE=$BASEDIR/$(tar --test-label -f "$restorearchive")
To
BASE=$BASEDIR/$(cat $BASEDIR/basebackup.txt)

philippebolduc commented Sep 28, 2016

Relax-and-Recover 1.18 / Git
cat /etc/rear/os.conf
OS_VENDOR=RedHatEnterpriseServer
OS_VERSION=5
cat /etc/rear/local.conf

# sample local configuration
# Create Rear rescue media as ISO image
OUTPUT=ISO
USE_DHCLIENT=yes 
# optionally define (non-default) backup software, e.g. TSM, NBU, DP, BACULA
# BACKUP=TSM
BACKUP_TYPE=incremental
FULLBACKUPDAY="Sun"
BACKUP=NETFS
BACKUP_URL=nfs://10.2.1.40/backup
# the following is required on older VMware VMs
MODULES_LOAD=( vmxnet )
# to see boot messages on the serial console (uncomment next line)
# KERNEL_CMDLINE="console=tty0 console=ttyS1

Restore do not work

Change line 53
From
BASE=$BASEDIR/$(tar --test-label -f "$restorearchive")
To
BASE=$BASEDIR/$(cat $BASEDIR/basebackup.txt)

@jsmeix

This comment has been minimized.

Show comment
Hide comment
@jsmeix

jsmeix Sep 28, 2016

Contributor

@gdha
because this issue is related to your
847323f
I assign it also to you.

I will prepare a pull request and then you can decide
if you accept it.

Contributor

jsmeix commented Sep 28, 2016

@gdha
because this issue is related to your
847323f
I assign it also to you.

I will prepare a pull request and then you can decide
if you accept it.

jsmeix added a commit to jsmeix/rear that referenced this issue Sep 28, 2016

jsmeix referenced this issue Sep 28, 2016

Fix the encrypted incremental restore (40_restore_backup.sh) - see is…
…sue #952

Proof:

2016-08-22 10:24:25 Including restore/NETFS/default/40_restore_backup.sh
2016-08-22 10:24:25 Decrypting archive with key defined in variable $BACKUP_PROG_CRYPT_KEY
2016-08-22 10:24:25 Restoring tar archive '/tmp/rear.C0XJtMMUTTzGhHm/outputfs/client/2016-08-22-1215-I.tar.gz'
2016-08-22 10:24:25 dd if=/tmp/rear.C0XJtMMUTTzGhHm/outputfs/client/2016-08-21-2037-F.tar.gz | /usr/bin/openssl aes256 -d -k brpback
up123 | tar --block-number --totals --verbose --anchored --gzip -C /mnt/local/ -x -f -
WARNING: can't open config file: /etc/pki/tls/openssl.cnf
1391362+1 records in
1391362+1 records out
712377632 bytes (712 MB) copied, 39.8033 s, 17.9 MB/s
Total bytes read: 1551441920 (1.5GiB, 38MiB/s)
2016-08-22 10:25:05 dd if=/tmp/rear.C0XJtMMUTTzGhHm/outputfs/client/2016-08-22-1215-I.tar.gz | /usr/bin/openssl aes256 -d -k brpback
up123 | tar --block-number --totals --verbose --anchored --gzip -C /mnt/local/ -x -f -
WARNING: can't open config file: /etc/pki/tls/openssl.cnf
452228+1 records in
452228+1 records out
231541072 bytes (232 MB) copied, 16.6899 s, 13.9 MB/s
Total bytes read: 581386240 (555MiB, 34MiB/s)
2016-08-22 10:25:23 Restored 531 MiB in 57 seconds [avg 9540 KiB/sec]
@jsmeix

This comment has been minimized.

Show comment
Hide comment
@jsmeix

jsmeix Sep 28, 2016

Contributor

This is the right pull request:
#1016

Contributor

jsmeix commented Sep 28, 2016

This is the right pull request:
#1016

@philippebolduc

This comment has been minimized.

Show comment
Hide comment
@philippebolduc

philippebolduc Sep 28, 2016

Ok what should I do now ?

Thanks.

Philippe Bolduc
Administrateur système Linux / Programmeur analyste
Linux System Administrator / Programmer Analyst

http://www.florenceinc.com/
3382, boul. Industriel
Laval (Québec)
H7L 4R9
T :: 514-875-7525 #235
F :: 514-875-7536
C :: philippe@florenceinc.com
W :: florenceinc.com
Le 2016-09-28 à 11:04, Johannes Meixner a écrit :

This is the right pull request:
#1016 #1016


You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#1014 (comment), or
mute the thread
https://github.com/notifications/unsubscribe-auth/AJuhY1vF6a7t4Cs4v1wOrf7TNADchK3-ks5quoH3gaJpZM4KI4SQ.

philippebolduc commented Sep 28, 2016

Ok what should I do now ?

Thanks.

Philippe Bolduc
Administrateur système Linux / Programmeur analyste
Linux System Administrator / Programmer Analyst

http://www.florenceinc.com/
3382, boul. Industriel
Laval (Québec)
H7L 4R9
T :: 514-875-7525 #235
F :: 514-875-7536
C :: philippe@florenceinc.com
W :: florenceinc.com
Le 2016-09-28 à 11:04, Johannes Meixner a écrit :

This is the right pull request:
#1016 #1016


You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#1014 (comment), or
mute the thread
https://github.com/notifications/unsubscribe-auth/AJuhY1vF6a7t4Cs4v1wOrf7TNADchK3-ks5quoH3gaJpZM4KI4SQ.

@jsmeix

This comment has been minimized.

Show comment
Hide comment
@jsmeix

jsmeix Sep 28, 2016

Contributor

@philippebolduc
do nothing - just wait (like me) what @gdha will tell us.

Contributor

jsmeix commented Sep 28, 2016

@philippebolduc
do nothing - just wait (like me) what @gdha will tell us.

@philippebolduc

This comment has been minimized.

Show comment
Hide comment
@philippebolduc

philippebolduc Sep 28, 2016

Ok thanks.

Philippe Bolduc
Administrateur système Linux / Programmeur analyste
Linux System Administrator / Programmer Analyst

http://www.florenceinc.com/
3382, boul. Industriel
Laval (Québec)
H7L 4R9
T :: 514-875-7525 #235
F :: 514-875-7536
C :: philippe@florenceinc.com
W :: florenceinc.com
Le 2016-09-28 à 11:25, Johannes Meixner a écrit :

@philippebolduc https://github.com/philippebolduc
do nothing - just wait (like me) what @gdha https://github.com/gdha
will tell us.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1014 (comment), or
mute the thread
https://github.com/notifications/unsubscribe-auth/AJuhY1P87YTbSWzSPgZeOOFhGcGY4la8ks5quobggaJpZM4KI4SQ.

philippebolduc commented Sep 28, 2016

Ok thanks.

Philippe Bolduc
Administrateur système Linux / Programmeur analyste
Linux System Administrator / Programmer Analyst

http://www.florenceinc.com/
3382, boul. Industriel
Laval (Québec)
H7L 4R9
T :: 514-875-7525 #235
F :: 514-875-7536
C :: philippe@florenceinc.com
W :: florenceinc.com
Le 2016-09-28 à 11:25, Johannes Meixner a écrit :

@philippebolduc https://github.com/philippebolduc
do nothing - just wait (like me) what @gdha https://github.com/gdha
will tell us.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1014 (comment), or
mute the thread
https://github.com/notifications/unsubscribe-auth/AJuhY1P87YTbSWzSPgZeOOFhGcGY4la8ks5quobggaJpZM4KI4SQ.

@gdha

This comment has been minimized.

Show comment
Hide comment
@gdha

gdha Sep 29, 2016

Member

@jsmeix IMHO we should work with tar_version=$(get_version tar --version) and make a decision based on a version of tar from when the option --test-label was introduced. I found evidence on the Internet it was introduced in version 1.15.90 (see http://cvs.savannah.gnu.org/viewvc/*checkout*/tar/NEWS?view=auto&root=tar&revision=1.125) (released on 2006-02-19).
@philippebolduc My suggestion to you is to upgrade tar to at least version 1.15.92 and your problem is over.

I have the feeling it is not worth the effort to modify the code in rear as the option --test-label is already 10 years present within tar.

Member

gdha commented Sep 29, 2016

@jsmeix IMHO we should work with tar_version=$(get_version tar --version) and make a decision based on a version of tar from when the option --test-label was introduced. I found evidence on the Internet it was introduced in version 1.15.90 (see http://cvs.savannah.gnu.org/viewvc/*checkout*/tar/NEWS?view=auto&root=tar&revision=1.125) (released on 2006-02-19).
@philippebolduc My suggestion to you is to upgrade tar to at least version 1.15.92 and your problem is over.

I have the feeling it is not worth the effort to modify the code in rear as the option --test-label is already 10 years present within tar.

@philippebolduc

This comment has been minimized.

Show comment
Hide comment
@philippebolduc

philippebolduc Sep 29, 2016

Do tar 1.15.92 avalaible easily via rpm on Centos 5 ?

Thanks.

Philippe Bolduc
Administrateur système Linux / Programmeur analyste
Linux System Administrator / Programmer Analyst

http://www.florenceinc.com/
3382, boul. Industriel
Laval (Québec)
H7L 4R9
T :: 514-875-7525 #235
F :: 514-875-7536
C :: philippe@florenceinc.com
W :: florenceinc.com
Le 2016-09-29 à 07:33, gdha a écrit :

@jsmeix https://github.com/jsmeix IMHO we should work with
|tar_version=$(get_version tar --version)| and make a decision based
on a version of |tar| from when the option |--test-label| was
introduced. I found evidence on the Internet it was introduced in
version 1.15.90 (see
http://cvs.savannah.gnu.org/viewvc/*checkout*/tar/NEWS?view=auto&root=tar&revision=1.125)
(released on 2006-02-19).
@philippebolduc https://github.com/philippebolduc My suggestion to
you is to upgrade |tar| to at least version 1.15.92 and your problem
is over.

I have the feeling it is not worth the effort to modify the code in
/rear/ as the option |--test-label| is already 10 years present within
|tar|.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1014 (comment), or
mute the thread
https://github.com/notifications/unsubscribe-auth/AJuhY1s7skTeF6psmSVjldOAuocDTkObks5qu6HzgaJpZM4KI4SQ.

philippebolduc commented Sep 29, 2016

Do tar 1.15.92 avalaible easily via rpm on Centos 5 ?

Thanks.

Philippe Bolduc
Administrateur système Linux / Programmeur analyste
Linux System Administrator / Programmer Analyst

http://www.florenceinc.com/
3382, boul. Industriel
Laval (Québec)
H7L 4R9
T :: 514-875-7525 #235
F :: 514-875-7536
C :: philippe@florenceinc.com
W :: florenceinc.com
Le 2016-09-29 à 07:33, gdha a écrit :

@jsmeix https://github.com/jsmeix IMHO we should work with
|tar_version=$(get_version tar --version)| and make a decision based
on a version of |tar| from when the option |--test-label| was
introduced. I found evidence on the Internet it was introduced in
version 1.15.90 (see
http://cvs.savannah.gnu.org/viewvc/*checkout*/tar/NEWS?view=auto&root=tar&revision=1.125)
(released on 2006-02-19).
@philippebolduc https://github.com/philippebolduc My suggestion to
you is to upgrade |tar| to at least version 1.15.92 and your problem
is over.

I have the feeling it is not worth the effort to modify the code in
/rear/ as the option |--test-label| is already 10 years present within
|tar|.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1014 (comment), or
mute the thread
https://github.com/notifications/unsubscribe-auth/AJuhY1s7skTeF6psmSVjldOAuocDTkObks5qu6HzgaJpZM4KI4SQ.

@jsmeix

This comment has been minimized.

Show comment
Hide comment
@jsmeix

jsmeix Sep 29, 2016

Contributor

@gdha
I agree that rear does not need to support very old versions
of tools that are called by the rear scripts - in particular
because basically nobody tests that.

On the other hand doc/rear-release-notes.txt currently reads

Rear-1.18 is supported on the following Linux based operating systems:
...
CentOS 5, 6 and 7

But on one more other hand (again a "tertium datur" ;-)
the rear scripts are meant to be adapted by the admin
as needed in his particular case so that it can be considered
to be o.k. when @philippebolduc must adapt rear to make it work
for his CentOS 5 (or upgrade tar on his CentOS 5).

I think we should not do the effort to keep
the current rear working also for tar < 1.15.92

As a consequence for the upcoming rear-1.19
we may drop officially support for CentOS 5.

Some forensics on that issue:

The string '--test-label' is only in
restore/NETFS/default/40_restore_backup.sh
and
"git log -p -w restore/NETFS/default/40_restore_backup.sh"
shows when the '--test-label' was introduced:

commit 354486da3191dbfe5c6a4b896e34977be10697d5
Author: test 
Date:   Mon Sep 2 11:39:52 2013 +0200
...
+                       BASE=$(dirname "$restorearchive")/$(tar --test-label -f "$restorearchive")

but "git log" does not show any useful message for that commit:

commit 354486da3191dbfe5c6a4b896e34977be10697d5
Author: test 
Date:   Mon Sep 2 11:39:52 2013 +0200
    Incrementap Patches

I.e. it was this commit
354486d

According to
https://en.wikipedia.org/wiki/CentOS
CentOS 5 was releasted in April 2007
and that commit happened in 2013
so that I assume before rear had worked for CentOS 5
but since that commit it does no longer work for CentOS 5.

Contributor

jsmeix commented Sep 29, 2016

@gdha
I agree that rear does not need to support very old versions
of tools that are called by the rear scripts - in particular
because basically nobody tests that.

On the other hand doc/rear-release-notes.txt currently reads

Rear-1.18 is supported on the following Linux based operating systems:
...
CentOS 5, 6 and 7

But on one more other hand (again a "tertium datur" ;-)
the rear scripts are meant to be adapted by the admin
as needed in his particular case so that it can be considered
to be o.k. when @philippebolduc must adapt rear to make it work
for his CentOS 5 (or upgrade tar on his CentOS 5).

I think we should not do the effort to keep
the current rear working also for tar < 1.15.92

As a consequence for the upcoming rear-1.19
we may drop officially support for CentOS 5.

Some forensics on that issue:

The string '--test-label' is only in
restore/NETFS/default/40_restore_backup.sh
and
"git log -p -w restore/NETFS/default/40_restore_backup.sh"
shows when the '--test-label' was introduced:

commit 354486da3191dbfe5c6a4b896e34977be10697d5
Author: test 
Date:   Mon Sep 2 11:39:52 2013 +0200
...
+                       BASE=$(dirname "$restorearchive")/$(tar --test-label -f "$restorearchive")

but "git log" does not show any useful message for that commit:

commit 354486da3191dbfe5c6a4b896e34977be10697d5
Author: test 
Date:   Mon Sep 2 11:39:52 2013 +0200
    Incrementap Patches

I.e. it was this commit
354486d

According to
https://en.wikipedia.org/wiki/CentOS
CentOS 5 was releasted in April 2007
and that commit happened in 2013
so that I assume before rear had worked for CentOS 5
but since that commit it does no longer work for CentOS 5.

@gdha

This comment has been minimized.

Show comment
Hide comment
@gdha

gdha Sep 29, 2016

Member

@jsmeix For CentOS 5: thetar --test-label is only used when BACKUP_TYPE=incremental has been selected, therefore, it is only problematic in some cases. I can comment it in the FAQ section of our web pages?

@philippebolduc If you do not find a suitable rpm for tar v1.15.9x you could try to build it from the sources?

Member

gdha commented Sep 29, 2016

@jsmeix For CentOS 5: thetar --test-label is only used when BACKUP_TYPE=incremental has been selected, therefore, it is only problematic in some cases. I can comment it in the FAQ section of our web pages?

@philippebolduc If you do not find a suitable rpm for tar v1.15.9x you could try to build it from the sources?

@jsmeix jsmeix added won't fix / can't fix and removed bug labels Sep 29, 2016

@jsmeix

This comment has been minimized.

Show comment
Hide comment
@jsmeix

jsmeix Sep 29, 2016

Contributor

Ah!
Now I understand (at least I think so).

The commit
354486d
did not break existing functionality for CentOS 5
but only implemented the new functionality
"BACKUP_TYPE=incremental" is a way that
does not work with tar < 1.15.92.

If I am right it means there is no real bug here
(i.e. I remove "bug" from that issue and keep
only "enhancement").

Because we will not do the work to make the new
"BACKUP_TYPE=incremental" functionality
working with old tar < 1.15.92 I close this
enhancement request as "won't fix".

Contributor

jsmeix commented Sep 29, 2016

Ah!
Now I understand (at least I think so).

The commit
354486d
did not break existing functionality for CentOS 5
but only implemented the new functionality
"BACKUP_TYPE=incremental" is a way that
does not work with tar < 1.15.92.

If I am right it means there is no real bug here
(i.e. I remove "bug" from that issue and keep
only "enhancement").

Because we will not do the work to make the new
"BACKUP_TYPE=incremental" functionality
working with old tar < 1.15.92 I close this
enhancement request as "won't fix".

@jsmeix

This comment has been minimized.

Show comment
Hide comment
@jsmeix

jsmeix Sep 29, 2016

Contributor

@philippebolduc
regardless that we will not fix the issue
many thanks for your valuable contribution
to Relax-and-Recover because now we learned under
what special circumstances tar < 1.15.92 fails and you
even provided a "quick and dirty" workaround that could
also be used by other users who have that issue.

Contributor

jsmeix commented Sep 29, 2016

@philippebolduc
regardless that we will not fix the issue
many thanks for your valuable contribution
to Relax-and-Recover because now we learned under
what special circumstances tar < 1.15.92 fails and you
even provided a "quick and dirty" workaround that could
also be used by other users who have that issue.

@jsmeix

This comment has been minimized.

Show comment
Hide comment
@jsmeix

jsmeix Sep 29, 2016

Contributor

@philippebolduc
in general regarding "BACKUP_TYPE=incremental" note its limits
that are described only in the current documentation
cf. #974 and
35fc7c9
in short:

The current implementation supports only to restore
one full backup plus one single incremental backup
so that currently BACKUP_TYPE=incremental
actually implements a differential backup
Contributor

jsmeix commented Sep 29, 2016

@philippebolduc
in general regarding "BACKUP_TYPE=incremental" note its limits
that are described only in the current documentation
cf. #974 and
35fc7c9
in short:

The current implementation supports only to restore
one full backup plus one single incremental backup
so that currently BACKUP_TYPE=incremental
actually implements a differential backup
@jsmeix

This comment has been minimized.

Show comment
Hide comment
@jsmeix

jsmeix Sep 29, 2016

Contributor

@philippebolduc
that "we will not fix the issue" does of course not mean
that another contributor (e.g. you) could not enhance
the current "BACKUP_TYPE=incremental" functionality
so that it also works with old tar < 1.15.92 provided
the enhancement is made in a way that it cannot
cause regressions for users with newer tar.

Of course we appreciate GitHub pull requests
that enhance Relax-and-Recover functionality.

You may have a look at
https://github.com/rear/rear/wiki/Coding-Style

Contributor

jsmeix commented Sep 29, 2016

@philippebolduc
that "we will not fix the issue" does of course not mean
that another contributor (e.g. you) could not enhance
the current "BACKUP_TYPE=incremental" functionality
so that it also works with old tar < 1.15.92 provided
the enhancement is made in a way that it cannot
cause regressions for users with newer tar.

Of course we appreciate GitHub pull requests
that enhance Relax-and-Recover functionality.

You may have a look at
https://github.com/rear/rear/wiki/Coding-Style

@gdha

This comment has been minimized.

Show comment
Hide comment
@gdha

gdha Oct 4, 2016

Member

Added a note in the release notes of 1.19

Member

gdha commented Oct 4, 2016

Added a note in the release notes of 1.19

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment