/qubes-issues

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to restore in R3.0 backup of VMs made in R2 #1124

Closed
dncohen opened this Issue Aug 16, 2015 · 7 comments

Comments

Assignees
No one assigned
Labels
Projects
None yet
Milestone
  Release 3.0
4 participants
@dncohen @wyory @marmarek @Chilinot
@dncohen

dncohen commented Aug 16, 2015

I've created a backup of several VMs in Qubes 2, and I'm trying to restore those backups to Qubes R3.0 rc2. If I understand correctly, this is the proper way to upgrade from R2 to R3.0.

However, I run into the error shown below. I get this error whether I use the command line (shown) or the Qubes VM Manager gui.

[dave@dom0 ~]$ qvm-backup-restore --verify-only /run/media/dave/ea45d096-05e3-4fb7-b5ff-e710c818c52e/qubes-backup-2015-08-15T211622                            
Please enter the pass phrase that will be used to decrypt/verify the backup:    
Checking backup content...                                                      
Extracting data: 1.0 MiB to restore                                            
Traceback (most recent call last):                                              
  File "/usr/bin/qvm-backup-restore", line 259, in <module>                    
    main()                                                                      
  File "/usr/bin/qvm-backup-restore", line 147, in main                        
    error_callback=error_callback)                                              
  File "/usr/lib64/python2.7/site-packages/qubes/backup.py", line 1784, in backup_restore_prepare                                                              
    backup_collection.load()                                                    
  File "/usr/lib64/python2.7/site-packages/qubes/qubes.py", line 813, in load  
    vm = vm_class(xml_element=element, collection=self)                        
  File "/usr/lib64/python2.7/site-packages/qubes/modules/01QubesDisposableVm.py", line 100, in __init__
    super(QubesDisposableVm, self).__init__(**kwargs)
  File "/usr/lib64/python2.7/site-packages/qubes/modules/000QubesVm.py", line 287, in __init__
    setattr(self, attr, attr_config['func'](value))
  File "/usr/lib64/python2.7/site-packages/qubes/modules/000QubesVm.py", line 110, in <lambda>
    ".conf"),
  File "/usr/lib64/python2.7/site-packages/qubes/modules/000QubesVm.py", line 373, in absolute_path
    return os.path.join(self.dir_path, (arg if arg is not None else default))
  File "/usr/lib64/python2.7/posixpath.py", line 77, in join
    elif path == '' or path.endswith('/'):
AttributeError: 'NoneType' object has no attribute 'endswith'
@wyory

This comment has been minimized.

Show comment
Hide comment
@wyory

wyory Aug 22, 2015

I had the same problem: https://groups.google.com/d/msg/qubes-devel/ZjFlqD2uKjM/I_kV7whKCQAJ

I started following the manual rescue procedure described here:
https://www.qubes-os.org/doc/BackupEmergencyRestoreV3/

I verified the integrity of the backup-header. Then I noticed that the
time names of the archive seem a bit off. Some read like this:

vm38/private.img.000
vm38/private.img.000.hmac
vm38/icon.png.000
vm38/icon.png.000.hmac
vm38/apps.templates.000
vm38/apps.templates.000.hmac
vm38/root.img.000
vm38/root.img.000.hmac
vm38/root.img.001
vm38/root.img.001.hmac

But others read like this:

vm25/..000
vm25/..000.hmac
vm25/..001
vm25/..001.hmac
vm25/..002
vm25/..002.hmac
vm25/..003
vm25/..003.hmac
vm25/..004
vm25/..004.hmac

Could this be causing the error?

wyory commented Aug 22, 2015

I had the same problem: https://groups.google.com/d/msg/qubes-devel/ZjFlqD2uKjM/I_kV7whKCQAJ

I started following the manual rescue procedure described here:
https://www.qubes-os.org/doc/BackupEmergencyRestoreV3/

I verified the integrity of the backup-header. Then I noticed that the
time names of the archive seem a bit off. Some read like this:

vm38/private.img.000
vm38/private.img.000.hmac
vm38/icon.png.000
vm38/icon.png.000.hmac
vm38/apps.templates.000
vm38/apps.templates.000.hmac
vm38/root.img.000
vm38/root.img.000.hmac
vm38/root.img.001
vm38/root.img.001.hmac

But others read like this:

vm25/..000
vm25/..000.hmac
vm25/..001
vm25/..001.hmac
vm25/..002
vm25/..002.hmac
vm25/..003
vm25/..003.hmac
vm25/..004
vm25/..004.hmac

Could this be causing the error?
@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Aug 26, 2015

Member

On Sat, Aug 22, 2015 at 12:38:22PM -0700, wyory wrote:

I had the same problem: https://groups.google.com/d/msg/qubes-devel/ZjFlqD2uKjM/I_kV7whKCQAJ

I started following the manual rescue procedure described here:
https://www.qubes-os.org/doc/BackupEmergencyRestoreV3/

I verified the integrity of the backup-header. Then I noticed that the
time names of the archive seem a bit off. Some read like this:

vm38/private.img.000
vm38/private.img.000.hmac
vm38/icon.png.000
vm38/icon.png.000.hmac
vm38/apps.templates.000
vm38/apps.templates.000.hmac
vm38/root.img.000
vm38/root.img.000.hmac
vm38/root.img.001
vm38/root.img.001.hmac

But others read like this:

vm25/..000
vm25/..000.hmac
vm25/..001
vm25/..001.hmac
vm25/..002
vm25/..002.hmac
vm25/..003
vm25/..003.hmac
vm25/..004
vm25/..004.hmac

Could this be causing the error?

This is ok - some VMs (namely: templates) have the whole VM directory
backed up ("."), not only selected files ("private.img", "icon.png",
etc).

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
Member

marmarek commented Aug 26, 2015

On Sat, Aug 22, 2015 at 12:38:22PM -0700, wyory wrote:

I had the same problem: https://groups.google.com/d/msg/qubes-devel/ZjFlqD2uKjM/I_kV7whKCQAJ

I started following the manual rescue procedure described here:
https://www.qubes-os.org/doc/BackupEmergencyRestoreV3/

I verified the integrity of the backup-header. Then I noticed that the
time names of the archive seem a bit off. Some read like this:

vm38/private.img.000
vm38/private.img.000.hmac
vm38/icon.png.000
vm38/icon.png.000.hmac
vm38/apps.templates.000
vm38/apps.templates.000.hmac
vm38/root.img.000
vm38/root.img.000.hmac
vm38/root.img.001
vm38/root.img.001.hmac

But others read like this:

vm25/..000
vm25/..000.hmac
vm25/..001
vm25/..001.hmac
vm25/..002
vm25/..002.hmac
vm25/..003
vm25/..003.hmac
vm25/..004
vm25/..004.hmac

Could this be causing the error?

This is ok - some VMs (namely: templates) have the whole VM directory
backed up ("."), not only selected files ("private.img", "icon.png",
etc).

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Aug 26, 2015

Member

On Sat, Aug 15, 2015 at 11:13:49PM -0700, dncohen wrote:

File "/usr/lib64/python2.7/site-packages/qubes/backup.py", line 1784, in backup_restore_prepare
backup_collection.load()
File "/usr/lib64/python2.7/site-packages/qubes/qubes.py", line 813, in load
vm = vm_class(xml_element=element, collection=self)
File "/usr/lib64/python2.7/site-packages/qubes/modules/01QubesDisposableVm.py", line 100, in init
super(QubesDisposableVm, self).init(**kwargs)

DisposableVM included in the backup? Strange...

Could you post qubes.xml from that backup? It is probably left in
/var/tmp/restore_*/. If you're not comfortable with posting full list
of your VMs, just get the line starting with "<QubesDisposableVm".

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
Member

marmarek commented Aug 26, 2015

On Sat, Aug 15, 2015 at 11:13:49PM -0700, dncohen wrote:

File "/usr/lib64/python2.7/site-packages/qubes/backup.py", line 1784, in backup_restore_prepare
backup_collection.load()
File "/usr/lib64/python2.7/site-packages/qubes/qubes.py", line 813, in load
vm = vm_class(xml_element=element, collection=self)
File "/usr/lib64/python2.7/site-packages/qubes/modules/01QubesDisposableVm.py", line 100, in init
super(QubesDisposableVm, self).init(**kwargs)

DisposableVM included in the backup? Strange...

Could you post qubes.xml from that backup? It is probably left in
/var/tmp/restore_*/. If you're not comfortable with posting full list
of your VMs, just get the line starting with "<QubesDisposableVm".

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
@wyory

This comment has been minimized.

Show comment
Hide comment
@wyory

wyory Aug 28, 2015

Marek Marczykowski-Górecki:

On Sat, Aug 15, 2015 at 11:13:49PM -0700, dncohen wrote:

File "/usr/lib64/python2.7/site-packages/qubes/backup.py", line 1784, in backup_restore_prepare
backup_collection.load()
File "/usr/lib64/python2.7/site-packages/qubes/qubes.py", line 813, in load
vm = vm_class(xml_element=element, collection=self)
File "/usr/lib64/python2.7/site-packages/qubes/modules/01QubesDisposableVm.py", line 100, in init
super(QubesDisposableVm, self).init(**kwargs)

DisposableVM included in the backup? Strange...

Could you post qubes.xml from that backup? It is probably left in
/var/tmp/restore_*/. If you're not comfortable with posting full list
of your VMs, just get the line starting with "<QubesDisposableVm".

I also have reference to disposable VM in my error message:

line: super(QubesDisposableVm, self).init(**kwargs)
func: init
line no.: 100
file:
/usr/lib64/python2.7/site-packages/qubes/modules/01QubesDisposableVm.py

I found qubes.xml.000 in my backup file, but when I try to open it with
less or vi, it's in binary. How do I find the line your refer to? Is it
just gzip or something?

wyory commented Aug 28, 2015

Marek Marczykowski-Górecki:

On Sat, Aug 15, 2015 at 11:13:49PM -0700, dncohen wrote:

File "/usr/lib64/python2.7/site-packages/qubes/backup.py", line 1784, in backup_restore_prepare
backup_collection.load()
File "/usr/lib64/python2.7/site-packages/qubes/qubes.py", line 813, in load
vm = vm_class(xml_element=element, collection=self)
File "/usr/lib64/python2.7/site-packages/qubes/modules/01QubesDisposableVm.py", line 100, in init
super(QubesDisposableVm, self).init(**kwargs)

DisposableVM included in the backup? Strange...

Could you post qubes.xml from that backup? It is probably left in
/var/tmp/restore_*/. If you're not comfortable with posting full list
of your VMs, just get the line starting with "<QubesDisposableVm".

I also have reference to disposable VM in my error message:

line: super(QubesDisposableVm, self).init(**kwargs)
func: init
line no.: 100
file:
/usr/lib64/python2.7/site-packages/qubes/modules/01QubesDisposableVm.py

I found qubes.xml.000 in my backup file, but when I try to open it with
less or vi, it's in binary. How do I find the line your refer to? Is it
just gzip or something?
@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Aug 28, 2015

Member

On Thu, Aug 27, 2015 at 10:54:29PM -0700, wyory wrote:

I found qubes.xml.000 in my backup file, but when I try to open it with
less or vi, it's in binary. How do I find the line your refer to? Is it
just gzip or something?

It is encrypted, the same as the other files. See emergency restore
instruction - step 5 and 6.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
Member

marmarek commented Aug 28, 2015

On Thu, Aug 27, 2015 at 10:54:29PM -0700, wyory wrote:

I found qubes.xml.000 in my backup file, but when I try to open it with
less or vi, it's in binary. How do I find the line your refer to? Is it
just gzip or something?

It is encrypted, the same as the other files. See emergency restore
instruction - step 5 and 6.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
@wyory

This comment has been minimized.

Show comment
Hide comment
@wyory

wyory Aug 30, 2015

Marek Marczykowski-Górecki:

On Thu, Aug 27, 2015 at 10:54:29PM -0700, wyory wrote:

I found qubes.xml.000 in my backup file, but when I try to open it with
less or vi, it's in binary. How do I find the line your refer to? Is it
just gzip or something?

It is encrypted, the same as the other files. See emergency restore
instruction - step 5 and 6.

Thanks. Here are the lines starting with "<QubesDisposableVM"

wyory commented Aug 30, 2015

Marek Marczykowski-Górecki:

On Thu, Aug 27, 2015 at 10:54:29PM -0700, wyory wrote:

I found qubes.xml.000 in my backup file, but when I try to open it with
less or vi, it's in binary. How do I find the line your refer to? Is it
just gzip or something?

It is encrypted, the same as the other files. See emergency restore
instruction - step 5 and 6.

Thanks. Here are the lines starting with "<QubesDisposableVM"

@marmarek marmarek added bug C: core P: critical labels Sep 2, 2015

@marmarek marmarek added this to the Release 3.0 milestone Sep 2, 2015

@marmarek marmarek closed this in marmarek/old-qubes-core-admin@76bddaa Sep 4, 2015

marmarek added a commit to marmarek/old-qubes-core-admin that referenced this issue Sep 4, 2015

@marmarek
tests: add DispVM to backup compatibility test (r2b2) 
Test for QubesOS/qubes-issues#1124
496a74f

marmarek added a commit to marmarek/old-qubes-core-admin that referenced this issue Sep 4, 2015

@marmarek
tests: backup compatibility with R2 (#1124) 
QubesOS/qubes-issues#1124
c2cd849

marmarek added a commit to marmarek/old-qubes-core-admin-linux that referenced this issue Sep 4, 2015

@marmarek
core: use vm.absolute_path to parse paths in qubes.xml 
This makes easier to handle some corner cases.

QubesOS/qubes-issues#1124
Reported by @doncohen, thanks @wyory for providing more details.
4e1f6b5

marmarek added a commit to marmarek/old-qubes-core-admin that referenced this issue Sep 5, 2015

@marmarek
backup: fix R2B3 and older backup restore (#1124) 
Similar issue to QubesOS/qubes-issues#1124
9205fbb

@marmarek marmarek referenced this issue in marmarek/old-qubes-core-admin Sep 5, 2015

Merged

backup: fix R2B3 and older backup restore (#1124) #4

marmarek added a commit to marmarek/old-qubes-core-admin that referenced this issue Oct 2, 2015

@marmarek
backup: fix R2B3 and older backup restore (#1124) 
Similar issue to QubesOS/qubes-issues#1124

(cherry picked from commit 9205fbb)
Verified
This commit was signed with a verified signature.
@marmarek marmarek Marek Marczykowski-Górecki
GPG key ID: 063938BA42CFA724 Learn about signing commits
d75544a

@andrewdavidwong andrewdavidwong referenced this issue Nov 30, 2015

Open

Backup: Invalid HMAC on verification #1471

@Chilinot

This comment has been minimized.

Show comment
Hide comment
@Chilinot

Chilinot Jul 13, 2016

I'm still encountering this issue on release 3.1, as well as on release candidate 3.2.

EDIT: My bad, not the exact same error output; but similar. I will create a new issue.

Chilinot commented Jul 13, 2016 •
edited
Edited 1 time
  • @Chilinot Chilinot edited Jul 13, 2016 (most recent)

I'm still encountering this issue on release 3.1, as well as on release candidate 3.2.

EDIT: My bad, not the exact same error output; but similar. I will create a new issue.

@Chilinot Chilinot referenced this issue Jul 13, 2016

Closed

Unable to restore backup from R3.1 to R3.1 & R3.2 #2168

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment