[R4.0 RC2] qvm-backup-restore or qvm-backup error #3211

na-- · Oct 25, 2017

Qubes OS version:

R4.0 RC2

Affected TemplateVMs:

none (dom0 issue)

Steps to reproduce the behavior:

Create a backup with qvm-backup --compress /some/file some-vm and then try to restore from it via qvm-backup-restore /some/file

Expected behavior:

Successful restore of the backed up VM.

Actual behavior:

The following error is displayed:

$ qvm-backup-restore -v /backup/test1 
Please enter the passphrase to verify and (if encrypted) decrypt the backup: 
2017-10-25 23:20:08,831 [MainProcess qvm_backup_restore.main:215] app: Checking backup content...
2017-10-25 23:20:08,833 [MainProcess restore._start_retrieval_process:882] qubesadmin.backup: Run command['tar', '-ixv', '--occurrence=1', '-C', '/var/tmp/restorecvmwuhc8', 'backup-header', 'backup-header.hmac']
2017-10-25 23:20:08,855 [MainProcess restore._retrieve_backup_header:1132] qubesadmin.backup: Got backup header and hmac: backup-header, backup-header.hmac
2017-10-25 23:20:08,856 [MainProcess restore._verify_hmac:945] qubesadmin.backup: Verifying file backup-header
2017-10-25 23:20:13,437 [MainProcess restore._start_retrieval_process:882] qubesadmin.backup: Run command['tar', '-ixv', '--occurrence=1', '-C', '/var/tmp/restorecvmwuhc8', 'qubes.xml.000.enc']
2017-10-25 23:20:16,934 [MainProcess restore._start_inner_extraction_worker:1181] qubesadmin.backup: Starting extraction worker in /var/tmp/restorecvmwuhc8, file handlers map: {'qubes.xml': (functools.partial(<function BackupRestore._save_qubes_xml at 0x7d5095af2598>, '/var/tmp/restorecvmwuhc8/qubes-restored.xml'), None)}
2017-10-25 23:20:16,942 [ExtractWorker3-1 restore.__run__:532] qubesadmin.backup.extract: Started sending thread
2017-10-25 23:20:16,942 [ExtractWorker3-1 restore.__run__:533] qubesadmin.backup.extract: Moving to dir /var/tmp/restorecvmwuhc8
2017-10-25 23:20:16,943 [ExtractWorker3-1 restore.__run__:545] qubesadmin.backup.extract: Extracting file qubes.xml.000
2017-10-25 23:20:16,943 [ExtractWorker3-1 restore.__run__:589] qubesadmin.backup.extract: Running command ['tar', '-xvvO', '--use-compress-program=gzip', 'qubes.xml']
2017-10-25 23:20:16,960 [ExtractWorker3-1 restore.__run__:670] qubesadmin.backup.extract: Removing file qubes.xml.000
2017-10-25 23:20:16,964 [ExtractWorker3-1 restore.collect_tar_output:357] qubesadmin.backup.extract: tar2_stderr: 
2017-10-25 23:20:16,964 [ExtractWorker3-1 restore.__run__:682] qubesadmin.backup.extract: Finished extracting thread
Traceback (most recent call last):
  File "/usr/bin/qvm-backup-restore", line 9, in <module>
    load_entry_point('qubesadmin==4.0.9', 'console_scripts', 'qvm-backup-restore')()
  File "/usr/lib/python3.5/site-packages/qubesadmin/tools/qvm_backup_restore.py", line 219, in main
    appvm, passphrase)
  File "/usr/lib/python3.5/site-packages/qubesadmin/backup/restore.py", line 838, in __init__
    self.backup_app = self._process_qubes_xml()
  File "/usr/lib/python3.5/site-packages/qubesadmin/backup/restore.py", line 1237, in _process_qubes_xml
    backup_app = Core3Qubes(qubes_xml_path)
  File "/usr/lib/python3.5/site-packages/qubesadmin/backup/core3.py", line 60, in __init__
    super(Core3Qubes, self).__init__(store)
  File "/usr/lib/python3.5/site-packages/qubesadmin/backup/__init__.py", line 33, in __init__
    self.load()
  File "/usr/lib/python3.5/site-packages/qubesadmin/backup/core3.py", line 159, in load
    self.import_core3_vm(element)
  File "/usr/lib/python3.5/site-packages/qubesadmin/backup/core3.py", line 134, in import_core3_vm
    vm.label = self.labels[vm.properties.pop('label')]
KeyError: 'red'

General notes:

I'm not even sure why the KeyError is red, since the backup contained only a single VM - the default debian-8 template that has a black label...

Related issues:

None that I could find

Please see @jwoytek's remarks on #3219.

andrewdavidwong · Oct 27, 2017

Please see @jwoytek's remarks on #3219.

Thanks! I'll test the proposed fix later to see if it works. In my case it was Qubes 4.0 RC2 backups failing to be restored in the same RC2 machine, but it looks like the same issue and error.

Earlier today I had only very minor problems restring 3.2 backups in 4.0 RC2 - application shortcuts needed to be refreshed and virt_mode defaults to pv, but the restored VMs work perfectly. My current issue is that it's very hard to know how much free space I have left in 4.0, but that's a separate issue 😺

na-- · Oct 27, 2017

Thanks! I'll test the proposed fix later to see if it works. In my case it was Qubes 4.0 RC2 backups failing to be restored in the same RC2 machine, but it looks like the same issue and error.

Earlier today I had only very minor problems restring 3.2 backups in 4.0 RC2 - application shortcuts needed to be refreshed and virt_mode defaults to pv, but the restored VMs work perfectly. My current issue is that it's very hard to know how much free space I have left in 4.0, but that's a separate issue 😺

For the record, after implementing the quick fix, we were able to successfully restore rc1 to rc1 and rc2, and rc2 to rc2.

On Fri, Oct 27, 2017 at 5:11 AM na-- ***@***.***> wrote: Thank, I'll test the proposed fix later to see if it works. In my case it was Qubes 4.0 RC2 backups failing to be restored in the same RC2 machine, but it looks like the same issue and error. Earlier today I had only very minor problems restring 3.2 backups in 4.0 RC2 - application shortcuts needed to be refreshed and virt_mode defaults to pv, but the restored VMs work perfectly. My current issue is that it's very hard to know how much free space I have left in 4.0, but that's a separate issue 😺 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3211 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AOvthvPviu1U2RxcBAEsTK4U3noib14kks5swZ0jgaJpZM4QGm-K> .

-- Sent from my Commodore64

jwoytek · Oct 27, 2017

For the record, after implementing the quick fix, we were able to successfully restore rc1 to rc1 and rc2, and rc2 to rc2.

On Fri, Oct 27, 2017 at 5:11 AM na-- ***@***.***> wrote: Thank, I'll test the proposed fix later to see if it works. In my case it was Qubes 4.0 RC2 backups failing to be restored in the same RC2 machine, but it looks like the same issue and error. Earlier today I had only very minor problems restring 3.2 backups in 4.0 RC2 - application shortcuts needed to be refreshed and virt_mode defaults to pv, but the restored VMs work perfectly. My current issue is that it's very hard to know how much free space I have left in 4.0, but that's a separate issue 😺 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3211 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AOvthvPviu1U2RxcBAEsTK4U3noib14kks5swZ0jgaJpZM4QGm-K> .

-- Sent from my Commodore64

I encountered this error when trying --verify-only on a new backup, and unsuccessfully tried @jwoytek workaround like this:
"At line 134 in file /usr/lib/python3.5/site-packages/qubesadmin/backup/core3.py:"

vm.label = vm.properties.pop('label')
try:
    vm.label = self.labels[vm.label]
except KeyError:
    pass

This yields the same error. I then tried commenting-out the first "vm.label =" line and got:
File "/usr/lib64/python3.5/posixpath.py", line 139, in basename
i = p.rfind(sep) + 1
AttributeError: 'NoneType' object has no attribute 'rfind'

I successfully restored an R3.2 backup a few days ago, but currently I have to assume R4 qvm-backup is broken and that the new archive file is bad, so I am looking for a simple alternative.

tasket · Nov 1, 2017

I encountered this error when trying --verify-only on a new backup, and unsuccessfully tried @jwoytek workaround like this:
"At line 134 in file /usr/lib/python3.5/site-packages/qubesadmin/backup/core3.py:"

vm.label = vm.properties.pop('label')
try:
    vm.label = self.labels[vm.label]
except KeyError:
    pass

This yields the same error. I then tried commenting-out the first "vm.label =" line and got:
File "/usr/lib64/python3.5/posixpath.py", line 139, in basename
i = p.rfind(sep) + 1
AttributeError: 'NoneType' object has no attribute 'rfind'

I successfully restored an R3.2 backup a few days ago, but currently I have to assume R4 qvm-backup is broken and that the new archive file is bad, so I am looking for a simple alternative.

The backup archive produced by R4 qvm-backup is ok. The restore code is broken. I'm working on this right now.

marmarek · Nov 1, 2017

The backup archive produced by R4 qvm-backup is ok. The restore code is broken. I'm working on this right now.

See the commit I've just pushed (automatically linked above).

marmarek · Nov 1, 2017

See the commit I've just pushed (automatically linked above).

@marmarek Maybe there is a different issue related to file paths, as I'm still getting the AttributeError shown above (KeyError is gone).

tasket · Nov 1, 2017

@marmarek Maybe there is a different issue related to file paths, as I'm still getting the AttributeError shown above (KeyError is gone).

@marmarek I also see the @tasket attribute error when posixpath gets called with a NoneType path argument, which is contained somewhere in my 4.0rc2 backup set. A try/except{return ""} works, at least as a band-aid.

opposablebrain · Nov 10, 2017

@marmarek I also see the @tasket attribute error when posixpath gets called with a NoneType path argument, which is contained somewhere in my 4.0rc2 backup set. A try/except{return ""} works, at least as a band-aid.

Automated announcement from builder-github

The package qubes-core-admin-client_4.0.10-1+deb8u1 has been pushed to the r4.0 testing repository for the Debian jessie template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing jessie-testing, then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

QubesOS/qubes-issues

Join GitHub today

[R4.0 RC2] qvm-backup-restore or qvm-backup error #3211

Comments

na-- commented Oct 25, 2017

Qubes OS version:

Affected TemplateVMs:

Steps to reproduce the behavior:

Expected behavior:

Actual behavior:

General notes:

Related issues:

na-- changed the title from qvm-backup-restore error to [R4.0 RC2] qvm-backup-restore or qvm-backup error Oct 25, 2017

andrewdavidwong added bug C: core P: major labels Oct 26, 2017

andrewdavidwong added this to the Release 4.0 milestone Oct 26, 2017

andrewdavidwong referenced this issue Oct 27, 2017

This comment has been minimized.

andrewdavidwong commented Oct 27, 2017

This comment has been minimized.

na-- commented Oct 27, 2017 • edited Edited 1 time na-- edited Oct 27, 2017 (most recent)

This comment has been minimized.

jwoytek commented Oct 27, 2017

marmarek self-assigned this Oct 29, 2017

This comment has been minimized.

tasket commented Nov 1, 2017

This comment has been minimized.

marmarek commented Nov 1, 2017

This comment has been minimized.

marmarek commented Nov 1, 2017

This comment has been minimized.

tasket commented Nov 1, 2017

marmarek closed this in marmarek/qubes-core-admin-client@7f728e2 Nov 9, 2017

This comment has been minimized.

opposablebrain commented Nov 10, 2017 • edited Edited 1 time opposablebrain edited Nov 10, 2017 (most recent)

marmarek reopened this Nov 10, 2017

This comment has been minimized.

qubesos-bot commented Nov 14, 2017

qubesos-bot added the r4.0-jessie-cur-test label Nov 14, 2017

qubesos-bot referenced this issue in QubesOS/updates-status Nov 14, 2017

This comment has been minimized.

qubesos-bot commented Nov 14, 2017

qubesos-bot added the r4.0-stretch-cur-test label Nov 14, 2017

This comment has been minimized.

qubesos-bot commented Nov 14, 2017

qubesos-bot added the r4.0-fc24-cur-test label Nov 14, 2017

This comment has been minimized.

qubesos-bot commented Nov 14, 2017

qubesos-bot added the r4.0-fc25-cur-test label Nov 14, 2017

This comment has been minimized.

qubesos-bot commented Nov 14, 2017

qubesos-bot added the r4.0-fc26-cur-test label Nov 14, 2017

This comment has been minimized.

qubesos-bot commented Nov 14, 2017

qubesos-bot added the r4.0-dom0-cur-test label Nov 14, 2017

This comment has been minimized.

qubesos-bot commented Dec 11, 2017

qubesos-bot added r4.0-jessie-stable and removed r4.0-jessie-cur-test labels Dec 11, 2017

This comment has been minimized.

qubesos-bot commented Dec 11, 2017

qubesos-bot added r4.0-stretch-stable and removed r4.0-stretch-cur-test labels Dec 11, 2017

This comment has been minimized.

qubesos-bot commented Dec 11, 2017

qubesos-bot added r4.0-dom0-stable and removed r4.0-dom0-cur-test labels Dec 11, 2017

This comment has been minimized.

qubesos-bot commented Jan 5, 2018

qubesos-bot added r4.0-fc24-stable and removed r4.0-fc24-cur-test labels Jan 5, 2018

This comment has been minimized.

qubesos-bot commented Jan 5, 2018

qubesos-bot added r4.0-fc25-stable and removed r4.0-fc25-cur-test labels Jan 5, 2018

This comment has been minimized.

qubesos-bot commented Jan 5, 2018

qubesos-bot added r4.0-fc26-stable and removed r4.0-fc26-cur-test labels Jan 5, 2018

mossy-nw referenced this issue Jan 16, 2018

This comment has been minimized.

marmarek commented Feb 27, 2018

This comment has been minimized.

na-- commented Feb 27, 2018

na-- closed this Feb 27, 2018

na-- commented Oct 27, 2017 •

edited

Edited 1 time

na-- edited Oct 27, 2017 (most recent)

marmarek closed this in `marmarek/qubes-core-admin-client@7f728e2` Nov 9, 2017

opposablebrain commented Nov 10, 2017 •

edited

Edited 1 time

opposablebrain edited Nov 10, 2017 (most recent)