[Build 2024060404-4.3] openQA test fails in TC_00_Basic.test_203_udev_block_exclude_varlibqubes:starting

[marmarek]

Observation

openQA test in scenario qubesos-4.3-update-x86_64-system_tests_basic_vm_qrexec_gui@hw1 fails in
TC_00_Basic

qubes.tests.integ.basic/TC_00_Basic/test_203_udev_block_exclude_varlibqubes
Check if VM images are excluded from udev parsing - ... CRITICAL:qubes.tests.integ.basic.TC_00_Basic.test_203_udev_block_exclude_varlibqubes:starting
WARNING:vm.test-inst-appvm:Activating the test-inst-appvm VM
device-mapper: remove ioctl on origin-fd04:655927  failed: Device or resource busy
Command failed.
ERROR
ERROR:asyncio:Task exception was never retrieved
future: <Task finished name='Task-7112' coro=<QubesVM._domain_stopped_coro() done, defined at /usr/lib/python3.11/site-packages/qubes/vm/qubesvm.py:1308> exception=CalledProcessError(1, '/usr/lib/qubes/destroy-snapshot', None, None)>
Traceback (most recent call last):
  File "/usr/lib64/python3.11/unittest/case.py", line 57, in testPartExecutor
    yield
  File "/usr/lib64/python3.11/unittest/case.py", line 623, in run
    self._callTestMethod(testMethod)
  File "/usr/lib/python3.11/site-packages/qubes/tests/never_awaited.py", line 51, in wrapper
    func(*args, **kwargs)
  File "/usr/lib64/python3.11/unittest/case.py", line 579, in _callTestMethod
    if method() is not None:
       ^^^^^^^^
  File "/usr/lib64/python3.11/contextlib.py", line 81, in inner
    return func(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/qubes/tests/integ/basic.py", line 336, in test_203_udev_block_exclude_varlibqubes
    self.assertVolumesExcludedFromUdev(self.vm)
  File "/usr/lib/python3.11/site-packages/qubes/tests/integ/basic.py", line 269, in assertVolumesExcludedFromUdev
    self.loop.run_until_complete(vm.shutdown(wait=True))
  File "/usr/lib64/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/qubes/vm/qubesvm.py", line 1371, in shutdown
    await asyncio.wait_for(waiter, timeout=timeout)
  File "/usr/lib64/python3.11/asyncio/tasks.py", line 489, in wait_for
    return fut.result()
           ^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/qubes/vm/qubesvm.py", line 1319, in _domain_stopped_coro
    await self.fire_event_async('domain-stopped')
  File "/usr/lib/python3.11/site-packages/qubes/events.py", line 234, in fire_event_async
    effect = task.result()
             ^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/qubes/vm/qubesvm.py", line 1335, in on_domain_stopped
    await self.storage.stop()
  File "/usr/lib/python3.11/site-packages/qubes/storage/__init__.py", line 756, in stop
    await qubes.utils.void_coros_maybe(
  File "/usr/lib/python3.11/site-packages/qubes/utils.py", line 277, in void_coros_maybe
    task.result()  # re-raises exception if task failed
    ^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/qubes/storage/file.py", line 483, in stop
    await self._destroy_blockdev()
  File "/usr/lib/python3.11/site-packages/qubes/storage/file.py", line 273, in _destroy_blockdev
    await qubes.utils.run_program(DESTROY_SCRIPT,
  File "/usr/lib/python3.11/site-packages/qubes/utils.py", line 262, in run_program
    raise subprocess.CalledProcessError(p.returncode,
subprocess.CalledProcessError: Command '/usr/lib/qubes/destroy-snapshot' returned non-zero exit status 1.

Test suite description

Reproducible

Fails since (at least) Build 2024052808-4.3

Expected result

Last good: (unknown) (or more recent)

Further details

Always latest result in this scenario: latest

[rustybird]

Changing the dmsetup -- remove calls in /usr/lib/qubes/destroy-snapshot (there's 3 of them, 1 without the -- end-of-options separator) to dmsetup remove --retry -- might help to paper over whatever is happening here