/qubes-issues

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Standalone VM kernelopts not applied (PCI reset errors) #2579

Closed
tasket opened this Issue Jan 14, 2017 · 11 comments

Comments

Assignees
No one assigned
Labels
Projects
None yet
Milestone
  Release 3.2 updates
3 participants
@tasket @andrewdavidwong @marmarek
@tasket

tasket commented Jan 14, 2017

Qubes OS version (e.g., R3.2):

R3.2

Expected behavior:

VM starts with assigned USB controllers, using iommu=soft swiotlb=8192

Actual behavior:

Immediate 'internal error' unable to reset PCI device ... no FLR, PM reset or bus reset available.

Steps to reproduce the behavior:

Assign USB controllers to standalone appVM, disable memory balancing, add kernelopts iommu=soft swiotlb=8192 then start VM.

General notes:

The USB assignment seems to work fine for a template-based service VM such as sys-usb.

Related issues:
@tasket

This comment has been minimized.

Show comment
Hide comment
@tasket

tasket Jan 14, 2017

Also does not work if I try it the qubesctl way:
$ qubesctl top.enable qvm.bt-test
[ERROR] Could not cache minion ID: [Error 13] Permission denied: /etc/salt/minion.id
Failed to create directory path "/etc/salt/pki/minion" - Errno 13 permission denied: /etc/salt/pki

tasket commented Jan 14, 2017

Also does not work if I try it the qubesctl way:
$ qubesctl top.enable qvm.bt-test
[ERROR] Could not cache minion ID: [Error 13] Permission denied: /etc/salt/minion.id
Failed to create directory path "/etc/salt/pki/minion" - Errno 13 permission denied: /etc/salt/pki
@andrewdavidwong

This comment has been minimized.

Show comment
Hide comment
@andrewdavidwong

andrewdavidwong Jan 14, 2017

Member

Preliminary questions:

  • Are you sure that the device isn't already assigned to another VM?
  • Have you tried setting pci_strictreset to false?
Member

andrewdavidwong commented Jan 14, 2017

Preliminary questions:

  • Are you sure that the device isn't already assigned to another VM?
  • Have you tried setting pci_strictreset to false?

@andrewdavidwong andrewdavidwong added bug C: core labels Jan 14, 2017

@andrewdavidwong andrewdavidwong added this to the Release 3.2 updates milestone Jan 14, 2017

@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Jan 14, 2017

Member
Member

marmarek commented Jan 14, 2017

@tasket

This comment has been minimized.

Show comment
Hide comment
@tasket

tasket Jan 14, 2017

I'm positive its not already assigned (actually, there are 3 USB PCI devices which I think all belong to the same controller chip).

I just tried pci_strictreset=false and that had no effect, either with or without the other options.

tasket commented Jan 14, 2017

I'm positive its not already assigned (actually, there are 3 USB PCI devices which I think all belong to the same controller chip).

I just tried pci_strictreset=false and that had no effect, either with or without the other options.
@tasket

This comment has been minimized.

Show comment
Hide comment
@tasket

tasket Jan 14, 2017

@marmarek I tried running qubesctl as root also, and just got a much longer set of exceptions. Would you like to see them?

tasket commented Jan 14, 2017

@marmarek I tried running qubesctl as root also, and just got a much longer set of exceptions. Would you like to see them?
@tasket

This comment has been minimized.

Show comment
Hide comment
@tasket

tasket Jan 14, 2017 

$ sudo qubesctl top.enable qvm.f24-clone
[ERROR   ] An un-handled exception was caught by salt's global exception handler:
SaltRenderError: Could not find relpath for qvm.f24-clone.top
Traceback (most recent call last):
  File "/bin/qubesctl", line 91, in <module>
    salt_call()
  File "/usr/lib/python2.7/site-packages/salt/scripts.py", line 227, in salt_call
    client.run()
  File "/usr/lib/python2.7/site-packages/salt/cli/call.py", line 71, in run
    caller.run()
  File "/usr/lib/python2.7/site-packages/salt/cli/caller.py", line 236, in run
    ret = self.call()
  File "/usr/lib/python2.7/site-packages/salt/cli/caller.py", line 138, in call
    ret['return'] = func(*args, **kwargs)
  File "/srv/salt/_extensions/modules/topd.py", line 74, in enable
    return TopUtils(__opts__, **kwargs).enable(paths, saltenv)
  File "/srv/salt/_extensions/utils/toputils.py", line 600, in enable
    toppaths, unseen = self.prepare_paths(paths)
  File "/srv/salt/_extensions/utils/toputils.py", line 453, in prepare_paths
    toppath = self.toppath(path)
  File "/srv/salt/_extensions/utils/toputils.py", line 332, in toppath
    saltenv = saltenv or self.saltenv(path, saltenv)
  File "/srv/salt/_extensions/utils/pathutils.py", line 127, in saltenv
    relpath = self.relpath(path)
  File "/srv/salt/_extensions/utils/pathutils.py", line 485, in relpath
    return self.path(path, saltenv)
  File "/srv/salt/_extensions/utils/toputils.py", line 269, in path
    return super(TopUtils, self).path(path, saltenv, path_type=path_type)
  File "/srv/salt/_extensions/utils/pathutils.py", line 444, in path
    raise SaltRenderError('Could not find relpath for {0}'.format(path))
SaltRenderError: Could not find relpath for qvm.f24-clone.top
Traceback (most recent call last):
  File "/bin/qubesctl", line 91, in <module>
    salt_call()
  File "/usr/lib/python2.7/site-packages/salt/scripts.py", line 227, in salt_call
    client.run()
  File "/usr/lib/python2.7/site-packages/salt/cli/call.py", line 71, in run
    caller.run()
  File "/usr/lib/python2.7/site-packages/salt/cli/caller.py", line 236, in run
    ret = self.call()
  File "/usr/lib/python2.7/site-packages/salt/cli/caller.py", line 138, in call
    ret['return'] = func(*args, **kwargs)
  File "/srv/salt/_extensions/modules/topd.py", line 74, in enable
    return TopUtils(__opts__, **kwargs).enable(paths, saltenv)
  File "/srv/salt/_extensions/utils/toputils.py", line 600, in enable
    toppaths, unseen = self.prepare_paths(paths)
  File "/srv/salt/_extensions/utils/toputils.py", line 453, in prepare_paths
    toppath = self.toppath(path)
  File "/srv/salt/_extensions/utils/toputils.py", line 332, in toppath
    saltenv = saltenv or self.saltenv(path, saltenv)
  File "/srv/salt/_extensions/utils/pathutils.py", line 127, in saltenv
    relpath = self.relpath(path)
  File "/srv/salt/_extensions/utils/pathutils.py", line 485, in relpath
    return self.path(path, saltenv)
  File "/srv/salt/_extensions/utils/toputils.py", line 269, in path
    return super(TopUtils, self).path(path, saltenv, path_type=path_type)
  File "/srv/salt/_extensions/utils/pathutils.py", line 444, in path
    raise SaltRenderError('Could not find relpath for {0}'.format(path))
salt.exceptions.SaltRenderError: Could not find relpath for qvm.f24-clone.top
DOM0 configuration failed, not continuing

tasket commented Jan 14, 2017 

$ sudo qubesctl top.enable qvm.f24-clone
[ERROR   ] An un-handled exception was caught by salt's global exception handler:
SaltRenderError: Could not find relpath for qvm.f24-clone.top
Traceback (most recent call last):
  File "/bin/qubesctl", line 91, in <module>
    salt_call()
  File "/usr/lib/python2.7/site-packages/salt/scripts.py", line 227, in salt_call
    client.run()
  File "/usr/lib/python2.7/site-packages/salt/cli/call.py", line 71, in run
    caller.run()
  File "/usr/lib/python2.7/site-packages/salt/cli/caller.py", line 236, in run
    ret = self.call()
  File "/usr/lib/python2.7/site-packages/salt/cli/caller.py", line 138, in call
    ret['return'] = func(*args, **kwargs)
  File "/srv/salt/_extensions/modules/topd.py", line 74, in enable
    return TopUtils(__opts__, **kwargs).enable(paths, saltenv)
  File "/srv/salt/_extensions/utils/toputils.py", line 600, in enable
    toppaths, unseen = self.prepare_paths(paths)
  File "/srv/salt/_extensions/utils/toputils.py", line 453, in prepare_paths
    toppath = self.toppath(path)
  File "/srv/salt/_extensions/utils/toputils.py", line 332, in toppath
    saltenv = saltenv or self.saltenv(path, saltenv)
  File "/srv/salt/_extensions/utils/pathutils.py", line 127, in saltenv
    relpath = self.relpath(path)
  File "/srv/salt/_extensions/utils/pathutils.py", line 485, in relpath
    return self.path(path, saltenv)
  File "/srv/salt/_extensions/utils/toputils.py", line 269, in path
    return super(TopUtils, self).path(path, saltenv, path_type=path_type)
  File "/srv/salt/_extensions/utils/pathutils.py", line 444, in path
    raise SaltRenderError('Could not find relpath for {0}'.format(path))
SaltRenderError: Could not find relpath for qvm.f24-clone.top
Traceback (most recent call last):
  File "/bin/qubesctl", line 91, in <module>
    salt_call()
  File "/usr/lib/python2.7/site-packages/salt/scripts.py", line 227, in salt_call
    client.run()
  File "/usr/lib/python2.7/site-packages/salt/cli/call.py", line 71, in run
    caller.run()
  File "/usr/lib/python2.7/site-packages/salt/cli/caller.py", line 236, in run
    ret = self.call()
  File "/usr/lib/python2.7/site-packages/salt/cli/caller.py", line 138, in call
    ret['return'] = func(*args, **kwargs)
  File "/srv/salt/_extensions/modules/topd.py", line 74, in enable
    return TopUtils(__opts__, **kwargs).enable(paths, saltenv)
  File "/srv/salt/_extensions/utils/toputils.py", line 600, in enable
    toppaths, unseen = self.prepare_paths(paths)
  File "/srv/salt/_extensions/utils/toputils.py", line 453, in prepare_paths
    toppath = self.toppath(path)
  File "/srv/salt/_extensions/utils/toputils.py", line 332, in toppath
    saltenv = saltenv or self.saltenv(path, saltenv)
  File "/srv/salt/_extensions/utils/pathutils.py", line 127, in saltenv
    relpath = self.relpath(path)
  File "/srv/salt/_extensions/utils/pathutils.py", line 485, in relpath
    return self.path(path, saltenv)
  File "/srv/salt/_extensions/utils/toputils.py", line 269, in path
    return super(TopUtils, self).path(path, saltenv, path_type=path_type)
  File "/srv/salt/_extensions/utils/pathutils.py", line 444, in path
    raise SaltRenderError('Could not find relpath for {0}'.format(path))
salt.exceptions.SaltRenderError: Could not find relpath for qvm.f24-clone.top
DOM0 configuration failed, not continuing
@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Jan 14, 2017

Member

pci_strictreset is exactly to suppress the message you've got. Does it fail with the same message, or maybe some else?

As for qubesctl - it looks like some problem with your salt configuration - it can't find qvm.f24-clone.top file.
Member

marmarek commented Jan 14, 2017

pci_strictreset is exactly to suppress the message you've got. Does it fail with the same message, or maybe some else?

As for qubesctl - it looks like some problem with your salt configuration - it can't find qvm.f24-clone.top file.
@tasket

This comment has been minimized.

Show comment
Hide comment
@tasket

tasket Jan 14, 2017

It fails with the same error. FWIW, I do not need that option with sys-usb.

I don't know what the issue is with qubesctl-- Is it only usable with sys-usb? Or with service VMs?

tasket commented Jan 14, 2017

It fails with the same error. FWIW, I do not need that option with sys-usb.

I don't know what the issue is with qubesctl-- Is it only usable with sys-usb? Or with service VMs?
@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Jan 15, 2017

Member

It fails with the same error. FWIW, I do not need that option with sys-usb.

For sys-usb, pci_strictreset is set to false using salt. It shouldn't be possible to get an error "unable to reset PCI device" if pci_strictreset is set to false, so make sure you really set it.

I don't know what the issue is with qubesctl-- Is it only usable with sys-usb? Or with service VMs?

qubesctl is just a frontend for salt, so should work with any valid salt configuration. And any VM can be configured this way. Anyway this is offtopic here, better to discuss this on qubes-users.
Member

marmarek commented Jan 15, 2017

It fails with the same error. FWIW, I do not need that option with sys-usb.

For sys-usb, pci_strictreset is set to false using salt. It shouldn't be possible to get an error "unable to reset PCI device" if pci_strictreset is set to false, so make sure you really set it.

I don't know what the issue is with qubesctl-- Is it only usable with sys-usb? Or with service VMs?

qubesctl is just a frontend for salt, so should work with any valid salt configuration. And any VM can be configured this way. Anyway this is offtopic here, better to discuss this on qubes-users.
@tasket

This comment has been minimized.

Show comment
Hide comment
@tasket

tasket Jan 15, 2017

$ qvm-prefs -l f24-clone
nopat iommu=soft swiotlb=8192 pci_strictreset=false

I created a discussion thread about qubesctl here:
https://groups.google.com/d/msgid/qubes-users/a2d335ea-eace-7b9d-4be1-1a713401ec50%40openmailbox.org

tasket commented Jan 15, 2017

$ qvm-prefs -l f24-clone
nopat iommu=soft swiotlb=8192 pci_strictreset=false

I created a discussion thread about qubesctl here:
https://groups.google.com/d/msgid/qubes-users/a2d335ea-eace-7b9d-4be1-1a713401ec50%40openmailbox.org
@tasket

This comment has been minimized.

Show comment
Hide comment
@tasket

tasket Jan 16, 2017

As shown in the prior post, I mis-handled the pci_strictreset setting as a kernel option. (Oddly, this setting does not carry over when cloning a VM.) Setting it properly allows standalone VM to use controllers; Closing.

tasket commented Jan 16, 2017

As shown in the prior post, I mis-handled the pci_strictreset setting as a kernel option. (Oddly, this setting does not carry over when cloning a VM.) Setting it properly allows standalone VM to use controllers; Closing.

@tasket tasket closed this Jan 16, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment