[BUG] CRC does not allow passing credentials to virsh when authentication is set up

[concaf]

General information

• OS: Linux
• Hypervisor: KVM
• Did you run crc setup before starting it (Yes/No)? Yes
• Running CRC on: Desktop

CRC version

CodeReady Containers version: 1.19.0+94b0362
OpenShift version: 4.6.3 (embedded in executable)

CRC status

DEBU CodeReady Containers version: 1.19.0+94b0362 
DEBU OpenShift version: 4.6.3 (embedded in executable) 
Checking file: /home/server1/.crc/machines/crc/.crc-exist
Machine 'crc' does not exist. Use 'crc start' to create it

CRC config

Host Operating System

NAME="CentOS Linux"
VERSION="8"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="CentOS Linux 8"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:8"
HOME_URL="https://centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"
CENTOS_MANTISBT_PROJECT="CentOS-8"
CENTOS_MANTISBT_PROJECT_VERSION="8"

Steps to reproduce

I have oVirt installed on my machine and I believe it has set up SASL authentication for virsh.
When I run sudo virsh list, I need to provide authentication -

sudo virsh list
Please enter your authentication name: 
Please enter your password: 

or provide an authfile to virsh as such -

$ sudo virsh -c qemu:///system?authfile=/etc/ovirt-hosted-engine/virsh_auth.conf net-list
 Name             State    Autostart   Persistent
---------------------------------------------------
 ;vdsmdummy;      active   no          no
 vdsm-ovirtmgmt   active   yes         yes

Now, the problem is that CRC gives me no such option to pass credentials to virsh and hence it fails.

1. Install oVirt or set up authentication for virsh
2. Run crc setup

Expected

crc setup should work.

Actual

$ crc setup --log-level debug

level=debug msg="CodeReady Containers version: 1.19.0+94b0362\n"
level=debug msg="OpenShift version: 4.6.3 (embedded in executable)\n"
level=debug msg="Couldn't set key PLATFORM_ID, no corresponding struct field found"
level=debug msg="Couldn't set key CPE_NAME, no corresponding struct field found"
level=debug msg="Couldn't set key CENTOS_MANTISBT_PROJECT, no corresponding struct field found"
level=debug msg="Couldn't set key CENTOS_MANTISBT_PROJECT_VERSION, no corresponding struct field found"
level=info msg="Checking if podman remote executable is cached"
level=debug msg="Currently podman remote is not supported"
level=info msg="Checking if goodhosts executable is cached"
level=debug msg="goodhost executable already cached"
level=info msg="Checking if CRC bundle is cached in '$HOME/.crc'"
level=info msg="Checking minimum RAM requirements"
level=debug msg="Total memory of system is 20713512960 bytes"
level=info msg="Checking if running as non-root"
level=info msg="Checking if Virtualization is enabled"
level=debug msg="Checking if the vmx/svm flags are present in /proc/cpuinfo"
level=debug msg="CPU virtualization flags are good"
level=info msg="Checking if KVM is enabled"
level=debug msg="Checking if /dev/kvm exists"
level=debug msg="/dev/kvm was found"
level=info msg="Checking if libvirt is installed"
level=debug msg="Checking if 'virsh' is available"
level=debug msg="'virsh' was found in /usr/bin/virsh"
level=debug msg="Checking 'virsh capabilities' for libvirtd/qemu availability"
level=debug msg="Running 'virsh capabilities'"
level=debug msg="Found x86_64 hypervisor with 'hvm' capabilities"
level=info msg="Checking if user is part of libvirt group"
level=debug msg="Checking if current user is part of the libvirt group"
level=debug msg="Running '/usr/bin/groups server1'"
level=debug msg="Current user is already in the libvirt group"
level=info msg="Checking if libvirt daemon is running"
level=debug msg="Checking if libvirtd service is running"
level=debug msg="Running 'systemctl status virtqemud.socket'"
level=debug msg="Command failed: exit status 3"
level=debug msg="stdout: * virtqemud.socket - Libvirt qemu local socket\n   Loaded: loaded (/usr/lib/systemd/system/virtqemud.socket; disabled; vendor preset: disabled)\n   Active: inactive (dead)\n   Listen: /run/libvirt/virtqemud-sock (Stream)\n"
level=debug msg="stderr: "
level=debug msg="virtqemud.socket is not running"
level=debug msg="Running 'systemctl status libvirtd.socket'"
level=debug msg="libvirtd.socket is running"
level=info msg="Checking if a supported libvirt version is installed"
level=debug msg="Checking if libvirt version is >=3.4.0"
level=debug msg="Running 'virsh -v'"
level=info msg="Checking if crc-driver-libvirt is installed"
level=debug msg="Checking if crc-driver-libvirt is installed"
level=debug msg="Running '/home/server1/.crc/bin/crc-driver-libvirt version'"
level=debug msg="Found crc-driver-libvirt version 0.12.12"
level=debug msg="crc-driver-libvirt is already installed"
level=info msg="Checking for obsolete crc-driver-libvirt"
level=debug msg="Checking if an older libvirt driver crc-driver-libvirt is installed"
level=debug msg="No older crc-driver-libvirt installation found"
level=info msg="Checking if libvirt 'crc' network is available"
level=debug msg="Checking if libvirt 'crc' network exists"
level=debug msg="Running 'virsh --connect qemu:///system net-info crc'"
level=debug msg="Command failed: exit status 1"
level=debug msg="stdout: Please enter your authentication name: Please enter your password: "
level=debug msg="stderr: error: failed to connect to the hypervisor\nerror: authentication failed: Failed to start SASL negotiation: -1 (SASL(-1): generic failure: All-whitespace username.)\n"
level=debug msg="Libvirt network crc not found"
level=info msg="Setting up libvirt 'crc' network"
level=debug msg="Creating libvirt 'crc' network"
level=debug msg="Running 'virsh --connect qemu:///system net-destroy crc'"
level=debug msg="Command failed: exit status 1"
level=debug msg="stdout: Please enter your authentication name: Please enter your password: "
level=debug msg="stderr: error: failed to connect to the hypervisor\nerror: authentication failed: Failed to start SASL negotiation: -1 (SASL(-1): generic failure: All-whitespace username.)\n"
level=debug msg="Running 'virsh --connect qemu:///system net-undefine crc'"
level=debug msg="Command failed: exit status 1"
level=debug msg="stdout: Please enter your authentication name: Please enter your password: "
level=debug msg="stderr: error: failed to connect to the hypervisor\nerror: authentication failed: Failed to start SASL negotiation: -1 (SASL(-1): generic failure: All-whitespace username.)\n"
level=debug msg="exit status 1 : error: failed to connect to the hypervisor\nerror: authentication failed: authentication failed\n"
Failed to create libvirt 'crc' network

Logs

Already posted above

[cfergeau]

It's correct that crc provides no way of overriding qemu:///system, it expects that by adding the current user to the libvirt group, then access to qemu:///system will be passwordless.
Regarding your situation, reading https://libvirt.org/auth.html#Auth_client_config it seems setting LIBVIRT_AUTH_FILE=/etc/ovirt-hosted-engine/virsh_auth.conf could avoid this issue?

[concaf]

@cfergeau thanks for the prompt reply.
So the user is already a part of the libvirt group -

$ groups
server1 wheel libvirt

but that doesn't help. However, your other suggestion of setting LIBVIRT_AUTH_FILE worked for me.

Unfortunately, it works fine for crc setup but then fails again for crc run, see logs here - https://paste.centos.org/view/7ca20233

[praveenkumar]

Did you exported LIBVIRT_AUTH_FILE=<> or you used LIBVIRT_AUTH_FILE=<> crc setup? export should work for both the commands and also there are other way listed on same doc.

[cfergeau]

setup uses virsh, which uses virConnectOpenAuth(uri, virConnectAuthPtrDefault, 0) to connect to libvirt, while start goes through the libvirt machine driver, which uses libvirt-go directly, which uses virConnectOpen(). Maybe the different behaviour comes from here.

[praveenkumar]

@cfergeau In that case we should create/move this issue to driver side so we can able to create connection accordingly?

[cfergeau]

@cfergeau In that case we should create/move this issue to driver side so we can able to create connection accordingly?

If my theory is correct, at this point this is just an unverified guess :)

[concaf]

Yes, I exported LIBVIRT_AUTH_FILE - it worked for crc setup but not for crc start. @cfergeau theory seems correct :)
One would expect all crc subcommands to respect a given env var.

[cfergeau]

I filed https://gitlab.com/libvirt/libvirt-go/-/merge_requests/24 , when this is merged we'll have to figure out how to make use of this in the libvirt driver (where we prefer to keep a fixed libvirt-go version).

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.