Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow specifying a username in service config #109

Merged
merged 6 commits into from
Oct 6, 2023

Conversation

DemiMarie
Copy link
Contributor

This also dramatically improves the configuration parser. Configuration files now use a strict subset of TOML rather than an ad-hoc format with no validation.

Fixes: QubesOS/qubes-issues#6354
Fixes: QubesOS/qubes-issues#8153

@marmarek
Copy link
Member

Fuzzer makefile needs an update.

@qubesos-bot
Copy link

qubesos-bot commented Apr 28, 2023

OpenQA test summary

Complete test suite and dependencies: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.2&build=2023092018-4.2&flavor=pull-requests

New failures, excluding unstable

Compared to: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.2&build=2023071104-4.2&flavor=update

  • system_tests_manager

    • QubeManagerTest: test_226_global_settings (failure)
      AssertionError: Expected 'check_call' to be called once. Called 0 t...

    • QubeManagerTest: test_231_template_manager (failure)
      AssertionError: Expected 'check_call' to be called once. Called 0 t...

  • system_tests_vm_qrexec_gui_pipewire

  • system_tests_pvgrub_salt_storage

    • TC_41_HVMGrub_fedora-38-xfce: test_010_template_based_vm (error)
      qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  • system_tests_splitgpg

Failed tests

6 failures
  • system_tests_manager

    • QubeManagerTest: test_226_global_settings (failure)
      AssertionError: Expected 'check_call' to be called once. Called 0 t...

    • QubeManagerTest: test_231_template_manager (failure)
      AssertionError: Expected 'check_call' to be called once. Called 0 t...

  • system_tests_vm_qrexec_gui_pipewire

  • system_tests_pvgrub_salt_storage

    • [unstable] TC_41_HVMGrub_fedora-38-xfce: test_000_standalone_vm (error)
      qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...

    • TC_41_HVMGrub_fedora-38-xfce: test_010_template_based_vm (error)
      qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...

  • system_tests_splitgpg

Fixed failures

Compared to: https://openqa.qubes-os.org/tests/77326#dependencies

23 fixed
  • system_tests_whonix

  • system_tests_manager

    • VMSettingsTest: test_26_default_dispvm (failure)
      self.assertTrue(self.vm.property_... AssertionError: False is not true
  • system_tests_guivm_gui_interactive

    • update_templates: Failed (test died)
      # Test died: command 'curl --form upload=@template-fedora-38-xfce-v...
  • system_tests_qrexec

  • system_tests_network_ipv6

    • VmIPv6Networking_fedora-38: test_040_inter_vm (error)
      qubes.exc.QubesMemoryError: Not enough memory to start domain 'test...

    • VmIPv6Networking_fedora-38: test_203_fake_ip_inter_vm_allow (error)
      qubes.exc.QubesMemoryError: Not enough memory to start domain 'test...

  • system_tests_dispvm

  • system_tests_network

  • system_tests_splitgpg

Unstable tests

  • system_tests_whonix

    whonix_torbrowser/ (1/5 times with errors)
    whonix_torbrowser/ (1/5 times with errors)
    whonix_torbrowser/Failed (1/5 times with errors)
    • job 80225 # Test died: no candidate needle with tag(s) 'anon-whonix-tor-brows...
  • system_tests_basic_vm_qrexec_gui

    TC_00_Basic/test_120_start_standalone_with_cdrom_dom0 (1/5 times with errors)
    • job 80971 AssertionError: 1 != 0 : b'Timeout waiting for dom0:loop4 device to...
    TC_20_AudioVM_Pulse_debian-12-xfce/test_221_audio_rec_muted_pulseaudio (1/5 times with errors)
    • job 80238 AssertionError: source-output for VM test-inst-vm1 not found
    TC_20_AudioVM_PipeWire_fedora-38-xfce/test_226_audio_playback_pipewire (1/5 times with errors)
    • job 80839 AssertionError: Command 'pw-play --format=f32 --rate=44100 --channe...
    TC_20_AudioVM_PipeWire_fedora-38-xfce/test_227_audio_rec_muted_pipewire (1/5 times with errors)
    • job 80839 subprocess.CalledProcessError: Command 'pkill pw-record' returned n...
    TC_20_AudioVM_PipeWire_fedora-38-xfce/test_228_audio_rec_unmuted_pipewire (1/5 times with errors)
    • job 80839 subprocess.CalledProcessError: Command 'pkill pw-record' returned n...
  • system_tests_usbproxy

    TC_20_USBProxy_core3_whonix-workstation-17/test_070_attach_not_installed_front (1/5 times with errors)
    • job 81882 qubesusbproxy.core3ext.QubesUSBException: Device attach failed: 202...
  • system_tests_qrexec

    TC_00_Qrexec_whonix-workstation-17/test_081_qrexec_service_argument_allow_specific (1/5 times with errors)
    • job 80852 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
    TC_00_Qrexec_whonix-gateway-17/test_090_qrexec_service_socket_dom0 (1/5 times with errors)
    • job 80251 AssertionError: service timeout, probably EOF wasn't transferred to...
    TC_00_Qrexec_whonix-workstation-17/test_090_qrexec_service_socket_dom0 (1/5 times with errors)
    • job 80251 AssertionError: service timeout, probably EOF wasn't transferred to...
  • system_tests_network_ipv6

    VmIPv6Networking_fedora-38-xfce/test_540_ipv6_inter_vm (1/5 times with errors)
    • job 80991 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  • system_tests_dispvm

    TC_20_DispVM_whonix-gateway-17/test_010_simple_dvm_run (1/5 times with errors)
    TC_20_DispVM_whonix-workstation-17/test_010_simple_dvm_run (1/5 times with errors)
    • job 81874 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^... AssertionError
    TC_20_DispVM_whonix-gateway-17/test_020_gui_app (1/5 times with errors)
    TC_20_DispVM_fedora-38-xfce/test_030_edit_file (2/5 times with errors)
    • job 81874 AssertionError: Timeout while waiting for disp[0-9]* window to show
    • job 82011 AssertionError: Timeout while waiting for disp[0-9]* window to show
    TC_20_DispVM_whonix-gateway-17/test_030_edit_file (1/5 times with errors)
    TC_20_DispVM_whonix-workstation-17/test_030_edit_file (3/5 times with errors)
    • job 80844 AssertionError: Timeout while waiting for disp[0-9]* window to show
    • job 81874 AssertionError: Timeout while waiting for disp[0-9]* window to show
    • job 82011 AssertionError: Timeout while waiting for disp[0-9]* window to show
    TC_20_DispVM_fedora-38-xfce/test_100_open_in_dispvm (1/5 times with errors)
    • job 81874 AssertionError: Timeout while waiting for disp[0-9]* window to show
    TC_20_DispVM_whonix-gateway-17/test_100_open_in_dispvm (1/5 times with errors)
    TC_20_DispVM_whonix-workstation-17/test_100_open_in_dispvm (3/5 times with errors)
    • job 80243 AssertionError: Timeout while waiting for disp[0-9]* window to show
    • job 81874 AssertionError: Timeout while waiting for disp[0-9]* window to show
    • job 82011 AssertionError: Timeout while waiting for disp[0-9]* window to show
  • system_tests_basic_vm_qrexec_gui@hw1

    TC_00_Basic/test_120_start_standalone_with_cdrom_dom0 (1/5 times with errors)
    • job 80971 AssertionError: 1 != 0 : b'Timeout waiting for dom0:loop4 device to...
    TC_20_AudioVM_Pulse_debian-12-xfce/test_221_audio_rec_muted_pulseaudio (1/5 times with errors)
    • job 80238 AssertionError: source-output for VM test-inst-vm1 not found
    TC_20_AudioVM_PipeWire_fedora-38-xfce/test_226_audio_playback_pipewire (1/5 times with errors)
    • job 80839 AssertionError: Command 'pw-play --format=f32 --rate=44100 --channe...
    TC_20_AudioVM_PipeWire_fedora-38-xfce/test_227_audio_rec_muted_pipewire (1/5 times with errors)
    • job 80839 subprocess.CalledProcessError: Command 'pkill pw-record' returned n...
    TC_20_AudioVM_PipeWire_fedora-38-xfce/test_228_audio_rec_unmuted_pipewire (1/5 times with errors)
    • job 80839 subprocess.CalledProcessError: Command 'pkill pw-record' returned n...
  • system_tests_network

    VmNetworking_debian-12-xfce/test_212_custom_ip_firewall (1/5 times with errors)
    • job 80247 raise exceptions.TimeoutError() from exc... TimeoutError
  • system_tests_pvgrub_salt_storage

    TC_41_HVMGrub_fedora-38-xfce/test_000_standalone_vm (4/5 times with errors)
    • job 80851 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
    • job 80858 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
    • job 81879 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
    • job 82004 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  • system_tests_basic_vm_qrexec_gui_ext4

    TC_30_Gui_daemon/test_000_clipboard (1/5 times with errors)
    • job 80996 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
    TC_00_Basic/test_120_start_standalone_with_cdrom_dom0 (1/5 times with errors)
    • job 81994 AssertionError: 1 != 0 : b'Timeout waiting for dom0:loop12 device t...
    TC_20_AudioVM_PipeWire_debian-12-xfce-pool/test_226_audio_playback_pipewire (1/5 times with errors)
    • job 81870 AssertionError: Command 'pw-play --format=f32 --rate=44100 --channe...
    TC_20_AudioVM_PipeWire_fedora-38-xfce-pool/test_226_audio_playback_pipewire (1/5 times with errors)
    • job 80855 AssertionError: Command 'pw-play --format=f32 --rate=44100 --channe...
    TC_20_AudioVM_PipeWire_debian-12-xfce-pool/test_227_audio_rec_muted_pipewire (1/5 times with errors)
    • job 80855 subprocess.CalledProcessError: Command 'pkill pw-record' returned n...
    TC_20_AudioVM_PipeWire_fedora-38-xfce-pool/test_227_audio_rec_muted_pipewire (1/5 times with errors)
    • job 80855 subprocess.CalledProcessError: Command 'pkill pw-record' returned n...
    TC_20_AudioVM_PipeWire_fedora-38-xfce-pool/test_228_audio_rec_unmuted_pipewire (1/5 times with errors)
    • job 80855 subprocess.CalledProcessError: Command 'pkill pw-record' returned n...
  • system_tests_basic_vm_qrexec_gui_xfs

    TC_20_AudioVM_PipeWire_fedora-38-xfce-pool/test_226_audio_playback_pipewire (1/5 times with errors)
    • job 80856 AssertionError: Command 'pw-play --format=f32 --rate=44100 --channe...
    TC_20_AudioVM_PipeWire_fedora-38-xfce-pool/test_227_audio_rec_muted_pipewire (1/5 times with errors)
    • job 80856 subprocess.CalledProcessError: Command 'pkill pw-record' returned n...
    TC_20_AudioVM_PipeWire_fedora-38-xfce-pool/test_228_audio_rec_unmuted_pipewire (1/5 times with errors)
    • job 80856 subprocess.CalledProcessError: Command 'pkill pw-record' returned n...
    TC_20_NonAudio_whonix-gateway-17-pool/test_300_bug_1028_gui_memory_pinning (1/5 times with errors)
    • job 81871 AssertionError: Dom0 window doesn't match VM window content, saved ...
  • system_tests_basic_vm_qrexec_gui_zfs

    TC_00_Basic/test_120_start_standalone_with_cdrom_dom0 (2/5 times with errors)
    • job 80857 AssertionError: 1 != 0 : b'Timeout waiting for dom0:loop4 device to...
    • job 81872 AssertionError: 1 != 0 : b'Timeout waiting for dom0:loop4 device to...
    TC_06_AppVM_debian-12-xfce-pool/test_121_start_standalone_with_cdrom_vm (1/5 times with errors)
    • job 80226 AssertionError: 1 != 0 : b"Not enough memory to start domain 'test-...
    TC_00_Basic/test_202_udev_block_exclude_default (1/5 times with errors)
    • job 81872 raise exceptions.TimeoutError() from exc... TimeoutError
    TC_20_AudioVM_Pulse_debian-12-xfce-pool/test_222_audio_rec_unmuted_pulseaudio (1/5 times with errors)
    • job 81872 AssertionError: frequency 3379.6121459556007 not in specified range...
    TC_20_AudioVM_PipeWire_fedora-38-xfce-pool/test_226_audio_playback_pipewire (1/5 times with errors)
    • job 80857 AssertionError: Command 'pw-play --format=f32 --rate=44100 --channe...
    TC_20_AudioVM_PipeWire_fedora-38-xfce-pool/test_227_audio_rec_muted_pipewire (1/5 times with errors)
    • job 80857 subprocess.CalledProcessError: Command 'pkill pw-record' returned n...
    TC_20_AudioVM_PipeWire_fedora-38-xfce-pool/test_228_audio_rec_unmuted_pipewire (1/5 times with errors)
    • job 80857 subprocess.CalledProcessError: Command 'pkill pw-record' returned n...
  • system_tests_basic_vm_qrexec_gui_btrfs

    TC_20_AudioVM_Pulse_debian-12-xfce-pool/test_220_audio_play_pulseaudio (1/5 times with errors)
    • job 81808 AssertionError: Command 'paplay --format=float32le --rate=44100 --c...
    TC_20_AudioVM_PipeWire_debian-12-xfce-pool/test_226_audio_playback_pipewire (1/5 times with errors)
    • job 81995 AssertionError: Command 'pw-play --format=f32 --rate=44100 --channe...
    TC_20_AudioVM_PipeWire_fedora-38-xfce-pool/test_226_audio_playback_pipewire (1/5 times with errors)
    • job 80840 AssertionError: Command 'pw-play --format=f32 --rate=44100 --channe...
    TC_20_AudioVM_PipeWire_fedora-38-xfce-pool/test_227_audio_rec_muted_pipewire (1/5 times with errors)
    • job 80840 subprocess.CalledProcessError: Command 'pkill pw-record' returned n...
    TC_20_AudioVM_PipeWire_fedora-38-xfce-pool/test_228_audio_rec_unmuted_pipewire (1/5 times with errors)
    • job 80840 subprocess.CalledProcessError: Command 'pkill pw-record' returned n...
  • system_tests_whonix@hw1

    whonix_torbrowser/ (1/5 times with errors)
    whonix_torbrowser/ (1/5 times with errors)
    whonix_torbrowser/Failed (1/5 times with errors)
    • job 80225 # Test died: no candidate needle with tag(s) 'anon-whonix-tor-brows...

@marmarek
Copy link
Member

PipelineRetry

@marmarek
Copy link
Member

Fuzzer makefile needs an update.

This is still the case: fuzzer fails to link:

/usr/bin/ld: libqrexec-exec.o: in function `load_service_config':
/builds/QubesOS/qubes-core-qrexec/fuzz/../libqrexec/exec.c:273: undefined reference to `qubes_toml_config_parse'

@marmarek
Copy link
Member

I may be missing something, but does it actually change the user used for the call? IIUC it only uses the user setting to send it to the qubes.WaitForSession service, which BTW doesn't even read that value anymore.
The actual user is set in parse_qubes_rpc_command() and used in execute_qubes_rpc_command() -> execute_parsed_qubes_rpc_command()

@codecov
Copy link

codecov bot commented May 28, 2023

Codecov Report

Merging #109 (0fa8ecc) into main (5f68a98) will not change coverage.
Report is 1 commits behind head on main.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main     #109   +/-   ##
=======================================
  Coverage   85.09%   85.09%           
=======================================
  Files          31       31           
  Lines        5174     5174           
=======================================
  Hits         4403     4403           
  Misses        771      771           
Files Coverage Δ
qrexec/tests/socket/agent.py 98.88% <100.00%> (ø)

@marmarek
Copy link
Member

marmarek commented Jul 2, 2023

I may be missing something, but does it actually change the user used for the call? IIUC it only uses the user setting to send it to the qubes.WaitForSession service, which BTW doesn't even read that value anymore.
The actual user is set in parse_qubes_rpc_command() and used in execute_qubes_rpc_command() -> execute_parsed_qubes_rpc_command()

This still applies.

@DemiMarie
Copy link
Contributor Author

I may be missing something, but does it actually change the user used for the call? IIUC it only uses the user setting to send it to the qubes.WaitForSession service, which BTW doesn't even read that value anymore.
The actual user is set in parse_qubes_rpc_command() and used in execute_qubes_rpc_command() -> execute_parsed_qubes_rpc_command()

This still applies.

handle_server_exec_request_init() calls load_service_config() and (if it picks a user) overwrites the user set by parse_qubes_rpc_command().

@marmarek
Copy link
Member

Every single test failed on setup with qrexec-client.c:497:wait_for_vchan_client_with_timeout: vchan connection timeout. I don't see anything obvious in VM's journal, but it might be in VM's .xsession-errors which wasn't saved.

@DemiMarie
Copy link
Contributor Author

Every single test failed on setup with qrexec-client.c:497:wait_for_vchan_client_with_timeout: vchan connection timeout. I don't see anything obvious in VM's journal, but it might be in VM's .xsession-errors which wasn't saved.

I’ll try installing in a StandaloneVM and see how things break.

@marmarek
Copy link
Member

marmarek commented Aug 3, 2023

Any update on this?

@DemiMarie
Copy link
Contributor Author

I’ll test this and see what goes wrong.

@DemiMarie DemiMarie force-pushed the username-service-config branch 2 times, most recently from c001c0c to 3d9698a Compare August 4, 2023 03:47
@marmarek
Copy link
Member

There is a merge conflict now :(

This allows the C compiler to type-check arguments to qrexec_log() and
thus to the logging macros.
exit() will double-flush streams, among other problems.
This avoids crashes if a buggy qrexec agent sends a malformed message.
It also avoids variable-length arrays (VLAs), which have been removed
from Linux due to security concerns.  Finally, it avoids storing the
entire message on the stack, which makes stack overflows less likely.
This is especially true on musl libc, which uses a much smaller default
stack size.
This requires some internal refactoring, which is split off into a
separate commit to ease reviewing.
libqrexec/toml.c Outdated Show resolved Hide resolved
libqrexec/toml.c Show resolved Hide resolved
libqrexec/toml.c Outdated Show resolved Hide resolved
This also dramatically improves the configuration parser.  Configuration
files now use a strict subset of TOML rather than an ad-hoc format with
no validation.

Fixes: QubesOS/qubes-issues#6354
Fixes: QubesOS/qubes-issues#8153
They are a bad idea and the Linux kernel has banned them from its code.
@marmarek marmarek merged commit e5c78a0 into QubesOS:main Oct 6, 2023
4 checks passed
@DemiMarie DemiMarie deleted the username-service-config branch October 14, 2023 22:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants