Salt fails to run in minimal/whonix templates

[marmarek]

Qubes OS version:

R4.0 (but R3.2 probably too)

Affected component(s):

salt

---

Steps to reproduce the behavior:

sudo qubesctl --skip-dom0 --target=whonix-ws-14 --show-output pkg.upgrade

Expected behavior:

Install updates in the template.

Actual behavior:

whonix-ws-14:
      ----------
      _error:
          Failed to return clean data
      retcode:
          0
      stderr:
          Traceback (most recent call last):
            File "/var/tmp/.root_62a99a_salt/salt-call", line 15, in <module>
              salt_call()
            File "/var/tmp/.root_62a99a_salt/py2/salt/scripts.py", line 395, in salt_call
              import salt.cli.call
            File "/var/tmp/.root_62a99a_salt/py2/salt/cli/call.py", line 5, in <module>
              import salt.utils.parsers
            File "/var/tmp/.root_62a99a_salt/py2/salt/utils/parsers.py", line 27, in <module>
              import salt.config as config
            File "/var/tmp/.root_62a99a_salt/py2/salt/config/__init__.py", line 98, in <module>
              _DFLT_IPC_WBUFFER = _gather_buffer_space() * .5
            File "/var/tmp/.root_62a99a_salt/py2/salt/config/__init__.py", line 88, in _gather_buffer_space
              import salt.grains.core
            File "/var/tmp/.root_62a99a_salt/py2/salt/grains/core.py", line 44, in <module>
              import salt.utils.dns
            File "/var/tmp/.root_62a99a_salt/py2/salt/utils/dns.py", line 32, in <module>
              import salt.modules.cmdmod
            File "/var/tmp/.root_62a99a_salt/py2/salt/modules/cmdmod.py", line 34, in <module>
              import salt.utils.templates
            File "/var/tmp/.root_62a99a_salt/py2/salt/utils/templates.py", line 32, in <module>
              import salt.utils.http
            File "/var/tmp/.root_62a99a_salt/py2/salt/utils/http.py", line 41, in <module>
              import salt.loader
            File "/var/tmp/.root_62a99a_salt/py2/salt/loader.py", line 28, in <module>
              import salt.utils.event
            File "/var/tmp/.root_62a99a_salt/py2/salt/utils/event.py", line 85, in <module>
              import salt.transport.ipc
            File "/var/tmp/.root_62a99a_salt/py2/salt/transport/ipc.py", line 21, in <module>
              from tornado.locks import Semaphore
            File "/var/tmp/.root_62a99a_salt/py2/tornado/locks.py", line 18, in <module>
              from concurrent.futures import CancelledError
          ImportError: No module named concurrent.futures
      stdout:

General notes:

Looks like python2-futures package is needed and it isn't handled (bundled?) by salt.

---

Related issues:

See https://openqa.qubes-os.org/tests/233#downloads, update-qubesctl-upgrade.log file.

[marmarek]

I'm not sure what is the best solutions, I see those:

1. Report this as a salt upstream bug - the reasoning is: comments in the code says it use "python for ultra-maximum compatibility", so it should handle minimal python installation too.
2. Install python2-futures even in minimal template, just for salt.
3. Don't change anything and document that python2-futures needs to be installed manually before using salt.

I prefer option 2, to make salt usable also for initial configuration.

@adrelanos @andrewdavidwong any other opinion on this one?

[andrewdavidwong]

1. Yes, but it might be resolved by just changing the comment.
2. I suppose so, but note Meta-ticket: suggest/remove default applications in official templates #1781 (comment).
3. This would be fine.

[conorsch]

Thanks for opening, @marmarek! Ran into this problem recently, and the proposed solution 2 in your list strikes me as the best approach to resolve going forward. In the meantime, as a temporary workaround:

1. Open terminal in whonix-ws-14 TemplateVM.
2. Run sudo apt install -y python-futures. (N.B. python-futures is a virtual package, pointing to python-concurrent.futures.)
3. Shut down TemplateVM.

You can now use Salt to provision AppVMs based on whonix-ws-14. Let me know if any further testing would be useful on your end.

[adrelanos]

1. Yes, please.

2. Yes, also please.

I am unsure about the minimal templates but for Whonix it would be best if Qubes packages (core agent or something) made sure all needed dependencies are installed.

[unman]

Interestingly, this doesn't seem to be a problem in a stretch minimal template.
Has the fix already been pushed?

[marmarek]

BTW it's already fixed upstream: saltstack/salt#48234 saltstack/salt#48289
And looks to be queued for salt 2018.3.3.

[marmarek]

Until salt 2018.3.3 is released (and packaged at least for Fedora), I'm going to add python-concurrent.futures package to our templates.

[hexagonrecursion]

Until salt 2018.3.3 is released (and packaged at least for Fedora), I'm going to add python-concurrent.futures package to our templates.

[user@fedora-30 ~]$ rpm -q salt
salt-2019.2.0-1.fc30.noarch

[SaswatPadhi]

This issue appears to be fixed now (at least on R4.0).