Workaround for Salt config against Whonix

[conorsch]

Partial progress toward #139. Doesn't resolve, but at least gets the
tests passing on sd-journalist again. Still struggling to get all tests
passing against sd-whonix.

[conorsch]

Marking as WIP because I still get a single failing test, against sd-whonix.

[conorsch]

All tests are now passing for me. You might need to reboot all VMs after applying the fixes, to ensure that sd-whonix was created from the customized whonix-gw-14 image.

[emkll]

Running make all and make test on both master and this branch, I am get the same 5 errors, on both branches:

======================================================================
ERROR: test_do_not_open_here (test_journalist_vm.SD_Journalist_Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/m/securedrop-workstation/tests/test_journalist_vm.py", line 21, in test_do_not_open_here
    "sd-journalist/do-not-open-here")
  File "/home/m/securedrop-workstation/tests/base.py", line 55, in assertFilesMatch
    remote_content = self._get_file_contents(remote_path)
  File "/home/m/securedrop-workstation/tests/base.py", line 51, in _get_file_contents
    "/bin/cat {}".format(path)])
  File "/usr/lib64/python2.7/subprocess.py", line 219, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
CalledProcessError: Command '['qvm-run', '-p', 'sd-journalist', '/bin/cat /usr/local/bin/do-not-open-here']' returned non-zero exit status 1

======================================================================
ERROR: test_move_to_svs (test_journalist_vm.SD_Journalist_Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/m/securedrop-workstation/tests/test_journalist_vm.py", line 13, in test_move_to_svs
    "sd-journalist/move-to-svs")
  File "/home/m/securedrop-workstation/tests/base.py", line 55, in assertFilesMatch
    remote_content = self._get_file_contents(remote_path)
  File "/home/m/securedrop-workstation/tests/base.py", line 51, in _get_file_contents
    "/bin/cat {}".format(path)])
  File "/usr/lib64/python2.7/subprocess.py", line 219, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
CalledProcessError: Command '['qvm-run', '-p', 'sd-journalist', '/bin/cat /usr/local/bin/move-to-svs']' returned non-zero exit status 1

======================================================================
ERROR: test_sd_process_display (test_journalist_vm.SD_Journalist_Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/m/securedrop-workstation/tests/test_journalist_vm.py", line 29, in test_sd_process_display
    "sd-journalist/sd-process-display")
  File "/home/m/securedrop-workstation/tests/base.py", line 55, in assertFilesMatch
    remote_content = self._get_file_contents(remote_path)
  File "/home/m/securedrop-workstation/tests/base.py", line 51, in _get_file_contents
    "/bin/cat {}".format(path)])
  File "/usr/lib64/python2.7/subprocess.py", line 219, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
CalledProcessError: Command '['qvm-run', '-p', 'sd-journalist', '/bin/cat /usr/local/bin/sd-process-display']' returned non-zero exit status 1

======================================================================
ERROR: test_sd_process_download (test_journalist_vm.SD_Journalist_Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/m/securedrop-workstation/tests/test_journalist_vm.py", line 17, in test_sd_process_download
    "sd-journalist/sd-process-download")
  File "/home/m/securedrop-workstation/tests/base.py", line 55, in assertFilesMatch
    remote_content = self._get_file_contents(remote_path)
  File "/home/m/securedrop-workstation/tests/base.py", line 51, in _get_file_contents
    "/bin/cat {}".format(path)])
  File "/usr/lib64/python2.7/subprocess.py", line 219, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
CalledProcessError: Command '['qvm-run', '-p', 'sd-journalist', '/bin/cat /usr/local/bin/sd-process-download']' returned non-zero exit status 1

======================================================================
ERROR: test_sd_process_feedback (test_journalist_vm.SD_Journalist_Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/m/securedrop-workstation/tests/test_journalist_vm.py", line 25, in test_sd_process_feedback
    "sd-journalist/sd-process-feedback")
  File "/home/m/securedrop-workstation/tests/base.py", line 55, in assertFilesMatch
    remote_content = self._get_file_contents(remote_path)
  File "/home/m/securedrop-workstation/tests/base.py", line 51, in _get_file_contents
    "/bin/cat {}".format(path)])
  File "/usr/lib64/python2.7/subprocess.py", line 219, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
CalledProcessError: Command '['qvm-run', '-p', 'sd-journalist', '/bin/cat /usr/local/bin/sd-process-feedback']' returned non-zero exit status 1

======================================================================
FAIL: test_accept_sd_xfer_extracted_file (test_sd_whonix.SD_Whonix_Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/m/securedrop-workstation/tests/test_sd_whonix.py", line 20, in test_accept_sd_xfer_extracted_file
    self.assertFileHasLine("/usr/local/etc/torrc.d/50_user.conf", line)
  File "/home/m/securedrop-workstation/tests/base.py", line 72, in assertFileHasLine
    raise AssertionError(msg)
AssertionError: File /usr/local/etc/torrc.d/50_user.conf does not contain expected line HidServAuth <something>.onion <hidserv_token>

----------------------------------------------------------------------

[conorsch]

@emkll Please try:

1. Re-run make all from this feature branch.
2. Re-re-run make all from this feature branch!
3. Run make test.

Assuming the changes presented here actually resolve the issue—even via a workaround—the tests will only pass if sd-journalist and sd-whonix are created from the updated Whonix templates. There's no ordering logic in place to ensure that the templates are updated, then shut down, before the child AppVMs are created, which is why I'm suggesting the additional VM management via a second destroy/create cycle.

[emkll]

Thanks for the clarification, @conorsch. Running the all makefile target does indeed fix the issue, all tests passing for me locally,

[conorsch]

Callooh callay! Thanks for confirming, @emkll. Will monitor the upstream issue so we can circle back and clean up here, but at least we're unblocked for near-term dev env usage now.