Windows suricataupdater.exe failure: pyyaml is required

[philrz]

I'd spotted this in a previous test artifact I'd created while working on #44, but I've now reproduced it with the draft release artifact suricata-v5.0.3-brim26.windows-amd64.zip as well. I just unpacked it and then:

C:\Users\Phil\Downloads\suricata-v5.0.3-brim26.windows-amd64\home\runneradmin\suricata>.\suricataupdater.exe
error: pyyaml is required
2020/11/29 12:42:53 launchSuricata failed exit status 1

[henridf]

I think this is related to the windows antivirus but I can't definitely prove it (*).

First, I repro-ed the issue on a Windows 2019 Server (gcloud) VM. As I launched suricata-updater, some little lower-right corner pop-up thingy flashed by about running downloaded code.

So on a hunch (and because I know the frozen updater worked when I added it), I downloaded the full Brim prerelease at https://storage.googleapis.com/brimsec/suricata/brim-package/windows/Brim-Setup.exe , and was able to run its the suricata-updater.exe (/c/Users/henridf/AppData/Local/Brim/app-0.19.0/resources/app/zdeps/suricata/suricataupdater.exe) ok. Since our Brim packages are signed, that might explain the difference.

(There's still something odd about the updater output... looking into that and will file a separate issue if nec).

(*) I tried disabling various "SmartScreen" controls to see if that would allow the un-signed updater to run, but it still failed. I can't claim I know those controls well enough to be sure I disabled whatever needed to (if this is indeed the culprit).

[henridf]

Well, the anti-virus explanation was bogus, as @philrz predicted. The problem was that the relevant python packages weren't installed on the host running pyinstaller. In investigating this today, I did confirm that an earlier version does start ok (https://storage.googleapis.com/brimsec/suricata/suricata-v5.0.3-brim11.windows-amd64.zip), where was brim12 (and onwards) exhibits that "pyyaml is required" error. I don't know how to explain that.

[philrz]

Verified using the "build-suricata" artifact suricata-v5.0.3-brimpre1.windows-amd64.

On a fresh Windows 2019 Server VM on Google Cloud, I unpacked the artifact and was immediately able to run suricataupdater.exe.

C:\Users\phil\Downloads\suricata-v5.0.3-brimpre1.windows-amd64\suricata>.\suricataupdater.exe
�[32m2/12/2020 -- 03:21:35�[0m - <�[33mInfo�[0m> -- Loading C:\Users\phil\Downloads\suricata-v5.0.3-brimpre1.windows-amd64\suricata\update.yaml�[0m
�[32m2/12/2020 -- 03:21:35�[0m - <�[33mInfo�[0m> -- Found Suricata version 5.0.3 at C:\Users\phil\Downloads\suricata-v5.0.3-brimpre1.windows-amd64\suricata\bin\suricata.exe.�[0m
�[32m2/12/2020 -- 03:21:35�[0m - <�[33mInfo�[0m> -- Loading C:\Users\phil\Downloads\suricata-v5.0.3-brimpre1.windows-amd64\suricata\brim-conf.yaml�[0m
�[32m2/12/2020 -- 03:21:36�[0m - <�[33mInfo�[0m> -- Disabling rules for protocol modbus�[0m
�[32m2/12/2020 -- 03:21:36�[0m - <�[33mInfo�[0m> -- Disabling rules for protocol dnp3�[0m
�[32m2/12/2020 -- 03:21:36�[0m - <�[33mInfo�[0m> -- Disabling rules for protocol enip�[0m
�[32m2/12/2020 -- 03:21:36�[0m - <�[33mInfo�[0m> -- No sources configured, will use Emerging Threats Open�[0m
�[32m2/12/2020 -- 03:21:36�[0m - <�[33mInfo�[0m> -- Last download less than 15 minutes ago. Not downloading https://rules.emergingthreats.net/open/suricata-5.0.3/emerging.rules.tar.gz.�[0m
�[32m2/12/2020 -- 03:21:36�[0m - <�[33mInfo�[0m> -- Loading distribution rule file C:\Users\phil\Downloads\suricata-v5.0.3-brimpre1.windows-amd64\suricata\share\suricata\rules\app-layer-events.rules�[0m
�[32m2/12/2020 -- 03:21:36�[0m - <�[33mInfo�[0m> -- Loading distribution rule file C:\Users\phil\Downloads\suricata-v5.0.3-brimpre1.windows-amd64\suricata\share\suricata\rules\decoder-events.rules�[0m
�[32m2/12/2020 -- 03:21:36�[0m - <�[33mInfo�[0m> -- Loading distribution rule file C:\Users\phil\Downloads\suricata-v5.0.3-brimpre1.windows-amd64\suricata\share\suricata\rules\dhcp-events.rules�[0m
�[32m2/12/2020 -- 03:21:36�[0m - <�[33mInfo�[0m> -- Loading distribution rule file C:\Users\phil\Downloads\suricata-v5.0.3-brimpre1.windows-amd64\suricata\share\suricata\rules\dnp3-events.rules�[0m
�[32m2/12/2020 -- 03:21:36�[0m - <�[33mInfo�[0m> -- Loading distribution rule file C:\Users\phil\Downloads\suricata-v5.0.3-brimpre1.windows-amd64\suricata\share\suricata\rules\dns-events.rules�[0m
�[32m2/12/2020 -- 03:21:36�[0m - <�[33mInfo�[0m> -- Loading distribution rule file C:\Users\phil\Downloads\suricata-v5.0.3-brimpre1.windows-amd64\suricata\share\suricata\rules\files.rules�[0m
�[32m2/12/2020 -- 03:21:36�[0m - <�[33mInfo�[0m> -- Loading distribution rule file C:\Users\phil\Downloads\suricata-v5.0.3-brimpre1.windows-amd64\suricata\share\suricata\rules\http-events.rules�[0m
�[32m2/12/2020 -- 03:21:36�[0m - <�[33mInfo�[0m> -- Loading distribution rule file C:\Users\phil\Downloads\suricata-v5.0.3-brimpre1.windows-amd64\suricata\share\suricata\rules\ipsec-events.rules�[0m
�[32m2/12/2020 -- 03:21:36�[0m - <�[33mInfo�[0m> -- Loading distribution rule file C:\Users\phil\Downloads\suricata-v5.0.3-brimpre1.windows-amd64\suricata\share\suricata\rules\kerberos-events.rules�[0m
�[32m2/12/2020 -- 03:21:36�[0m - <�[33mInfo�[0m> -- Loading distribution rule file C:\Users\phil\Downloads\suricata-v5.0.3-brimpre1.windows-amd64\suricata\share\suricata\rules\modbus-events.rules�[0m
�[32m2/12/2020 -- 03:21:36�[0m - <�[33mInfo�[0m> -- Loading distribution rule file C:\Users\phil\Downloads\suricata-v5.0.3-brimpre1.windows-amd64\suricata\share\suricata\rules\nfs-events.rules�[0m
�[32m2/12/2020 -- 03:21:36�[0m - <�[33mInfo�[0m> -- Loading distribution rule file C:\Users\phil\Downloads\suricata-v5.0.3-brimpre1.windows-amd64\suricata\share\suricata\rules\ntp-events.rules�[0m
�[32m2/12/2020 -- 03:21:36�[0m - <�[33mInfo�[0m> -- Loading distribution rule file C:\Users\phil\Downloads\suricata-v5.0.3-brimpre1.windows-amd64\suricata\share\suricata\rules\smb-events.rules�[0m
�[32m2/12/2020 -- 03:21:36�[0m - <�[33mInfo�[0m> -- Loading distribution rule file C:\Users\phil\Downloads\suricata-v5.0.3-brimpre1.windows-amd64\suricata\share\suricata\rules\smtp-events.rules�[0m
�[32m2/12/2020 -- 03:21:36�[0m - <�[33mInfo�[0m> -- Loading distribution rule file C:\Users\phil\Downloads\suricata-v5.0.3-brimpre1.windows-amd64\suricata\share\suricata\rules\stream-events.rules�[0m
�[32m2/12/2020 -- 03:21:36�[0m - <�[33mInfo�[0m> -- Loading distribution rule file C:\Users\phil\Downloads\suricata-v5.0.3-brimpre1.windows-amd64\suricata\share\suricata\rules\tls-events.rules�[0m
�[32m2/12/2020 -- 03:21:36�[0m - <�[33mInfo�[0m> -- Ignoring file rules/emerging-deleted.rules�[0m
�[32m2/12/2020 -- 03:21:37�[0m - <�[33mInfo�[0m> -- Loaded 28589 rules.�[0m
�[32m2/12/2020 -- 03:21:38�[0m - <�[33mInfo�[0m> -- Disabled 14 rules.�[0m
�[32m2/12/2020 -- 03:21:38�[0m - <�[33mInfo�[0m> -- Enabled 0 rules.�[0m
�[32m2/12/2020 -- 03:21:38�[0m - <�[33mInfo�[0m> -- Modified 0 rules.�[0m
�[32m2/12/2020 -- 03:21:38�[0m - <�[33mInfo�[0m> -- Dropped 0 rules.�[0m
�[32m2/12/2020 -- 03:21:38�[0m - <�[33mInfo�[0m> -- Enabled 145 rules for flowbit dependencies.�[0m
�[32m2/12/2020 -- 03:21:38�[0m - <�[33mInfo�[0m> -- Backing up current rules.�[0m
�[32m2/12/2020 -- 03:21:40�[0m - <�[33mInfo�[0m> -- Writing rules to C:\Users\phil\Downloads\suricata-v5.0.3-brimpre1.windows-amd64\suricata\var\lib\suricata\rules\suricata.rules: total: 28589; enabled: 21202; added: 0; removed 0; modified: 14�[0m
�[32m2/12/2020 -- 03:21:40�[0m - <�[33mInfo�[0m> -- Writing C:\Users\phil\Downloads\suricata-v5.0.3-brimpre1.windows-amd64\suricata\var\lib\suricata\rules\classification.config�[0m
�[32m2/12/2020 -- 03:21:40�[0m - <�[33mInfo�[0m> -- Skipping test, disabled by configuration.�[0m
�[32m2/12/2020 -- 03:21:40�[0m - <�[33mInfo�[0m> -- Done.�[0m

@henridf: Do you know what to make of the message about "Last download less than 15 minutes ago. Not downloading..."? I literally ran it first thing after I unpacked the ZIP, so I'm not sure what it's comparing to. Maybe the timestamps of the files I just unpacked to the filesystem?

[henridf]

@henridf: Do you know what to make of the message about "Last download less than 15 minutes ago. Not downloading..."? I literally ran it first thing after I unpacked the ZIP, so I'm not sure what it's comparing to. Maybe the timestamps of the files I just unpacked to the filesystem?

Yes, that is correct. The change in #57 addresses this.