crowdsec-firewall-bouncer binary / systemd path changed to /usr/bin vs /usr/sbin

[dakota-marshall]

Hey there! I am updating packages my machines today, and noticed that my package updates failed due to crowdsec-firewall-bouncer-iptables failing to restart:

Setting up crowdsec-firewall-bouncer-iptables (0.0.26) ...
Job for crowdsec-firewall-bouncer.service failed because the control process exited with error code.
See "systemctl status crowdsec-firewall-bouncer.service" and "journalctl -xe" for details.
dpkg: error processing package crowdsec-firewall-bouncer-iptables (--configure):
 installed crowdsec-firewall-bouncer-iptables package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 crowdsec-firewall-bouncer-iptables
E: Sub-process /usr/bin/dpkg returned an error code (1)

I looked and the systemd service is not happy, claiming that the binary doesn't exist at: /usr/sbin/crowdsec-firewall-bouncer, which is true because my PATH shows it at /usr/bin. It looks like 1 of two 2 things might have happened:

1. The binary changed locations, in which case it didnt get updated on the default systemd service file
2. The systemd service file changed the path for some reason.

It's worth noting that these are the default service files, and I have not modified them. I did however, have to create an override at /etc/systemd/system/crowdsec-firewall-bouncer.service.d/override.conf to get it working:

[Service]
ExecStart=
ExecStartPre=
ExecStart=/usr/bin/crowdsec-firewall-bouncer -c /etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml
ExecStartPre=/usr/bin/crowdsec-firewall-bouncer -c /etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml -t

I have about 5 to 8 hosts running the bouncer, and they all had this issue.

[buixor]

Thanks for the report, we're looking into it !

[buixor]

Hey again, what distribution are you using?

I just tried on my debian machine (upgrade from 0.0.25 to 0.0.26).
When I do the install and I let it deploy the maintainer's configuration file:

Preparing to unpack .../crowdsec-firewall-bouncer-iptables_0.0.26_amd64.deb ...
Removed /etc/systemd/system/multi-user.target.wants/crowdsec-firewall-bouncer.service.
locale: Cannot set LC_ALL to default locale: No such file or directory
Unpacking crowdsec-firewall-bouncer-iptables (0.0.26) over (0.0.25) ...
Setting up crowdsec-firewall-bouncer-iptables (0.0.26) ...

Configuration file '/etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml'
 ==> Modified (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** crowdsec-firewall-bouncer.yaml (Y/I/N/O/D/Z) [default=N] ? Y
Installing new version of config file /etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml ...
Installing new version of config file /etc/systemd/system/crowdsec-firewall-bouncer.service ...
cscli/crowdsec is present, generating API key
API Key successfully created
Created symlink /etc/systemd/system/multi-user.target.wants/crowdsec-firewall-bouncer.service → /etc/systemd/system/crowdsec-firewall-bouncer.service.

The service is running successfully. However, I do confirm that the path changed from /usr/sbin/ to /usr/bin/ over the upgrade.

[dakota-marshall]

These are all Ubuntu 22.04 instances.

I'll have to look and see if my logs show the line where it tries to update the systemd config file. It's probably worth mentioning that we deploy our updates with Ansibles apt module, but i don't know how much of a difference it makes.

I know no local file changes were made to that systemd file, so it shouldn't be an instance where the default option of 'no' was used when apt told us a config file that's been modified locally has changes.

Pure curiosity: what's the reason for updating the binary path?

[buixor]

Yes it is rather unfortunate. The change was initially done to bring consistency across the various bouncers.

We are looking for a way to avoid this, I will keep you posted

[buixor]

@dakota-marshall further question: Can you let me know if there are some specificities with your ansible install or some modification(s) to the systemd file? the installer should overwrite the existing file with the new one as long as it's untouched (at the end, I didn't manage to reproduce your issue on debian, no matter what I chose, the systemd file is overwritten with the new one, so the updated binary paths doesn't break anything).

Thanks,