Installation of wazuh-indexer fails if the noexec flag is set on the /tmp directory

[rauldpm]

Wazuh version	Install type	Action performed	Platform
4.3.0	Wazuh installation assistant	Install	-

Hello team, the user @jimmynarula mentioned in this issue, a possible bug with the Java temporary directory used by Wazuh indexer, it is possible that the temporary directory used by Java by default (/tmp) is mounted as a partition and has the noexec flag enabled, which prevents the correct installation of Wazuh indexer.

The solution proposed by the user is to change the directory to: -Djava.io.tmpdir=/var/log/wazuh-indexer

But I do not consider that using this directory in the installation is a good practice and we should look for an alternative directory in case the /tmp directory is not usable, that is, we have to carry out the following checks and steps, both in the documentation step by step as in the assistant itself.

1. Check that /tmp has exec permissions
2. Check that /tmp is or is not present in fstab, and if it is present, check that it does not have the noexec flag enabled.
3. If /tmp does not have exec permissions or have the noexec flag, the temporary Java directory specified in -Djava.io.tmpdir, must be changed to a directory that does allow its use

The AMI used by the user displays the following fstab content:

LABEL=cloudimg-rootfs   /        ext4   defaults,discard        0 1
tmpfs /tmp tmpfs defaults,rw,nosuid,nodev,noexec,relatime 0 0
tmpfs /dev/shm tmpfs defaults,nodev,nosuid,noexec 0 0

Regards, Raúl.

[gdiazlo]

Blocked by wazuh/wazuh-indexer#4

[kdebisschop]

As a way to handle this on Ubuntu without altering the distributed files, I used the reference to /etc/sysconfig/wazuh-indexer in the systemd file:

sudo mkdir -p /var/lib/wazuh-indexer/tmp
sudo chmod --reference /var/ossec/tmp /var/lib/wazuh-indexer/tmp
sudo chown wazuh-indexer:wazuh-indexer /var/lib/wazuh-indexer/tmp
sudo mkdir /etc/sysconfig
echo OPENSEARCH_TMPDIR=/var/lib/wazuh-indexer/tmp | sudo tee /etc/sysconfig/wazuh-indexer

[mrjson79]

As a way to handle this on Ubuntu without altering the distributed files, I used the reference to /etc/sysconfig/wazuh-indexer in the systemd file:

sudo mkdir -p /var/lib/wazuh-indexer/tmp
sudo chmod --reference /var/ossec/tmp /var/lib/wazuh-indexer/tmp
sudo chown wazuh-indexer:wazuh-indexer /var/lib/wazuh-indexer/tmp
sudo mkdir /etc/sysconfig
echo OPENSEARCH_TMPDIR=/var/lib/wazuh-indexer/tmp | sudo tee /etc/sysconfig/wazuh-indexer

I also managed to get it working with the quoted solution.

[f-galland]

Setting the OPENSEARCH_TMPDIR environment variable from the wazuh-indexer.service unit to something other than /tmp seems to solve the issue.

• /tmp is mounted with noexec:

root@ubuntu2204:~/tests# mount  | grep '/tmp'
tmpfs on /tmp type tmpfs (rw,noexec,relatime,inode64)

• The wazuh-indexer.service unit is modified to set the OPENSEARCH_TMPDIR variable:

root@ubuntu2204:~/tests# systemctl cat wazuh-indexer.service --no-pager | tail -3
# /etc/systemd/system/wazuh-indexer.service.d/override.conf
[Service]
Environment=OPENSEARCH_TMPDIR=/usr/share/wazuh-indexer

• The service starts normally:

root@ubuntu2204:~/tests# systemctl status wazuh-indexer --no-pager -n0
● wazuh-indexer.service - wazuh-indexer
     Loaded: loaded (/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: enabled)
    Drop-In: /etc/systemd/system/wazuh-indexer.service.d
             └─override.conf
     Active: active (running) since Thu 2024-05-16 15:00:30 -03; 8min ago
       Docs: https://documentation.wazuh.com
   Main PID: 34301 (java)
      Tasks: 60 (limit: 4572)
     Memory: 1.3G
        CPU: 29.693s
     CGroup: /system.slice/wazuh-indexer.service
             └─34301 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache…

[f-galland]

Changing the OPENSEARCH_TMPDIR variable allows the service to start, but /tmp seems to keep being used even if mounted with the noexec flag:

root@ubuntu2204:~# systemctl cat wazuh-indexer --no-pager
# /lib/systemd/system/wazuh-indexer.service
[Unit]
Description=wazuh-indexer
Documentation=https://documentation.wazuh.com
Wants=network-online.target
After=network-online.target

[Service]
Type=notify
RuntimeDirectory=wazuh-indexer
PrivateTmp=true
Environment=OPENSEARCH_HOME=/usr/share/wazuh-indexer
Environment=OPENSEARCH_PATH_CONF=/etc/wazuh-indexer
Environment=PID_DIR=/run/wazuh-indexer
Environment=OPENSEARCH_SD_NOTIFY=true
EnvironmentFile=-/etc/default/wazuh-indexer

WorkingDirectory=/usr/share/wazuh-indexer

User=wazuh-indexer
Group=wazuh-indexer

ExecStart=/usr/share/wazuh-indexer/bin/systemd-entrypoint -p ${PID_DIR}/wazuh-indexer.pid --quiet

# StandardOutput is configured to redirect to journalctl since
# some error messages may be logged in standard output before
# wazuh-indexer logging system is initialized. Wazuh-indexer
# stores its logs in /var/log/wazuh-indexer and does not use
# journalctl by default. If you also want to enable journalctl
# logging, you can simply remove the "quiet" option from ExecStart.
StandardOutput=journal
StandardError=inherit

# Specifies the maximum file descriptor number that can be opened by this process
LimitNOFILE=65535

# Specifies the maximum number of processes
LimitNPROC=4096

# Specifies the maximum size of virtual memory
LimitAS=infinity

# Specifies the maximum file size
LimitFSIZE=infinity

# Disable timeout logic and wait until process is stopped
TimeoutStopSec=0

# SIGTERM signal is used to stop the Java process
KillSignal=SIGTERM

# Send the signal only to the JVM rather than its control group
KillMode=process

# Java process is never killed
SendSIGKILL=no

# When a JVM receives a SIGTERM signal it exits with code 143
SuccessExitStatus=143

# Allow a slow startup before the systemd notifier module kicks in to extend the timeout
TimeoutStartSec=75

[Install]
WantedBy=multi-user.target

# Built for packages-4.9.0 (packages)

# /etc/systemd/system/wazuh-indexer.service.d/override.conf
[Service]
Environment=OPENSEARCH_TMPDIR=/usr/share/wazuh-indexer/tmp

root@ubuntu2204:~# ls -lsah /usr/share/wazuh-indexer/tmp/
total 8.0K
4.0K drwxr-xr-x 2 wazuh-indexer wazuh-indexer 4.0K May 17 09:59 .
4.0K drwxr-x--- 9 wazuh-indexer wazuh-indexer 4.0K May 17 09:58 ..

root@ubuntu2204:~# ls -lsah /tmp/
total 4.0K
   0 drwxrwxrwt  5 root root  100 May 17 09:59 .
4.0K drwxr-xr-x 22 root root 4.0K May 16 14:22 ..
   0 drwxr-xr-x  2 root root   40 May 17 08:44 hsperfdata_root
   0 drwx------  3 root root   60 May 17 09:24 systemd-private-d40bd8ebddd54982a0de1c73cb33a60b-fwupd.service-Rja2mD
   0 drwx------  3 root root   60 May 17 09:59 systemd-private-d40bd8ebddd54982a0de1c73cb33a60b-wazuh-indexer.service-YmIPlm

root@ubuntu2204:~# systemctl status wazuh-indexer --no-pager
● wazuh-indexer.service - wazuh-indexer
     Loaded: loaded (/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: enabled)
    Drop-In: /etc/systemd/system/wazuh-indexer.service.d
             └─override.conf
     Active: active (running) since Fri 2024-05-17 09:59:28 -03; 5min ago
       Docs: https://documentation.wazuh.com
   Main PID: 37086 (java)
      Tasks: 58 (limit: 4572)
     Memory: 1.3G
        CPU: 26.202s
     CGroup: /system.slice/wazuh-indexer.service
             └─37086 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache…

May 17 09:59:20 ubuntu2204 systemd-entrypoint[37086]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Ope…13.0.jar)
May 17 09:59:20 ubuntu2204 systemd-entrypoint[37086]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootst…penSearch
May 17 09:59:20 ubuntu2204 systemd-entrypoint[37086]: WARNING: System::setSecurityManager will be removed in a future release
May 17 09:59:20 ubuntu2204 systemd-entrypoint[37086]: May 17, 2024 9:59:20 AM sun.util.locale.provider.LocaleProviderAdapter <clinit>
May 17 09:59:20 ubuntu2204 systemd-entrypoint[37086]: WARNING: COMPAT locale provider will be removed in a future release
May 17 09:59:21 ubuntu2204 systemd-entrypoint[37086]: WARNING: A terminally deprecated method in java.lang.System has been called
May 17 09:59:21 ubuntu2204 systemd-entrypoint[37086]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Sec…13.0.jar)
May 17 09:59:21 ubuntu2204 systemd-entrypoint[37086]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootst….Security
May 17 09:59:21 ubuntu2204 systemd-entrypoint[37086]: WARNING: System::setSecurityManager will be removed in a future release
May 17 09:59:28 ubuntu2204 systemd[1]: Started wazuh-indexer.
Hint: Some lines were ellipsized, use -l to show in full.