Manual Hunting Notes Linux

Linux

Suspicious Users

grep -v '/nologin\|/false' /etc/passwd | cut -d: -f1

Current typical users on this system are:

• root
• sync
• champuser

Run groups [username] to see what groups the account is in. Run id [username] to show it with the id #s

• Can also run cat /etc/passwd | grep [username] to get more information
• Check if a user has admin privledges with sudo -l -U [username]
• See when a user was created with grep [username] /var/log/auth.log
• Check for associated processes with ps aux | grep [username]
• Check home directory for suspicious files sudo ls -la /home/[username]

General suspicious users checks

• Check for users with UID 0 (root privileges) awk -F: '$3 == 0 {print $1}' /etc/passwd
• Check for users with empty passwords (CRITICAL security issue) sudo awk -F: '($2 == "" || $2 == "!") {print $1}' /etc/shadow
• Check recently created users (look at /etc/passwd modification time) ls -lt /etc/passwd /etc/shadow /etc/group
• Check user's shell (suspicious if legitimate service account has /bin/bash) grep -E 'svc_|admin_|webadmin' /etc/passwd

Remove with sudo userdel -r [username]

Groups

cut -d: -f1 /etc/group - List all groups (to see a less cut down version, cat /etc/group

getent [groupname] - check users in groups

• Sudo for Debian, Wheel for CentOS and RHEL

Remove with sudo groupdel [groupname]

Network

ss -tuln - list all active network connections

• sudo ss -tulnp to list processes with it

Check details with:

• sudo lsof -i :[port]
• sudo ss -tulnp | grep [port]

Note the PID, then investigate:

• ps aux | grep [PID]
• ls -la /proc/[PID]/exe - investigate process location
• sudo lsof -p [PID] - check open files

End with kill [PID]

Processes

ps aux - show process tree with ps auxf

• Check for suspicious processes with ps aux | grep -E '(bash|sh|nc|ncat|python|perl)'
• Can also just do (bash|ncat|perl)
• Check if a process is long running ps -eo pid,etime,cmd | grep bash

Cat the location of the suspicious process to see what its doing

NOTE: The following locations are common places for malware

• /opt - commercial software also runs from here
• /tmp - temporary files and installation scripts
• /var/tmp - long term temporary files
• /dev/shm

ps aux | grep -E '(/tmp|/var/tmp|/dev/shm|/opt)' - running from suspicious locations

Key difference: Legitimate software has:

• Proper package installation (dpkg, rpm)
• Known vendor names
• Proper directory structure (/opt/vendor/product/)
• Service files in /etc/systemd/system/
• Not long-running bash scripts

Processes with no terminal: ps aux | awk '$7 == "?"'

Recently started processes: ps -eo pid,etime,cmd --sort=start_time | head -20

Services

systemctl list-unit-files --type=service

• systemctl status [service]
• systemctl cat [service]

Check when the service was created

• ls -la /etc/systemd/system/system-update-custom.service
• stat /etc/systemd/system/system-update-custom.service

To Remove:

• sudo systemctl stop [service]
• sudo systemctl disable [service]
• sudo rm [service location]

Scheduled Tasks

sudo crontab -l - root's crontab

Check all user crontabs

for user in $(cut -f1 -d: /etc/passwd); do 

    echo "=== Crontab for $user ==="

    sudo crontab -u $user -l 2>/dev/null

done

Check system-wide cron jobs

ls -la /etc/cron.d/

ls -la /etc/cron.daily/

ls -la /etc/cron.hourly/

ls -la /etc/cron.monthly/

ls -la /etc/cron.weekly/

cat /etc/crontab

For Further Analysis

cat the cron job scrips

Check when they were created:

• ls -la /opt/scheduled_tasks
• stat /opt/scheduled_tasks/[script]

Check cron execution logs:

• grep CRON /var/log/syslog | tail -20
• grep CRON /var/log/cron | tail -20 - RHEL/CentOS

Check for execution of suspicious scripts

• grep -E '(system_maintenance|data_backup)' /var/log/syslog

To Remove

Remove suspicious files, then:

• sudo crontab -e - select an editor then remove the jobs you don't want
  • Run this as not sudo for crontabs that aren't root

File Permissions

ls -la /tmp/system_config.txt

ls -la /opt/credentials.txt

SETUID and SETGUID

# Find all SUID files

find / -perm -4000 -type f 2>/dev/null

# Find all SGID files

find / -perm -2000 -type f 2>/dev/null

# Find both SUID and SGID files

find / -perm /6000 -type f 2>/dev/null