Bash for Pentesters

Various Bash commands and scripts I commonly use while pentesting

Run an arbitary command in parallel

#!/usr/bin/env bash

# Check if GNU Parallel is installed
if ! command -v parallel &> /dev/null; then
    echo "GNU Parallel is not installed. Please install it and try again."
    exit 1
fi

# Number of workers
workers=4

# Arbitrary command to run on each line of input
command="echo"

# Read multi-line input from stdin
input=$(cat)

# Export command for parallel execution
export command

# Use GNU Parallel to run the command on each line with the specified number of workers
echo "$input" | parallel -j "$workers" "$command" {}

Calculate and sort on entropy

The backstory here is I was performing an internal network pentest and had dumped a list of domain users from Active Directory. Normally I would manually review the account comments looking for plaintext passwords, but in this case the domain had almost 40k users and that would have taken far too many hours to reivew. The following code calculates the entropy of the 5th column of every line and prints the line number and the entropy. Yes, I realize this may not reveal simple passwords like "Password1".

#!/bin/bash

# Check if a file name and column number are provided
if [ "$#" -ne 2 ]; then
    echo "Usage: $0 filename column_number"
    exit 1
fi

filename=$1
column=$2

# Function to calculate Shannon entropy of a line
calculate_entropy() {
    line=$1
    len=${#line}
    declare -A freq

    # Calculate frequency of each character
    for (( i=0; i<$len; i++ )); do
        char="${line:$i:1}"
        ((freq["$char"]++))
    done

    # Calculate entropy
    entropy=0
    for f in "${freq[@]}"; do
        p=$(echo "$f / $len" | bc -l)
        entropy=$(echo "$entropy - $p * l($p) / l(2)" | bc -l)
    done

    echo $entropy
}

# Initialize a line counter
line_number=0

# Process the specified column of the file
while IFS= read -r line; do
    # Increment line counter
    ((line_number++))

    # Extract the column of interest
    text=$(echo "$line" | cut -d' ' -f$column)

    # Calculate the entropy
    entropy=$(calculate_entropy "$text")

    # Print the line number, followed by a space, followed by the entropy
    echo "$line_number $entropy"
done < "$filename"

Run that and redirect the output to a file. On the output file, run the following commands to sort based on entropy. This gave me results in a few minutes vs many hours to manually review.

$ cat output.txt | sort -k2,2n

1937 4.51364592935837067007                                                                                                                                             
36277 4.51364592935837067007                                                                                                                                            
1800 4.55026688517709218351                                                                                                                                             
1816 4.55026688517709218351                                                                                                                                             
1564 4.57246945877013537596                                                                                                                                             
1565 4.57246945877013537596                                                                                                                                             
1865 4.57246945877013537596                                                                                                                                             
1867 4.57246945877013537596                                                                                                                                             
1918 4.57246945877013537596

Use sort -kt,2nr to sort in reverse.

Split a file into equal chunks

This is useful for those times you have 9,000 hosts to scan and you have a time-boxed scan window every night.

#!/bin/bash

# Usage: ./split_ips.sh filename number_of_parts

FILENAME=$1
PARTS=$2

# Count the total number of lines (IP addresses) in the file
TOTAL_LINES=$(wc -l < $FILENAME)

# Calculate the number of lines per file, rounding up
(( LINES_PER_FILE = (TOTAL_LINES + PARTS - 1) / PARTS ))

# Split the file into parts
split -d -l $LINES_PER_FILE $FILENAME ${FILENAME}_part_

echo "Split $FILENAME into $PARTS parts."

Use the Chaos DNS API to enum subdomains, then check to see which domains resolve to scoped IP addresses

First, gather top level domains for the pentest in file tlds.txt. Then run:

while read -r line;do chaos -silent -d "$line";done<tlds.txt | tee alldomains.txt

Next, run the following script which runs the host command and checks the IP address to see if the IP is in scope:

#!/bin/bash

# Check if alldomains.txt and scope.txt exist
if [ ! -f alldomains.txt ]; then
    echo "Error: alldomains.txt not found."
    exit 1
fi

if [ ! -f scope.txt ]; then
    echo "Error: scope.txt not found."
    exit 1
fi

# Read each line in alldomains.txt
while read -r domain; do
    # Use the host command to find the IP address
    ip_addresses=$(host "$domain" | grep "has address" | awk '{ print $4 }')

    # Check each IP address
    for ip in $ip_addresses; do
        # Check if the IP address is in scope.txt
        if grep -q "$ip" scope.txt; then
            # If the IP is in scope.txt, print the domain
            echo "$domain"
            # Break the loop after finding the first match
            break
        fi
    done
done < alldomains.txt

Filter Nessus export of Host,IP to csv by port:

cat nessus.csv | awk -F, '$2 ~ /445/' | cut -d, -f1 | sed 's/"//g' | sort -uV

Process Nmap gnmap file, copy output to Word and create table (Split on tabs):

cat nmap.gnmap | grep 'Ports:' | tr '\t' ' ' | cut -d ' ' -f 2- | sed -e 's/  */\t/1;s/  */\t/1'

Certificate transparency hostname lookups:

curl -s 'https://crt.sh/?q=%.[domain.com]&output=json' | jq -r ".[].name_value" | sort -u

Filter based on column value

awk '$2 == "200"'

Grep for URLS:

grep -oE '\b(https?|ftp|file)://[-A-Za-z0-9+&@#/%?=~_|!:,.;]*[-A-Za-z0-9+&@#/%=~_|]'

Search for content in files 

egrep -lir --include=*.txt "172.16."

Grep for text that starts with:

cat no_owner_ips_sorted.txt | grep "^172.16." >> Wucher.txt

Sort IP addresses:

sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4 no_owner_ips.txt

Column to comma separated list:

paste -d, -s file

Use searchsploit in Kali and filter output

searchsploit -w SQL | grep 'Microsoft' | grep -v 'Denial of Service' | cut -d '|' -f 2 | cut -d ' ' -f 2 | xargs -I {} firefox -new-tab -url {}

Convert First Last name to usernames:

Convert First Last to first initial last name:

awk '{print tolower(substr($1,1,3) $2)}’

Convert First Last to first.last

awk '{print tolower($1 “." $2)}

Convert First Last to firstlast

awk '{print tolower($1 $2)}’

Employee names to first initial last name, in lowercase:

cat EmployeeNames.txt | while read -r line;do first=$(echo $line | cut -d ' ' -f 1 | head -c 1) && last=$(echo $line | cut -d ' ' -f 2) && echo "$first$last";done | awk '{print tolower($0)}' > flast_usernames.txt

Grep email addresses from a file:

grep -E -o "\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,6}\b” file.txt

Find files above or below a size and perform action on results:

find . -size -500b -exec cat {} \;

Extract IP addresses from a file:

grep -Eo '([0-9]{1,3}\.){3}[0-9]{1,3}' <file> | sort | uniq

Add user to sudoers:

echo “blumbergh ALL=(ALL) NOPASSWD:ALL” >> /etc/sudoers

Remove duplicates from john netntlmv2 files:

sort -u -t: -k1,1 johnpwfile_netntlmv2

Remove junk from john netntlmv2 files:

sed -i '/WORKGROUP/d' johnpwfile_netntlmv2

Masscan file of hostnames:

while read -r line;do dig +short $line | sed -n 2p | grep -E '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' | xargs -I {} masscan --open -sS -p 0-65535 {} --rate=1000 -oG {};done<targets.txt

Remove bash color codes

sed 's/\x1B\[[0-9;]\+[A-Za-z]//g'

Prefix slashes

awk '{print "//" $0}' | tr ' ‘ /

Grep interesting ports from gnmap files and output host IP’s to a file for each port.

echo "21,22,23,25,53,80,81,110,111,123,137-139,161,389,443,445,500,512,513,548,623-624,1099,1241,1433-1434,1521,2049,2375,2376,2483-2484,3306,3389,4333,4786,4848,5432,5800,5900,5901,6000,6001,7001,8000,8080,8181,8443,10000,16992-16993,27017,32764" | tr ',' '\n' | tr '-' '\n’ > limited-ports.txt
while read -r line; do grep " $line/open/tcp” [scan name].gnmap  | cut -d ' ' -f 2 > $line.txt;done <limited-ports.txt

Clean up Kerberoast hashes:

cat hashes.txt | sed 's/[[:blank:]]//g' | sed 's/TicketByteHexStream://' | sed 's/Hash://'

GoPhish: oneliner to parse out submitted credentials when you need a list of all credentials submitted (while using GoPhish)

strings gophish.db | grep "Submitted Data" | awk -F, '{print $9" "$6}'| awk -F'"' '{print tolower($4)" "$8}'  | sort -u

Grep NTLM hashes from ndts.dit:

grep '.*:.*:.*:.*:::' ntds.dit

Grep LM hashes from ntds.dit:

grep '.*:.*:.*:.*:::' ntds.dit | grep -v '\$' | grep -v 'aad3b435b51404eeaad3b435b51404ee'

Find all files recursively and run a command on each:

find . -type f -exec strings '{}' \;

Grep SMB Signing Results from nmap output:

grep -B 9 -e 'disabled' -e 'enabled but not required' internal/scans/smb2-security-mode.txt.nmap | grep 'Nmap scan report for' | grep -Eo '([0-9]{1,3}\.){3}[0-9]{1,3}'

Find domain name:

nslookup server | grep Server | grep -E -o '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' | nslookup | grep -o '[^ ]*$' | cut -d '.' -f 2- | sed -r 's/\.$//'

Find Domain Controllers:

domain=`nslookup server | grep Server | grep -E -o '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' | nslookup | grep -o '[^ ]*$' | cut -d '.' -f 2- | sed -r 's/\.$//'`;nslookup -type=srv _ldap._tcp."$domain" | grep $domain | grep -o '[^ ]*$' | sed -r 's/\.$//' | tr ' ' '\n'

Save a packet capture for five minutes:

sudo timeout 300 tcpdump -i eth0 -s0 -w pcapfile.pcap

Find HSRP plaintext "cisco" password in packet capture:

sudo timeout 300 tcpdump -i eth0 -s0 -w pcapfile.pcap && tcpdump -XX -r pcapfile.pcap udp port 1985 or udp port 2029 | grep -B4 cisco

Execute a script in a tmux session (detached):

tmux new-session -d -s "myTempSession" /opt/my_script.sh