Skip to content
This repository has been archived by the owner on Apr 4, 2023. It is now read-only.

Commit

Permalink
fixes for prunner, log, and checkpid (#17)
Browse files Browse the repository at this point in the history
* improvements for checkpid, log, and prunner
- removed command logging for prunner, as functions with their own logging would get logged twice. each command sent to prunner should deal with its own logging.
- checkpid now uses to check for duplicate process names and does not read the pid file. this allows processes to start that have lingering pid files. suggestions welcome to make it do both.
- enhanced log safety to prevent hangs on empty pipes.

* updated readme
  • Loading branch information
MrDrMcCoy committed Apr 5, 2018
1 parent 7f575af commit 8531d4a
Show file tree
Hide file tree
Showing 3 changed files with 56 additions and 42 deletions.
29 changes: 17 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
# bblib

The _"Better BASH Library"_: A set of functions to assist with creating well-written and reliable BASH scripts. The functions are documented in-line within `bblib.bash`, and a simple example that uses it is in `example.bash`.
The _"Better Bash Library"_: A set of functions to assist with creating well-written and reliable Bash scripts. The functions are documented in-line within `bblib.bash`, and a simple example that uses it is in `example.bash`.

## Usage

Add this to the top of your BASH script:
Add this to the top of your Bash script:

```bash
source <(wget -qO- https://raw.githubusercontent.com/MrDrMcCoy/bblib/1.1.7/bblib.bash)
source <(wget -qO- https://raw.githubusercontent.com/MrDrMcCoy/bblib/1.1.8/bblib.bash)
```

Alternately, clone this repo locally and use `source` with the full path to `bblib.bash`.
Expand Down Expand Up @@ -91,15 +91,14 @@ Once `bblib.bash` is sourced in your script, you may refer to any of its supplie
- Description: Checks to see if the user running the script matches the desired username and exits on failure.
- Usage: `requireuser [user]`
- `bash4check`
- Description: Checks to see if you are on BASH 4.0 or above and exits if not.
- Usage: Place `bash4check` at the beginning of any function that uses BASH 4+ features.
- Description: Checks to see if you are on Bash 4.0 or above and exits if not.
- Usage: Place `bash4check` at the beginning of any function that uses Bash 4+ features.
- `finally`
- Description: A function that runs extra commands before the script exits
- Usage: Add actions to its list by running: `FINALCMDS+=("command arg arg")`
- `checkpid`
- Description: Checks to see if another copy of this script is running by maintaining a PID file
- Description: Checks to see if another copy of this script is running by checking `ps` and maintaining a PID file.
- Usage: `checkpid`
- Notes: This function only works properly in Linux, as it depends on PROCFS.
- `prunner`
- Description: Executes commands in parallel.
- Usage:
Expand Down Expand Up @@ -130,7 +129,7 @@ Once `bblib.bash` is sourced in your script, you may refer to any of its supplie
- `LOGFILE`
- Description: Set this to have `log` additionally output to a file.
- Used by: `log`.
- Notes: This will capture debug output if BASH has `set -x`.
- Notes: This will capture debug output if Bash has `set -x`.
- Default: _unset_
- `PIDFILE`
- Description: The path to a file for tracking the PID of the script.
Expand Down Expand Up @@ -161,8 +160,14 @@ The commands that `bblib.bash` calls out to are listed here, in case you are on
- Used by: `usage`
- `fold`
- Used by: `pprint`
- `grep`
- Used by: `checkpid`
- `logger`
- Used by: `log`
- `ps`
- Used by: `checkpid`
- `rm`
- Used by: `checkpid`
- `tee`
- Used by: `log`
- `tr`
Expand All @@ -174,8 +179,8 @@ The commands that `bblib.bash` calls out to are listed here, in case you are on

If you would like to extend this library, some resources for advanced usage are available here:

- The BASH Beginners Guide, which is not just for beginners: <https://www.tldp.org/LDP/Bash-Beginners-Guide/html/>
- The Advanced BASH guide: <http://tldp.org/LDP/abs/html/>
- The BASH Hackers Wiki has great advanced usage description and examples for BASH: <http://wiki.bash-hackers.org/>
- BASH Style Guide: <https://google.github.io/styleguide/shell.xml>
- The Bash Beginners Guide, which is not just for beginners: <https://www.tldp.org/LDP/Bash-Beginners-Guide/html/>
- The Advanced Bash guide: <http://tldp.org/LDP/abs/html/>
- The Bash Hackers Wiki has great advanced usage description and examples for Bash: <http://wiki.bash-hackers.org/>
- Bash Style Guide: <https://google.github.io/styleguide/shell.xml>
- A very good `getopts` tutorial: <https://sookocheff.com/post/bash/parsing-bash-script-arguments-with-shopts/>
27 changes: 11 additions & 16 deletions bblib.bash
Original file line number Diff line number Diff line change
Expand Up @@ -78,9 +78,10 @@ uc () {
hr () {
# Print horizontal rule
# Usage: hr [character]
local CHARACTER="${1:0:1}"
local CHARACTER="${1:--}"
local CHARACTER="${CHARACTER:0:1}"
local -i COLUMNS=${COLUMNS:-$(tput cols)}
printf '%*s\n' "${COLUMNS:-80}" '' | tr ' ' "${CHARACTER:--}"
printf '%*s\n' "${COLUMNS:-80}" '' | tr ' ' "${CHARACTER}"
}

log () {
Expand Down Expand Up @@ -119,7 +120,7 @@ log () {
# Send message to logger
if [ "${NUMERIC_SEVERITY}" -le "${NUMERIC_LOGLEVEL}" ] ; then
tr '\n' ' ' <<< "${LOGMSG}" | logger -s -p "user.${NUMERIC_SEVERITY}" -t "${LOGTAG} " |& \
if [ -n "${LOGFILE:-}" ] ; then
if [ -n "${LOGFILE:-}" ] && [ ! -t 0 ] ; then
tee -a "${LOGFILE}" | pprint ${LOGCOLORS[$NUMERIC_SEVERITY]}
elif [ ! -t 0 ]; then
pprint ${LOGCOLORS[$NUMERIC_SEVERITY]} < /dev/stdin
Expand Down Expand Up @@ -150,7 +151,7 @@ bash4check () {
if [ "${BASH_VERSINFO[0]}" -lt 4 ] ; then
quit "ALERT" "Sorry, you need at least bash version 4 to run this function: ${FUNCNAME[1]}"
else
log "DEBUG" "This script is safe to enable BASH version 4 features"
log "DEBUG" "This script is safe to enable Bash version 4 features"
fi
}

Expand All @@ -167,18 +168,12 @@ checkpid () {
# Check for and maintain pidfile
# Usage: checkpid
local PIDFILE="${PIDFILE:-${0}.pid}"
if [[ ! -d "/proc/$$" ]]; then
quit "ERROR" "This function requires procfs. Are you on Linux?"
elif [[ ! -f "${PIDFILE}" ]] ; then
echo -n "$$" > "${PIDFILE}"
if [[ $( ps ao args | grep -wc "$(basename "$0")" ) -gt 3 ]] ; then
quit "ERROR" "Script '$(basename "$0")' is already running, exiting."
else
echo "$$" > "${PIDFILE}"
FINALCMDS+=("rm -v ${PIDFILE}")
log "DEBUG" "PID $$ has no conflicts and has been written to ${PIDFILE}"
elif [[ "$( cat "${PIDFILE}" || true )" -ne $$ ]] ; then
quit "ERROR" "This script is already running, exiting."
elif [[ "$( cat "${PIDFILE}" || true )" -eq $$ ]] ; then
log "DEBUG" "PID $$ matches the contents of ${PIDFILE}, proceeding."
else
quit "ALERT" "Unknown error verifying unique PID."
fi
}

Expand Down Expand Up @@ -261,7 +256,7 @@ prunner () {
local -i INDEX=0
until [ ${#PQUEUE[@]} == 0 ] ; do
if [ "$(jobs -rp | wc -l)" -lt "${THREADS:-8}" ] ; then
${PCMD} ${PQUEUE[$INDEX]} 2> >(log "ERROR") | log "DEBUG" &
${PCMD} ${PQUEUE[$INDEX]}
unset "PQUEUE[$INDEX]"
((INDEX++)) || true
fi
Expand All @@ -273,7 +268,7 @@ prunner () {
trap finally EXIT

# Trap for killing runaway processes and exiting
trap "quit 'ALERT' 'Exiting on signal' '3'" SIGINT SIGTERM
trap "quit 'ALERT' 'Exiting on signal' '3'" INT TERM QUIT HUP

# Trap to capture errors
trap 'quit "ALERT" "Command failed with exit code $?: $BASH_COMMAND" "$?"' ERR
Expand Down
42 changes: 28 additions & 14 deletions tests/test.bash
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,12 @@ main () {

log "INFO" "Starting tests"

# Pid check
log "INFO" "Test checkpid"
checkpid || true
log "INFO" "Test checkpid in second shell"
bash "$0" || true & wait

# Test argparser
log "INFO" "Test argparser with no options"
bash -c 'source ../bblib.bash ; source test.bash.conf ; argparser' || true
Expand All @@ -23,14 +29,28 @@ main () {
bash -c 'source ../bblib.bash ; source test.bash.conf ; argparser -y' || true

# Test hr
log "INFO" "Test hr ="
hr =
log "INFO" "Test hr"
hr
hr = | pprint yellow

# Test lc, uc, and pprint
# Test lc, uc, pprint line wrapping
log "INFO" "Test lc"
lc < lorem-ipsum.txt | pprint > lorem-ipsum-lc-pprint.out |& log "DEBUG"
head -n1 lorem-ipsum.txt | lc | pprint
log "INFO" "Test uc"
uc < lorem-ipsum.txt | pprint > lorem-ipsum-uc-pprint.out |& log "DEBUG"
tail -n1 lorem-ipsum.txt | uc | pprint

# Test pprint colors
log "INFO" "Test pprint"
pprint 0 <<< black
pprint 1 <<< red
pprint 2 <<< green
pprint 3 <<< yellow
pprint 4 <<< blue
pprint 5 <<< magenta
pprint 6 <<< cyan
pprint 7 <<< white
pprint 8 <<< bold
pprint 9 <<< underline

# Test shorthand log log loggers
log_debug "shorthand log_debug test"
Expand All @@ -46,12 +66,6 @@ main () {
log "INFO" "Test bash4check"
bash4check

# Pid check
log "INFO" "Test checkpid"
checkpid || true
log "INFO" "Test checkpid in second shell"
bash -c 'source ../bblib.bash ; source test.bash.conf ; checkpid' || true

# User check
log "INFO" "Test requireuser with \$REQUIREUSER"
bash -c 'source ../bblib.bash ; source test.bash.conf ; requireuser' || true
Expand All @@ -71,9 +85,9 @@ main () {

# Parallel test
log "INFO" "Test prunner gzipping the .out files with arguments for the jobs"
prunner -c "gzip -fk" *.out
log "INFO" "Test prunner echoing the .out files with stdin for the jobs"
find . -maxdepth 1 -type f -name "*.out" | head -n5 | prunner -c "echo found file " -t 6
prunner -c "gzip -f" *.out
log "INFO" "Test prunner echoing the .gz files with stdin for the jobs"
find . -maxdepth 1 -type f -name "*.gz" | head -n5 | prunner -c "echo found file " -t 6

# Add cleanup tasks
FINALCMDS+=('rm -v *.out')
Expand Down

0 comments on commit 8531d4a

Please sign in to comment.