Skip to content

Helper BASH scripts that make any project maintenance easier. versioning, dependencies, tools, CI/CD etc.

License

Notifications You must be signed in to change notification settings

OleksandrKucherenko/e-bash

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

22 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Enhanced BASH Scripts

Roadmap

  • High-level scripts should be in own bin OR scripts
  • Git helpers
  • GitLabs helper scripts (work with branches, forks, submodules)
  • Slack notifications helper scripts
  • Telemetry module (report metrics to CI or DataDog)
  • Globals module (declarative way of defining script dependencies to global environment variables)
  • Logs monitoring documentation (different streams/files/tty for different information: info, debug, telemetry, dependencies)
  • Copyright headers composing/parsing (extract from the file, update, insert)

Local Dev Environment - Requirements

Note: alternative Unit Test Frameworks, Bats - https://github.com/bats-core/bats-core

brew install direnv
brew install shellcheck
brew install shfmt
brew install shellspec
brew install kcov

TDD - Test Driven Development, run tests on file change

# run all unit tests on file change
watchman-make -p 'spec/*_spec.sh' '.scripts/*.sh' --run "shellspec"

# run failed only unit tests on file change
watchman-make -p 'spec/*_spec.sh' '.scripts/*.sh' --run "shellspec --quick"

Usage

Installation into your project:

# subtree approach (TODO: fix me!)
git subtree --squash -P vendor/bats-all add https://github.com/hyperupcall/bats-all HEAD

refs:

Colors

source ".scripts/_colors.sh"

echo -e "${cl_red}Hello World${cl_reset}"

Script Dependencies

Bootstrap

source ".scripts/_dependencies.sh"

dependency bash "5.*.*" "brew install bash"
dependency direnv "2.*.*" "curl -sfL https://direnv.net/install.sh | bash"
dependency shellspec "0.28.*" "brew install shellspec"
optional kcov "42" "brew install kcov"
dependency shellcheck "0.9.*" "curl -sS https://webi.sh/shellcheck | sh"
dependency shfmt "3.*.*" "curl -sS https://webi.sh/shfmt | sh"
dependency watchman "2023.07.*.*" "brew install watchman"

Logger

Requirements:

  • zero dependencies, pure BASH
  • prefix for all logger messages
  • work in pipe mode (forward logs to the named pipe)
    • write logs to pipe; single line or multiple lines in '|' pipe mode
    • read logs from the named pipe and output to the console (or file). Required PID and log tag.
    • run logs to file/stream in background process mode
  • support prefix for each log message
  • listen to DEBUG environment variable for enabling/disabling logs
    • enable/disable log by tag name or tag name prefix (support wildcards)
  • execute command with logging the command and it parameters first (ref: https://bpkg.sh/pkg/echo-eval)
source ".scripts/_logger.sh"
logger common "$@" # declare echo:Common and printf:Common functions, tag: common
logger debug "$@" # declare echo:Debug and printf:Debug functions, tag: debug

echo:Common "Hello World" # output "Hello World" only if tag common is enabled

export DEBUG=*          # enable logger output for all tags
export DEBUG=common     # enable logger output for common tag only
export DEBUG=*,-common  # enable logger output for all tags except common

# advanced functions
config:logger:Common "$@" # re-configure logger enable/disable for common tag

# echo in pipe mode
find . -type d -max-depth 1 | echo:Common

# echo in output redirect
find . -type d -max-depth 1 >echo:Common

Arguments Parsing

Requirements:

  • zero dependencies, pure BASH
  • support short and long arguments
  • support default values
  • support required arguments
  • support aliases for arguments
  • support destination variables for argument
  • compose help documentation from arguments definition
# pattern: "{argument},-{short},--{alias}={output_variable}:{default_initialize_value}:{reserved_args_quantity}"
# example: "-h,--help=args_help:true:0", on --help or -h set $args_help variable to true, expect no arguments;
# example: "$1,--id=args_id::1", expect first unnamed argument to be assigned to $args_id variable; can be also provided as --id=123
export ARGS_DEFINITION="-h,--help -v,--version=:1.0.0 --debug=DEBUG:*"

# will automatically parse script arguments with definition from $ARGS_DEFINITION global variable
source ".scripts/_arguments.sh"

# check variables that are extracted
echo "Is --help: $help"
echo "Is --version: $version"
echo "Is --debug: $DEBUG"

# advanced run. parse provided arguments with definition from $ARGS_DEFINITION global variable
parse:arguments "$@"

Common(s) Functions And Inputs

source ".scripts/_commons.sh"

# Extract parameter from global env variable OR from secret file (file content)
env:variable:or:secret:file "new_value" \
  "GITLAB_CI_INTEGRATION_TEST" \
  ".secrets/gitlab_ci_integration_test" \
  "{user friendly message}"

echo "Extracted: ${new_value}"

# validate/confirm input parameter by user input
# string
# Yes/No

UI: Selector

Selector

source ".scripts/_commons.sh"

# Select value from short list of choices
declare -A -g connections && connections=(["d"]="production" ["s"]="cors-proxy:staging" ["p"]="cors-proxy:local")
echo -n "Select connection type: " && tput civis # hide cursor
selected=$(input:selector "connections") && echo "${cl_blue}${selected}${cl_reset}"

UI: Ask for Password

Ask for Password

source ".scripts/_commons.sh"

# Usage:
echo -n "Enter password: "
password=$(input:readpwd) && echo "" && echo "Password: $password"

Semver - Semantic Versioning

Requirements:

  • parse version code, according to semver specification
  • compare version code
  • verify version constraints
  • compose version code from array of segments
source ".scripts/_semver.sh"

# verify that version is passing the constraints expression
semver:constraints "1.0.0-alpha" ">1.0.0-beta || <1.0.0" && echo "$? - OK!" || echo "$? - FAIL!" # expected OK

# more specific cases
semver:constraints:simple "1.0.0-beta.10 != 1.0.0-beta.2" && echo "OK!" || echo "$? - FAIL!"

# parse and recompose version code
semver:parse "2.0.0-rc.1+build.123" "V" \
  && for i in "${!V[@]}"; do echo "$i: ${V[$i]}"; done \
  && semver:recompose "V"
  
# test version code
echo "1" | grep -E "${SEMVER_LINE}" --color=always --ignore-case || echo "OK!"

Self-Update

Requirements:

  • detect a new version of the script
  • download multiple versions into folder and do a symbolic link to a specific version
  • download from GIT repo (git clone)
    • keep MASTER as default, extract version tags as sub-folders
  • download from GIT repo release URL (tar/zip archive)
    • extract archive to a version sub-folder
  • rollback to previous version (or specified one)
    • rollback to latest backup file (if exists)
  • partial update of the scripts, different versions of scripts from different version sub-folders
    • developer can bind file to a specific version by calling function self-update:version:bind
  • verify SHA1 hash of the scripts
    • compute file SHA1 hash and store it in *.sha1 file
  • understand version expressions
    • latest - latest stable version
    • * or next - any highest version tag (INCLUDING: alpha, beta, rc etc)
    • branch:{any_branch} or tag:{any_tag} - any branch name (also works for TAGs)
    • >, <, >=, <=, ~, !=, || - comparison syntax
    • 1.0.0 or =1.0.0 - exact version
    • ~1.0.0 - version in range >= 1.0.x, patch releases allowed
    • ^1.0.0 - version in range >= 1.x.x, minor & patch releases allowed
    • >1.0.0 <=1.5.0 - version in range > 1.0.0 && <= 1.5.0
    • >1.0.0 <1.1.0 || >1.5.0 - version in range (> 1.0.0 < 1.1.0) || (> 1.5.0)

refs:

source ".scripts/_self-update.sh"

# check for version update in range >= 1.0.x, stable versions
# try to update itself from https://github.com/OleksandrKucherenko/e-bash.git repository
self-update "~1.0.0"                          # patch releases allowed
self-update "^1.0.0"                          # minor releases allowed
self-update "> 1.0.0 <= 1.5.0"                # stay in range

# update specific file to latest version tag
self-update "latest" ".scripts/_colors.sh"    # latest stable
self-update "*" ".scripts/_colors.sh"         # any highest version tag

# update specific file to MASTER version (can be used any branch name)
self-update "branch:master" ".scripts/_colors.sh"   
self-update "tag:v1.0.0" ".scripts/_colors.sh"

# bind file to a specific version
self-update:version:bind "v1.0.0" ".scripts/_colors.sh"

# TBD

# INTEGRATION EXAMPLE

# do self-update on script exit
trap "self-update '^1.0.0'" EXIT

# OR:
function __exit() {
  # TODO: add more cleanup logic here
  self-update '^1.0.0'
}
trap "__exit" EXIT

Troubleshooting

# rollback with use of backup file(s)
source ".scripts/_self-update.sh" && self-update:rollback:backup "${full_path_to_file}"

# rollback to specific version
source ".scripts/_self-update.sh" && self-update:rollback:version "v1.0.0" "${full_path_to_file}"

Profile BASH script execution

Profiler

# print timestamp for each line of executed script
PS4='+ $(gdate "+%s.%N ($LINENO) ")' bash -x bin/version-up.sh

# save trace to file
PS4='+ $(echo -n "$EPOCHREALTIME [$LINENO]: ")' bash -x bin/version-up.sh 2>trace.log

# process output to more user-friendly format: `execution_time | line_number | line_content`
PS4='+ $(echo -n "$EPOCHREALTIME [$LINENO]: ")' bash -x bin/version-up.sh 2>trace.log 1>/dev/null && cat trace.log | bin/profiler/tracing.sh

# profile script execution and print summary
bin/profiler/profile.sh bin/version-up.sh

Colors support in my terminal

Terminal Colors

# print all colors for easier selection
demos/demo.colors.sh

References