Skip to content
Library for bash utility methods and tools
Shell Roff Python Other
Branch: master
Clone or download
ismarc Merge pull request #12 from jonahx/add-blog-link
Add blog post link to README
Latest commit 50a1eb7 Nov 10, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
filehandling All libraries are loaded by init script Oct 8, 2019
git
helpers Retry command is evaluated Oct 9, 2019
k8s All libraries are loaded by init script Oct 8, 2019
logging Initialise bash-lib repo Apr 25, 2019
test-utils All libraries are loaded by init script Oct 8, 2019
tests-for-this-repo Retry command is evaluated Oct 9, 2019
.gitignore Initial commit Apr 15, 2019
.gitmodules Initialise bash-lib repo Apr 25, 2019
.gittrees Initialise bash-lib repo Apr 25, 2019
CONTRIBUTING.md Apply Contributing suggestions from code review Jul 3, 2019
CyberArk_Open_Source_Contributor_Agreement.pdf OSS Requirements II Jun 20, 2019
Jenkinsfile Initialise bash-lib repo Apr 25, 2019
LICENSE
README.md Add blog post link to README Nov 10, 2019
init Bash-Lib functions are exported to sub-shells Oct 9, 2019
run-tests All libraries are loaded by init script Oct 8, 2019
secrets.yml

README.md

bash-lib

Introductory blog post: https://www.conjur.org/blog/stop-bashing-bash/

                   _______________  _______________
                 .'               .'               .|
               .'               .'               .' |
             .'_______________.'______________ .'   |
             | ___ _____ ___ || ___ _____ ___ |     |
             ||_=_|__=__|_=_||||_=_|__=__|_=_||     |
       ______||_____===_____||||_____===_____||     | __________
    .'       ||_____===_____||||_____===_____||    .'          .'|
  .'         ||_____===_____||||_____===_____||  .'          .'  |
.'___________|_______________||_______________|.'__________.'    |
|.----------.|.-----___-----.||.-----___-----.||    |_____.----------.
|]          |||_____________||||_____________|||  .'      [          |
||          ||.-----___-----.||.-----___-----.||.'        |          |
||          |||_____________||||_____________|||==========|          |
||          ||.-----___-----.||.-----___-----.||    |_____|          |
|]         o|||_____________||||_____________|||  .'      [        'o|
||          ||.-----___-----.||.-----___-----.||.'        |          |
||          |||             ||||_____________|||==========|          |
||          |||             |||.-----___-----.||    |_____|          |
|]          |||             ||||             |||  .'      [          |
||__________|||_____________||||_____________|||.'________|__________|
''----------'''------------------------------'''----------''
            (o)LGB                           (o)

The place to store functions that are used in pipelines for multiple repos.

Please add whatever is useful to you, but keep it tidy so its still useful to everyone else :)

Release Status: Alpha

TL;DR: Ready for use, but needs expansion.

The functions in this repo are tested and ready for use, but certain libs are pretty much place holders (eg logging). Those need further contributions before they provide a comprehensive solution.

License: Apache 2.0

See the license file

Usage

Add bash-lib into your project in the way that best fits your workflow. The only requirement is that you pin the version of bash-lib that you use. This is important so that changes to bash-lib do not have the power to break all projects that use bash-lib. Your project can then test updates to bash-lib and roll forward periodicly.

Options:

  • Add a submodule: they are an easy way to integrate bash-lib and automatically use a single SHA until manually updated. Submodules add a pointer from a mount point in your repo to the external repo (bash-lib), and require workflow changes to ensure that pointer is derferenced during clone, checkout and some other opertaions.
  • Add a subtree: This repo uses subtrees to pull in test dependencies. Subtrees copy an external repo into a subdirectory of the host repo, no workflow changes are required. Subtrees naturally keep a single version of bash-lib until explicitly updated. Note that subtree merge commits do not rebase well ⚠️, so best to keep subtree updates in separate PRs from normal commits.
  • Clone bash-lib in your deployment process, bash-lib doesn't have to be within your repo, just needs to be somewhere where your scripts can source init. This is where it's most important that you implement a mechanism to always use the same SHA, as a clone will track master by default, which is not an allowed use of bash-lib.

Once you have bash-lib cloned in your project, you source two things:

  1. Source bash-lib/init. This ensures submodules are initalised and sets the BASH_LIB_DIR env var to the absolute path to the bash-lib dir. This makes it easy to source libraries from other scripts.
  2. Source ${BASH_LIB_DIR}/lib-name/lib for any libraries you are interested in.

You are now ready to use bash-lib functions :)

Structure

The /init script sets up everything required to use the library, most importantly the BASH_LIB_DIR variable which gives the absolute path to the root of the library and should be used for sourcing the modules.

The repo is organized into libraries, each library is a directory that has a lib file. Sourcing the lib for a library should expose all the functions that library offers. The lib file may source or reference other supporting files within it's directory.

.
├── libname
│   ├── lib
│   └── supporting-file
├── init # init script, source this first
├── run-tests # top level test script, executes all tests
├── secrets.yml # secrets required for executing tests
├── test-utils
│   ├── bats # git subtree
│   ├── bats-assert-1 # git subtree
│   ├── bats-support # git subtree
│   ├── lib
│   └── tap2junit
└── tests-for-this-repo
    ├── filehandling.bats
    ├── fixtures #
    │   └── libname # Dir containing test fixtures for a library
    ├── tap2junit
    ├── libname.bats # contains tests for libname/lib
    ├── python-lint # supporting files for python lint
    ├── run-bats-tests # script to run bats tests
    ├── run-gitleaks # script to check for leaked secrets
    └── run-python-lint # script to run python lint

Contents

Library Description Functions
filehandling Functions relating to file and path handling
  1. abs_path: Ensure a path is absolute
git Git helpers
  1. repo_root: Find the root of the current git repo.
  2. all_files_in_repo: List files tracked by git.
  3. remote_latest_tag: Returns the symbolic name of the latest tag from a remote.
  4. remote_latest_tagged_commit: Returns the SHA of the most recently tagged commit in a remote repo (tag^{}).
  5. remote_sha_for_ref: Returns the SHA for a given ref from a named remote.
  6. remote_tag_for_sha: Returns the tag corresponding to a SHA from a named remote - if there is one.
  7. tracked_files_excluding_subtrees: List files tracked by git, but excluding any files that are in paths listed in .gittrees.
  8. cat_gittrees: Returns the contents of .gittrees from the top level of the repo, excluding any comments. Fails if .gittrees is not present.
helpers Bash scripting helpers
  1. die: print message and exit 1
  2. spushd/spopd: Safe verisons of pushd & popd that call die if the push/pop fails, they also drop stdout.
  3. retry: Retry a command until it succeeds up to a user specified maximum number of attempts. Escalating delay between attempts.
k8s Utils for connecting to K8s
  1. build_gke_image: Build docker image for running kubectl commands against GKE.
  2. delete_gke_image: Delete image from GKE.
  3. run_docker_gke_command: Run command in gke-utils container, already authenticated to k8s cluster.
logging Helpers related to login
  1. announce: Echo message in ascii banner to distinguish it from other log messages.
test-utils Helpers for executing tests
  1. shellcheck_script: Execute shellcheck against a script, uses docker.
  2. find_scripts: Find git tracked files with extension.
  3. tap2junit: Convert a subset of TAP to JUnit XML. Retains logs for errors.

Contributing

For further information on contributing, style & testing, please see CONTRIBUTING.md

Maintainers

You can’t perform that action at this time.