Skip to content
Lazy shell pipelines in plain JavaScript
TypeScript JavaScript Shell
Branch: master
Clone or download
Latest commit 92f0c96 Aug 19, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.vscode WIP Nov 23, 2017
src Update to V3. Refactoring. Subroutines. Aug 5, 2019
types/nodefunc-promisify Switch to typescript Dec 9, 2017
.gitignore WIP Nov 23, 2017
.npmignore WIP Nov 23, 2017
CHANGELOG.md Update to V3. Refactoring. Subroutines. Aug 5, 2019
LICENSE Switch to typescript Dec 9, 2017
README.md Update README.md Aug 19, 2019
build.sh Remove redundant files. Jan 17, 2018
package.json Update README Aug 5, 2019
tsconfig.json Switch to typescript Dec 9, 2017
yarn.lock Update to V3. Refactoring. Subroutines. Aug 5, 2019

README.md

Basho: Shell Automation with Plain JavaScript

Basho lets you to write complex shell tasks using plain JavaScript without having to dabble with shell scripting. But when needed, basho lets you easily integrate shell commands as well.

Install basho first. For now basho only works on Node v8.0 or above.

npm install -g basho

If you have npm > 5.2.0, you can use the npx command to try basho without installing.

# For example, Prints 100
npx basho -j 100

Basics

Basho evaluates a pipeline of instructions left to right. Instructions can be JavaScript code, reference to an external JS file, or a shell command. What makes basho interesting is Lazy Evaluation, more on this later.

To evaluate a JavaScript expression, use the option -j. Let’s start with a single item in the pipeline, a JavaScript constant.

# Prints 100
basho -j 100

# Prints true
basho -j true

# Prints 100
basho -j 10**2

The option -j can be omitted for the first expression.

# This prints 100 too
basho 100

# Prints 100
basho 10**2

The option -p avoids printing the final result. I am not sure where you'd need to use it, but it's there.

# Prints nothing
basho -p 100

Working with strings will need quoting, since bash will chew the quotes for itself. So you’ll need to use single quotes around your double quotes.

# Prints hello, world
basho '"hello, world"'

Piping Results

You can pipe an expression into a subsequent expression. The variable ‘x’ is always used as a placeholder for receiving the previous input.

# Prints 10100
basho 100 -j x**2 -j x+100

Quoting expressions

If an expression has spaces, it is important to quote it. In the following example, see how 'x + 100' is quoted.

# Prints 10100
basho 100 -j 'x**2' -j 'x + 100'

Similarly, if an expression contains bash special characters it is necessary to quote them. In the following example, the expression is quotes since '>' is the bash redirection operator.

# Prints 1
basho 100 -j 'x**2' -j 'x>100?1:2'

As a best practice, it is wise to quote all expressions (except maybe the trivially simple).

Lazy evaluation and exit conditions

You may choose to terminate the pipeline with the -t option when a condition is met. Since the pipeline is lazy, further expressions (or bash commands) are not evaluated.

# Prints 10 and 20. The rest are never evaluated.
basho [1,2,3,4,5] -t 'x>2' -j 'x*10'

Shell Commands

Execute shell commands with the -e option. The shell command is expanded as a JS template string, with the variable ‘x’ holding the input from the preceding command in the pipeline. Remember to quote or escape characters which hold a special meaning in your shell, such as $, >, <, |, () etc.

Tip: Single quotes are far easier to work with, since double quotes will try to expand $variables inside it.

# Prints 1000. Escape the $.
basho 1000 -e 'echo ${x}'

You can extend the pipeline further after a shell command. The shell command’s output becomes the input for the next command.

# echo 110 - which is (10^2) + 10
basho 10 -j 'x**2' -e 'echo ${x}' -j 'parseInt(x)+10' -e 'echo ${x}'

basho can receive input via stdin. As always, ‘x’ represents the input.

# Prints 100
echo 10 | basho 'parseInt(x)**2'

You can pipe multi-line output from other commands.

# Find all files and directories with the string 'git' in its name.
ls -al | basho -f 'x.includes("git")'

There’s nothing stopping you from piping basho's output either.

# Prints 100
basho 10 -j 'x**2' | xargs echo

Importing JS files

You can import a function from a JS file or an npm module with the --import option. The --import option takes two parameters; a filename or module name and an alias for the import. An import is available in all subsequent expressions throughout the pipeline.

# cat square.js
module.exports = function square(n) { return n ** 2; }

# prints 100. Imports square.js as sqr.
basho 10 --import square.js sqr -j 'sqr(x)'

# Prints 40000. Does sqr(10), then adds 100, then sqr(200)
basho 10 --import square.js sqr -j 'sqr(x)' -j 'x+100' -j 'sqr(x)'

Arrays, map, filter, flatMap and reduce

If the input to an expression is an array, the subsequent expression or command is executed for each item in the array. It's the equivalent of a map() function.

# echo 1; echo 2; echo 3; echo 4
basho [1,2,3,4] -e 'echo ${x}'

An input can also be an object, which you can expand in the template string.

basho '{ name: "jes", age: 100 }' -e 'echo ${x.name}, ${x.age}'

You can use an Array of objects.

# echo kai; echo niki
basho '[{name:"kai"}, {name: "niki"}]' -e 'echo ${x.name}'

Array of arrays, sure.

# echo 1 2 3; echo 3 4 5
basho '[[1,2,3], [3,4,5]]' -e 'echo ${x[0]} ${x[1]} ${x[2]}'

A command can choose to receive the entire array at once with the -a option.

# echo 4
basho [1,2,3,4] -a -j x.length -e 'echo ${x}'

Filter arrays with the -f option.

# echo 3; echo 4
basho [1,2,3,4] -f 'x>2' -e 'echo ${x}'

Reduce with the -r option. The first parameter is the lambda, the second parameter is the initial value of the accumulator.

# Prints the sum 10
basho [1,2,3,4] -r 'acc+x' 0 -e 'echo ${x}'

There's also flatMap, the -m option.

# Returns [11, 21, 12, 22, 13, 23]
basho [1,2,3] -m '[x+10,x+20]'

A flatMap can be used to flatten an array of arrays as well.

# Returns 1, 2, 3, 4
basho [[1,2],[2,3]] -m x

Btw, you could also access an array index in the template literal as the variable ‘i’ in lambdas and shell command templates.

# echo a1; echo b2; echo c3
basho '["a", "b", "c"]' -e 'echo ${x}${i}'

Reusable Expressions

Sometimes you want to reuse an expression multiple times in the pipeline. You can define expressions with the -d option and they get stored as fields in a variable named 'k'. See usage below.

Here's how to use it in JS expressions

# Prints 11, 12, 13
basho [10,11,12] -d add1 'x=>x+1' -j 'k.add1(x)'

Can be used in shell commands as well. Remember to quote though.

# Same as echo 10; echo 11; echo 12
basho [10,11,12] -d ECHO_CMD '"echo"' -e '${k.ECHO_CMD} N${x}'

Subroutines

Subroutines are mini-pipelines within a parent pipeline. This allows us to define a set of operations which could be repeatedly called for each item.

Subroutines are defined with the --sub option followed by the name of the sub. The sub continues till an --endsub is found. The sub is stored for subsequent usage is the variable 'k'.

# Multiplies by 200
basho [10,11,12] --sub multiply 'x*10' -j 'x*20' --endsub -j 'k.multiply(x)'

Nested Subroutines? Sure.

# Nested Subroutines
basho [10,11,12] \
  --sub multiply \
    --sub square 'x*x' --endsub \
    -j 'x*10' -j 'k.square(x)' \
  --endsub \
  -j 'k.multiply(x)'

Named expressions, Seeking and Combining expressions

The -n option gives a name to the result of the expression, so that you can recall it later with the -s (seek) or -c (combine) options.

# Prints 121; instead of (120*50) + 1
basho 10 -j x*10 -j x+20 -n add20 -j x*50 -s add20 -j x+1

The -s option allows you to seek a named result.

# Return [11, 21, 31, 41]
basho [10,20,30,40] -j x+1 -n add1 -j x+2 -n add2 -s add1

The -c option allows you to combine/multiplex streams into an sequence of arrays.

# Return [11, 13], [21, 23], [31, 33], [41, 43]
basho [10,20,30,40] -j x+1 -n add1 -j x+2 -n add2 -c add1,add2

Recursion

The -g option allows you to recurse to a previous named expression. It takes two parameters; (1) an expression name and (2) a predicate which stops the recursion.

Here's an expression that keeps recursing and adding 100 till it exceeds 1000.

# Prints 1025
basho 25 -j x+100 -n add1 -g add1 'x<1000'

Recursion is powerful. For instance, along with a promise that sleeps for a specified time, recursion can use used to periodically run a command. Usage is left to the reader as an exercise.

Promises!

If an JS expression evaluates to a promise, it is resolved before passing it to the next command in the pipeline.

# Prints 10
basho 'Promise.resolve(10)' -e 'echo ${x}'

# Something more useful
basho --import node-fetch fetch \
 -j 'fetch("http://oaks.nvg.org/basho.html")' \
 -e 'echo ${x}'

Logging

You can add a -l option anywhere in the pipeline to print the current value.

# Logs 10\n
basho 10 -l x -j x -e 'echo ${x}'

The -w option does the same thing, but without the newline.

# Logs 10 without a newline
basho 10 -w x -j x -e 'echo ${x}'

Error Handling

You can handle an error with the --error option, and choose to return an arbitrary value in its place. If unhandled, the pipeline is terminated immediately. In the following example, x.split() results in an exception on the second input (10) since a number does have the split() method. The error handler expression replaces the exception with the string 'skipped'.

basho '["a,b", 10, "c,d"]' -j 'x.split(",")' --error '"skipped"'

If the first argument to basho is --ignoreerror, basho will not exit on error. It will simply move to the next item.

basho --ignoreerror '["a,b", 10, "c,d"]' -j 'x.split(",")'

The --printerror option works like --ignoreerror, but prints the error.

basho --printerror '["a,b", 10, "c,d"]' -j 'x.split(",")'

Note that ignoreerror and printerror must not be preceded by any option except the --import option.

Real world examples

Count the number of occurences of a word in a line of text.

echo '"hello world hello hello"' | basho -j '(x.match(/hello/g) || []).length'

Recursively list all typescript files

find . | basho -f 'x.endsWith(".ts")'

Count the number of typescript files

find . | basho -f 'x.endsWith(".ts")' -a x.length

Get the weather in bangalore

echo '"Bangalore,in"' | basho --import node-fetch fetch 'fetch(`http://api.openweathermap.org/data/2.5/weather?q=${x}&appid=YOURAPIKEY&units=metric`)' -j 'x.json()' -j x.main.temp

Who wrote Harry Potter and the Philosopher's Stone?

basho --import node-fetch fetch 'fetch("https://www.googleapis.com/books/v1/volumes?q=isbn:0747532699")' -j 'x.json()' -j 'x.items[0].volumeInfo.authors'

Find all git hosted sub directories which might need a pull

ls | basho 'x.split("\t")' \
  -m x \
  -n dirname \
  -e 'cd ${x} && git remote update && git status' \
  -f 'x.some(_ => /branch is behind/.test(_))' \
  -s dirname

Find all git hosted sub directories which need to be pushed to remote

ls | basho 'x.split("\t")' \
  -m x \
  -n dirname \
  -e 'cd ${x} && git status' \
  -f '!x.some(_ => /nothing to commit/.test(_)) && !x.some(_ => /branch is up-to-date/.test(_))' \
  -s dirname

Check if basho version is at least 0.0.43

BASHO_VERSION=$(basho -v | basho 'x.split(".")' -j '(parseInt(x[0]) > 0 || parseInt(x[1]) > 0 || parseInt(x[2]) >= 43) && "OK"')

if [[ $BASHO_VERSION == "OK" ]]
then
  echo "All good. Format the universe."
else
  echo "Install basho version 0.0.43 or higher."
fi

That's it

Typing basho without any parameters does nothing but might make you happy. Or sad.

basho

Report issues or ping me on Twitter.

You can’t perform that action at this time.