`core`: functions module for shellfire

This module provides an essential framework for every shellfire application. It is included without any extra code being required. However, there is a template for programs that needs to be completed to make use of it.

The framework includes functions for most common needs, difficulties and complexities when writing shell script. The major areas it covers are:-

Core utilities
Base64 decoding
Command Line option parsing, including long options, help messages and exit codes
Managing child processes
Compatibility across shells and distros
Configuration
Binary dependency management and documentation
File reading utilities
Function manipulation, including existence and plugin registration
Miscellaneous init functions
Path utility functions
Embedding snippets
Temporary File handling
Coloured terminal output
Signal Handling
umasks
Argument validation
Variable and Value Helpers
Arrays, even in non-bash shells

Compatibility

Tag release_2015.0117.1750-1 is compatible with shellfire release release_2015.0117.1750-1.

Overview

Overridding a module's function(s)

If, for some reason, one of the functions isn't to your own liking then you can override its definition. Simply provide your own inside your shellfire application's _program() function. This override will then also be used for fattening automatically. Of course, we'd encourage you to submit a pull request to improve our definition for the benfit of everybody.

Namespaces

Shell script doesn't have namespaces. shellfire does. And by the simple way folks have done it in C for generations. With an underscore and a prefix.

Functions are namespaced, that is, they are prefixed by a hierarchial names separated by underscores, '_', so they don't collide. Wherever possible, all the functions for a particular namespace are in a file named after the namespace, in a folder path related to it. For instance, the function core_variable_array_initialise() is in the namespace core_variable_array and is located in the file core/variable/array.functions.

To use functions belonging to a namespace, all one has to do is use it, eg core_usesIn core/variable array. If there is only a single hierarchy, as there is for some modules (eg [urlencode]), then the syntax is core_usesIn urlencode. This logic automatically finds and loads the functions, and their dependencies, and so on. It can't enter a circular loop, either. And when a shellfire application is fattened (by which we make a complete standalone shell script from all these namespaces), only those functions so loaded are included.

A very small number of user definable definitions (see next section) are not in any namespace. These are 'namespaceless'.

If a variable or function starts with an underscore, _, then it is intended that its use is private to that namespace. The exception to this is _program, which is used for user definable definitions.

User Definable Definitions

When a global constant or function starts _program_, it is intended to be defined by the shellfire application outside of the _program() function. Most global functions are optional; defaults are assumed. Most global constants need to be defined. The shellfire tutorial 'overdrive' shows all the common settings. Each namespace adds _program_ definitions as _program_<namespace>_. Some _program_ definitions are outside of any namespace - these are 'namespaceless'.

A few criticial functions aren't where you'd expect

Some functions in core are so important that they are used to bootstrap a shellfire application; this is called init. These functions live in init.functions. However, many of them are also useful during the runtime of a shellfire application, and naturally belong to a specific namespace, eg the function core_variable_isSet() logically belongs to core_variable. Where this is the case, the corresponding file, in this case core/variable/variable.functions will contain a one-line entry core_init_defines core_variable_isSet documenting the fact.

These are those functions:-

`core_compatibility`

core_compatibility_basename()
core_compatibility_which()
core_compatibility_whichNoOutput()

`core_variable`

core_variable_isSet()
core_variable_isUnset()
core_variable_indirectValue()
core_variable_unset()

`core_dependency`

core_dependency_declares()
core_dependency_declaresAsArray()
core_dependency_requires()
core_dependency_oneOf()
core_dependency_fallback()

`core_terminal`

core_terminal_ansiSupported()
core_terminal_ansiSequence()
core_terminal_tput()
core_terminal_colour()
core_terminal_effect()
core_terminal_reset()

`core`

core_message()
core_exitError()
core_TODO()
core_uses()
core_usesIn()

Importing

To import this module, add a git submodule to your repository. From the root of your git repository in the terminal, type:-

mkdir -p lib/shellfire
cd lib/shellfire
git submodule add "https://github.com/shellfire-dev/core.git"
cd -
git submodule init --update

You may need to change the url https://github.com/shellfire-dev/core.git above if using a fork.

You will also need to add paths - include the module paths.d.

Namespaceless

User-Definable Constants

User-Definable constants are intended to be implemented in the global scope of your shellfire application. Most of these constants are mandatory.

User Definable Constant	Optional?	Purpose	Example	Notes
`_program_name`	No	Name of your program once fattened	`_program_name='overdrive'`	Ideally, make this match the file name
`_program_version`	No	Version of your program once fattened	`_program_name='unversioned'`	Always specify `unversioned`
`_program_package_or_build`	No	branch or package variant details	`_program_package_or_build=''`	Always specify `''`. Used by custom packagers.
`_program_copyrightAndLicenseStatement`	Yes	Copyright and license statement. Warranties, disclaimers and authorship.	May include line feeds.	fattening will embed your `COPYRIGHT` file if not specified.
`_program_path`	No	Location of `_program_*Path` sub paths	`_program_path="$([ "${_program_fattening_program_path+set}" = 'set' ] && printf '%s\n' "$_program_fattening_program_path"
`_program_libPath`	No	Location of `lib`	`_program_libPath="${_program_path}/lib"`
`_program_etcPath`	No	Location of `etc`	`_program_etcPath="${_program_path}/etc"`
`_program_varPath`	No	Location of `var`	`_program_varPath="${_program_path}/var"`
`_program_entrypoint`	No	Entry point for your program	`_program_entrypoint='overdrive'`	Ensure a function `overdrive()` exists inside `_program()`
`_program_namespace`	Yes	Namespace of your program's functions	`_program_entrypoint='overdrive'`	Defaults to `_program_name`, but needed if this contains anything other than A-Z, a-z, 0-9 and underscore (eg hyphens)
`_program_arrayDelimiter`	Yes	Override the array delimited in use to support older shells	`_program_arrayDelimiter=','`	Defaults to `\r`. Override with `,` to support MinGW / MSYS Bash 3.1.7 and GoW. Use "$(printf '\001')" if supporting bash 4.2+, ash or dash and you need to deal with data containing `\r`.
`_program_ignoreDependencies`	Yes	Disables the automatic dependency installation framework	`_program_ignoreDependencies=yes`	This can also be done by fattening.

User-Definable Functions

`_program()`

Mandatory. Define all your functions inside this, and any calls to core_usesIn(), core_dependency_requires, etc. Your entry point function (see above section User-Definable Constants, _program_entrypoint) should be in here.

Namespace `core`

These are helper functions to manipulate files character-by-character in shells that don't support a sophisticated read.

To use in code

This namespace is included by default. No additional actions are required. All of these functions are actually defined in core/init.functions.

Functions

`core_message`

Parameter	Value	Optional
`messageKind`	Category of the message	No
`message`	Text of the message	No

Writes message to standard out, with a prefix of messageKind. Colour-codes the message if standard out is a supported terminal. Does not write the message if the messageKind is not suitable for the current verbosity. Valid values of messageKind are:-

`messageKind`	Written When
`FAIL`	Always written
`WARN`	Always written
`NOTICE`	`core_init_verbosity()` is at least `1`
`INFO`	`core_init_verbosity()` is at least `2`
`TODO`	`core_init_verbosity()` is at least `2`
`DEBUG`	`core_init_verbosity()` is at least `3`

If messageKindis unrecognised the message is always written.

`core_exitError`

Parameter	Value	Optional
`exitCode`	Non-zero exit code of the application, ideally one of the constants `core_commandLine_exitCode_*`	No
`message`	Text of the message	No

Writes message as a messageKind of FAIL and then exits the application with exit code exitCode.

`core_TODO`

Parameter	Value	Optional
`message`	Text of the message	No

Writes message as a messageKind of TODO. Useful for documenting incomplete features and being reminded when the application runs.

`core_uses`

Parameter	Value	Optional
`parentNamespace`	Absolute path containing functions to load	No

or

Parameter	Value	Optional
`parentNamespace`	Root namsepace	No
`…`	Zero or more relative namespaces to load, specified as relative paths if nested, with no leading `/`. Should not end in `.functions`.	Yes

This function should only be called from global scope or immediately inside the function _program().

Examples:-

# Load the `urlencode` namespace from lib/shellfire/urlencode
core_usesIn urlencode
# Load the `urlencode_template` namespace from lib/shellfire/urlencode
core_usesIn urlencode template
# Load the `github_api_v3_releases` namespace
core_usesIn github/api/v3 releases
# Or, if two namespaces are wanted
core_usesIn github/api v3 v3/releases

`core_uses`

Parameter	Value	Optional
`libPath`	Absolute path containing functions to load	No
`libraryName`	A folder that should exist under `libPath`	No
`…`	Zero or more libraries to dynamically load as plug ins; specified as relative paths if nested, with no leading `/`. Should not end in `.functions`.	Yes

This is an advanced function for implementing which can be used to implement run-time plugin loading. Handles circular dependencies. Can cause havoc; use with care.

Namespace `core_base64_decode`

This namespace allows you to base64 decode strings and files in a cross-script, cross-distro way irrespective of the foibles of whatever base64 programs (or none) you may have.

To use in code

This namespace is included by default.

Functions

`core_base64_decode_string`

Parameter	Value	Optional
`string`	Base64 encoded string to decode.	No
`decodedFilePath`	Where to write decoded string to.	No
`append`	Boolean; `yes` to append to `decodedFilePath`, `no` to overwrite.	No
`index62Character`	Character for index 62 in base64. Typically `+`	No
`index63Character`	Character for index 63 in base64. Typically `\`	No

Decoding to a file, decodedFilePath, sees counter-intuitive but is required because base64 encoded data can contain embedded ASCII NUL characters, which are incorrectly handled by most shells. index*Character allows you to use different base64 alphabets, such as base64url.

`core_base64_decode_file`

Parameter	Value	Optional
`encodedFilePath`	Base64 encoded file to decode.	No
`decodedFilePath`	Where to write decoded string to.	No
`append`	Boolean; `yes` to append to `decodedFilePath`, `no` to overwrite.	No
`index62Character`	Character for index 62 in base64. Typically `+`	No
`index63Character`	Character for index 63 in base64. Typically `\`	No

Decodes encodedFilePath to decodedFilePath. index*Character allows you to use different base64 alphabets, such as base64url.

Namespace `core_commandLine`

This namespace is mostly invisible, but provides a very rich and sophisticated command line parser so that you can use mixed short and long options, varargs, optional arguments and more. It provides semi-automatic help message generation, including licensing information. It does not depend on any external getopt programs.

All of its behaviour is controlled by defining _program_* functions and setting _program_* variables in your shellfire application. By default, it includes several common options for --help, --verbose and --version use and parsing. Options can be 'golfed' (eg -abc4 is supported, and can be interpreted as -a bc4, -a -b -c -4 and -a -b c4, etc, depending on what's defined) but this isn't recommend as it's ambiguous. XFree86-style (eg as used by openssl) options with a single hyphen aren't supported. -- for end-of-options is, too.

Command line parsing happens after configuration has been loaded, so that command line options can override configured values. Implementations can take advantage of this to note if a value was set by configuration or command line when issuing error messages.

To use in code

This namespace is included by default.

Global Constants

Exit Codes

This constants provide values for common exit conditions, and align with the BSD sysexits.h values, stdlib.h values and a value used by bash.

Constant	Value	Explanation
core_commandLine_exitCode_OK	0	successful termination (aka EXIT_SUCCESS in stdlib.h)
core_commandLine_exitCode_FAILURE	1	Not in sysexist.h, but so commonly used (aka EXIT_FAILURE in stdlib.h)
core_commandLine_exitCode_MISUSEBUILTIN	2	Not in sysexits.h, misuse of builtin (bash specific)
core_commandLine_exitCode_USAGE	64	command line usage error
core_commandLine_exitCode_DATAERR	65	data format error
core_commandLine_exitCode_NOINPUT	66	cannot open input
core_commandLine_exitCode_NOUSER	67	addressee unknown
core_commandLine_exitCode_NOHOST	68	host name unknown
core_commandLine_exitCode_UNAVAILABLE	69	service unavailable
core_commandLine_exitCode_SOFTWARE	70	internal software error
core_commandLine_exitCode_OSERR	71	system error (e.g., can't fork)
core_commandLine_exitCode_OSFILE	72	critical OS file missing
core_commandLine_exitCode_CANTCREAT	73	can't create (user) output file
core_commandLine_exitCode_IOERR	74	input/output error
core_commandLine_exitCode_TEMPFAIL	75	temp failure; user is invited to retry
core_commandLine_exitCode_PROTOCOL	76	remote error in protocol
core_commandLine_exitCode_NOPERM	77	permission denied
core_commandLine_exitCode_CONFIG	78	configuration error

These constants are not readonly, as this doesn't work in some ksh derivatives.

Functions

There are no public functions.

User-Definable Functions

User-Definable functions are intended to be implemented in the global scope of your shellfire application. You are not obligated to implement any of these functions; if you don't, a naive default is used instead.

`_program_commandLine_parse()`

Parameter	Value	Optional
`"$@"`	Zero or More	Yes

Implement this function if you want to completely bypass the POSIX compliant command line parsing. The functions _program_commandLine_parseInitialise(), _program_commandLine_helpMessage(), _program_commandLine_optionExists(), etc will not be called (unless your code in this function calls them).

Reasons to implement this function could be:-

To support scripts that just take arguments as part of an existing API (eg Heroku Buildpack Functions)
To support scripts that take an initial command argument, like git
- In this case, you can re-use the POSIX command line parsing logic, eg

	_program_commandLine_parse()
	{
		mycommand="$1"
		shift 1
		
		case "$mycommand" in
			
			update|delete)
				:
			;;
			
			*)
				core_commandLine_exitBadCommandLine "Unknown command '$mycommand'"
			;;
		esac
		
		core_commandLine_parsePosix "$@"
	}
	
	# Sample implementation
	_program_commandLineOptionExists()
	{
		case "$mycommand" in
			
			update)
				case "$optionName" in
		
					# c would be -c
					# clean would be --clean
					c|clean)
						echo yes-argumentless
					;;
		
					# eg -t mytunnel
					# eg --tunnel=mytunnel
					# eg --tunnel mytunnel
					# If there are additional equals signs, they're handled correctly, eg
					# eg --tunnel=mytunnel=something
					t|tunnel)
						echo 'yes-argumented'
					;;
		
					tunnel-tls-use-der)
						echo 'yes-optionally-argumented'
					;;
		
					*)
						echo 'no'
					;;
		
				esac
			;;
			
			delete)
				case "$optionName" in
				
					...
				
				esac
			
			;;
			
		esac	
	}

`_program_commandLine_parseInitialise()`

Parameter	Value	Optional

No parameters.

Implement this function if you need to define some variables or perform any actions before the command line is parsed. A typical use case is to define default values to be used if an option isn't specified and isn't configured. It can also be used to set any variables that wold be used in generated help message (see below).

`_program_commandLine_helpMessage()`

Parameter	Value	Optional

No parameters.

Implement this function to argment the help message that is displayed when a user asks for help (-h or --help), or a command line parse error occurs. When this function is executed by the parser, you can set some _program_commandLine_helpMessage_* variables that can then be used to provide a richer help:-

Variable	Use	Example
`_program_commandLine_helpMessage_usage`	A one line usage message	`[OPTION]... -- [SCRIPTLETS]...`
`_program_commandLine_helpMessage_description`	Multi-line description	`Connects to a MQTT server and runs SCRIPTLETS.\nMay be used as a command interpreter in scripts starting #!/usr/bin/env ${_program_name}\nif on the PATH.`*
`_program_commandLine_helpMessage_options`	Multi-line options	See below†
`_program_commandLine_helpMessage_optionsSpacing`	Space characters to align things
`_program_commandLine_helpMessage_configurationKeys`	Multi-line configuration settings	See below‡
`_program_commandLine_helpMessage_examples`	A one line example usage	`${_program_name} --server test.mosquitto.org`

Note that using variables such as ${_program_name} allows you to change your program name in the future without search-replace fury.

* The '\n represents a line feed; in shell script, you'd write:-

_program_commandLine_helpMessage_description="Connects to a MQTT server and runs SCRIPTLETS.
May be used as a command interpreter in scripts starting #!/usr/bin/env ${_program_name}
if on the PATH."

† Multi-line options:-

_program_commandLine_helpMessage_options="
-t, --tunnel TUNNEL       Network tunnel TUNNEL to use. One of 'none', 'tls',
                        'cryptcat'. Defaults to '${_program_default_tunnel}'.
                        'WebSockets', 'WebSocketsSecure', 'SSH' and 'telnet'
                        are unimplemented but possible if there's demand.
-s, --server HOST         Server hostname, HOST, is a hostname,
                        IPv4 address, IPv6 address, unix domain socket
                        path or serial character device path."

‡ Multi-line configuration settings:-

_program_commandLine_helpMessage_configurationKeys="
  bishbosh_tunnel         Equivalent to --tunnel
  bishbosh_server         Equivalent to --server
  bishbosh_port           Equivalent to --port
"

`_program_commandLine_optionExists()`

Parameter	Value	Optional

No parameters.

Implement this function to tell the command line parser that a short or long option exists, and, if it does, what kind of argument it takes. The variable optionName is in scope, and contains the command line option without any leading hyphens and without any trailing spaces or equals sign, =. This function is usually implemented using a case statement.

The function should write to standard out one of four values:-

Value	Description
`yes-argumentless`	Option exists but does not take an argument (`--help` would be an example)
`yes-argumented`	Option exists but requires an argument
`yes-optionally-argumented`	Option exists but may have an argument. Use this with care, as it can lead to ambiguous results.
`no`	Option doesn't exist.

The options h, help, v, verbose, version are handled for you and can not be overridden.

An example illustrates best:-

_program_commandLine_optionExists()
{
	case "$optionName" in
		
		# c would be -c
		# clean would be --clean
		c|clean)
			echo yes-argumentless
		;;
		
		# eg -t mytunnel
		# eg --tunnel=mytunnel
		# eg --tunnel mytunnel
		# If there are additional equals signs, they're handled correctly, eg
		# eg --tunnel=mytunnel=something
		t|tunnel)
			echo 'yes-argumented'
		;;
		
		tunnel-tls-use-der)
			echo 'yes-optionally-argumented'
		;;
		
		*)
			echo 'no'
		;;
		
	esac
}

`_program_commandLine_processOptionWithoutArgument()`

Parameter	Value	Optional

No parameters.

Implement this function to be able to act on an option you specified as either yes-argumentless or yes-optionally-argumented (and which had no argument). The variable optionName is in scope. For example:-

_program_commandLine_processOptionWithoutArgument()
{
	case "$optionName" in
		
		r|random-client-id)
			bishbosh_randomClientId=1
		;;

		tunnel-tls-use-der)
			bishbosh_tunnelTlsUseDer=1
		;;
		
		tunnel-tls-verify)
			bishbosh_tunnelTlsVerify=1
		;;
		
	esac
}

There is not need for a default * case as it can not occur.

`_program_commandLine_processOptionWithArgument()`

Parameter	Value	Optional

No parameters.

Implement this function to be able to act on an option you specified as either yes-argumentless or yes-optionally-argumented (and which had no argument). The variable optionName is in scope. For example:-

_program_commandLine_processOptionWithoutArgument()
{
	case "$optionName" in
		
		r|random-client-id)
			bishbosh_randomClientId=1
		;;

		tunnel-tls-use-der)
			bishbosh_tunnelTlsUseDer=1
		;;
		
		tunnel-tls-verify)
			bishbosh_tunnelTlsVerify=1
		;;
		
	esac
}

There is not need for a default * case as it can not occur.

May be called more than once for a given optionName if it is present more than once on the command line.

`_program_commandLine_processOptionWithArgument()`

Parameter	Value	Optional

No parameters.

Implement this function to be able to act on an option you specified as yes-argumented. The variables optionName and optionValue are in scope. For example:-

_program_commandLine_processOptionWithArgument()
{
	case "$optionName" in
		
		p|port)
			core_validate_nonDynamicPort $core_commandLine_exitCode_USAGE 'option' "$optionNameIncludingHyphens" "$optionValue"
			bishbosh_port="$optionValue"
		;;
		
		t|tunnel)
			bishbosh_validate_tunnel $core_commandLine_exitCode_USAGE 'option' "$optionNameIncludingHyphens" "$optionValue"
			bishbosh_tunnel="$optionValue"
		;;
		
		s|server)
			bishbosh_validate_address $core_commandLine_exitCode_USAGE 'option' "$optionNameIncludingHyphens" "$optionValue"
			bishbosh_server="$optionValue"
		;;
		
	esac
}

There is not need for a default * case as it can not occur. Note the use of the core_validate namespace and a private ${_program_name}_validate namespace.

May be called more than once for a given optionName if it is present more than once on the command line.

`_program_commandLine_handleNonOptions()`

Parameter	Value	Optional
`…`	Zero or more non options	Yes in the sense that there may be zero

The command line parse calls this function once for the gathered set of non-options (ie all the values after -- in the command line or when option parsing has ended) in the command line.

Use the value as necessary; you can add them to a cross-shell compliant array with core_variable_array_append my_non_options_array "$@".

This example treats the non options as shell-source:-

_program_commandLine_handleNonOptions()
{
	local scriptletFilePath
	local suitableForSourceBuiltInFilePath
	for scriptletFilePath in "$@"
	do
		suitableForSourceBuiltInFilePath=$(core_compatibility_dirname "$scriptletFilePath")/"$(core_compatibility_basename "$scriptletFilePath")"
		. "$suitableForSourceBuiltInFilePath" || core_exitError "Could not source SCRIPTLET '$scriptletFilePath'"
	done
}

`_program_commandLine_validate()`

Parameter	Value	Optional

No parameters.

The command line parser calls function after all parsing is complete. Use this function to check for things like:-

a mandatory command line option has been passed
a given combination of options is valid
an option wasn't passed, but was set in configuration, so now we should validate the configuration value

Typically, this logic makes calls to functions in the core_validate namespace. For example:-

_program_commandLine_validate()
{
	if core_variable_isUnset bishbosh_sessionPath; then
		# _program_default_sessionPath might be specified in _program_commandLine_parseInitialise if it is a computed value
		bishbosh_sessionPath="$_program_default_sessionPath"
		core_validate_folderPathReadableAndSearchableAndWritable $core_commandLine_exitCode_CONFIG 'defaulted value for' '--session-path' "$bishbosh_sessionPath"
	else
		core_validate_folderPathReadableAndSearchable $core_commandLine_exitCode_CONFIG 'configuration setting' 'bishbosh_sessionPath' "$bishbosh_sessionPath"
	fi
}

Namespace `core_children`

This namespace exists to provide simple management of child processes.

To use in code

This namespace is included by default.

Functions

`core_children_killOnExit()`

Parameter	Value	Optional
`…`	Zero or more PIDs of child processes to kill on application exit	Yes

After starting a background process, use this function to register child processes to clean up (kill) on exit using a TERM signal. For example:-

(
	sleep 1
	core_compatibility_echo 'tick-tock'
) &
core_children_killOnExit $?

Namespace `core_compatibility`

This namespace exists to ensure cross-shell and cross PATH compatibility. It creates the functions pushd and popd if they don't exist, and, if they do, redirects their standard out to /dev/null.

To use in code

This namespace is included by default.

Functions

`core_compatibility_basename()`

Parameter	Value	Optional
`path`	File path	No

Writes the file portion of path to standard out. If path is just an unqualified string (eg my-file), writes my-file. If path is /path/to/file, writes file. Lastly, if file is path/to/file, writes file. Does not rely on the basename executable being on the PATH.

`core_compatibility_dirname()`

Parameter	Value	Optional
`path`	File path	No

Writes the directory portion of path to standard out. If path is just an unqualified string (eg my-file), writes ., otherwise writes everything before the final /. Does not rely on the dirname executable being on the PATH.

`core_compatibility_which()`

Parameter	Value	Optional
`executable`	executable	No

Acts like the which external command, but does not rely on it. Writes to standard out the full path to executable if it exists, otherwise writes to standard out executable if it is a function or built in. Returns an exit code of 0 if the executable exists (or is a function or built in) or returns 1 if not. Does not rely on the which executable being on the PATH.

`core_compatibility_whichNoOutput()`

Parameter	Value	Optional
`executable`	executable	No

Acts like core_compatibility_which() but writes nothing to standard out.

`core_compatibility_echo()`

Parameter	Value	Optional
`message`	string	No

Writes message followed by new line. Escape sequences in message are not acted on.

`core_compatibility_sleepSupportsFractionalSeconds()`

Parameter	Value	Optional

No parameters.

Returns an exit code of 0 if fractional values are accepted by sleep, otherwise returns 1. Most modern versions of sleep support fractional seconds, except on AIX (whose user binaries are horrendously dated - and IBM has the gall to charge for it, too)!

Namespace `core_configuration`

A shellfire application can have any of its command line options set in configuration files. These can be monolithic or composed of fragments (like Debian's run-parts). Or both. And they can exist in many locations. The approach lets an administrator of your application manage settings per-machine, per-user and per-environment; and they can still override settings per-run using command line options. Using fragments, sensitive passwords or credentials can be managed separately to more mundance configuration, so minimising failure when moving from development to production, say, or making it difficult to accidentally check in sensitive data to source control.

Nearly all the functions in this namespace are used internally to configure your shellfire application. The only one you need to know about is core_configuration_blacklist, which lets you state that an environment variable path is no longer to be overridden. And, if paranoid, you can also use readonly to prevent configuration settings from being changed.

Configuration files are just regular shell script, although it's advisable to avoid doing anything more mundane than:-

myapp_inputPath="/path/to/input"
myapp_outputPath="/path/to/output"
# etc

# But you can also blacklist, to prevent these locations being used
core_configuration_blacklist HOME SHELLFIRE_RC SHELLFIRE_RC_D

Configuration Locations

Anything you can do with a command line switch, you can do as configuration. But configuration can also be used with scripts. Indeed, the configuration syntax is simply shell script. Configuration files should not be executable. This means that if you really want to, you can let an administrator override just about any feature or behaviour of a program - although that's not explicitly supported. Configuration can be in any number of locations. Configuration may be a single file, or a folder of files; in the latter case, every file in the folder is parsed in 'shell glob-expansion order' (typically ASCII sort order of file names). Locations are searched in order as follows:-

Global (Per-machine)
The file INSTALL_PREFIX/etc/shellfire/rc where INSTALL_PREFIX is where ${_program_name} has been installed.
Any files in the folder INSTALL_PREFIX/etc/shellfire/rc.d
The file INSTALL_PREFIX/etc/${_program_name}/rc where INSTALL_PREFIX is where ${_program_name} has been installed.
Any files in the folder INSTALL_PREFIX/etc/${_program_name}/rc.d
Per User, where HOME is your home folder path*
The file HOME/.shellfire/rc
Any files in the folder HOME/.shellfire/rc.d
The file HOME/.${_program_name}/rc
Any files in the folder HOME/.${_program_name}/rc.d
Per Environment
The file in the environment variable shellfire_RC (if the environment variable is set and the path is readable)
Any files in the folder in the environment variable shellfire_RC_D (if the environment variable is set and the path is searchable)
The file in the environment variable ${_program_name}_RC (if the environment variable is set and the path is readable)
Any files in the folder in the environment variable ${_program_name}_RC_D (if the environment variable is set and the path is searchable)

Nothing stops any of these paths, or files in them, being symlinks.

* An installation as a daemon using a service account would normally set HOME to something like /var/lib/bishbosh.

To use in code

This namespace is included by default. No additional actions are required.

Functions

`core_configuration_blacklist`

Parameter	Value	Optional
`…`	Zero or more environment variables to blacklist	Yes

Use this function inside a configuration file (such as a per-machine one like INSTALL_PREFIX/etc/${_program_name}/rc above) and specify any environment variables that should not be used to load configuration from. This prevents an user overriding machine-level configuration. The environment variables you can blacklist are:-

HOME
shellfire_RC
shellfire_RC_D
${_program_name}_RC
${_program_name}_RC_D

Namespace `core_dependency`

This namespace is mostly invisible but provides a powerful framework to:-

automatically install dependent executables (in packages) using distro-specific package managers
modify the PATH so that known-good executables are used
document global variables that need to be included in any fattened shellfire application.
document complex requirements

To use in code

This namespace is included by default.

Functions

`core_dependency_declares()`

Parameter	Value	Optional
`…`	Zero or more global variables in to include in a fattened program	Yes

The processing of fattening a program needs to know if any global variables are required. This function should only be called from global scope or immediately inside the function _program() with the names of any variables. Commonly, this done immediately after declaring and setting a global variable (or constant). This is how the core_commandLine namespace exposes global constants for exit codes. For example:-

# command line usage error
core_commandLine_exitCode_USAGE=64
core_dependency_declares core_commandLine_exitCode_USAGE

`core_dependency_declaresAsArray()`

Parameter	Value	Optional
`…`	Zero or more global array variables in to include in a fattened program	Yes

This function operates like core_dependency_declares() above, but is for array variables. It exists because the declaration of a global array that needs to be included in a fattened program can occur before the array is defined.

`core_dependency_requires()`

Parameter	Value	Optional
`packageManager`	Name of a package manager, or `'*` for all known package managers.	Yes
`…`	Zero or more executables	Yes

This function should only be called from global scope or immediately inside the function _program() with the names of any variables.

This function informs the dependency management framework that a cerain executable (binary), as name in …, should be on the PATH. If it isn't, it arranges at application bootstrap time for it to be downloaded and installed. This depends on a .path file in etc/shellfire/paths.d/${packageManager}. For example, for the executable dd on Mac OS X, there should be etc/shellfire/paths.d/Homebrew/dd.path (this ensures that the dd is the same across multiple platforms with the same switches). It is usually used to 'decorate' a function (a bit like a Java annotation), so that there is an obvious and logical relationship to a function's implementation and its external dependencies.

The packageManager denotes if this dependency is needed for all operating environments ('*') or just a particular one (eg Homebrew).

For example:-

core_dependency_requires '*' dd
my_function()
{
	dd if=/path/to/input/file of=/path/to/output/file
}

Using this approach, it becomes very easy to find all your programs dependencies and document them, even if you don't use the automatic installation process. A veru useful thing to do it you're looking to fine tune your BusyBox configuration, say.

`core_dependency_fallback()`

Design subject to change; do not use. Explicitly not fully documented. Intended to provide a way to document fallbacks.

`core_dependency_oneOf()`

Design may be subject to change; use with caution. Explicitly not fully documented. Intended to provide a way to document alternatives.

Namespace `core_file`

These are helper functions to manipulate files character-by-character in shells that don't support a sophisticated read.

To use in code

This namespace is included by default. No additional actions are required.

Functions

`core_file_characterByCharacter`

Parameter	Value	Optional
`filePath`	A file to read character-by-character	No
`…`	Zero or more function names to use as callbacks	Yes

Iterates over a file character-by-character. For each character, it calls each callback in … with the variable character set to the character. If the character is a line feed, then character is empty (this is a shell limitation, but not a problem in use).

`core_file_characterByCharacterCreate`

Parameter	Value	Optional
`inputFile`	A file to read character-by-character	No
`outputFile`	A file to read character-by-character	No

Converts a file into a set of lines (line feed delimited), each of which contains one character (or, in the case of line feed, no character).

Namespace `core_functions`

These are helper functions to manipulate shell functions.

To use in code

This namespace is included by default. No additional actions are required.

Functions

`core_functions_exist`

Parameter	Value	Optional
`functionName`	A name of a function	No

Returns an exit code of 0 if the function (not binary on the PATH) exists in the current program, and 1 if it doesn't.

`core_functions_register`

Parameter	Value	Optional
`functionsVariableName`	A variable to use to store the function names	No
`…`	Zero or more function names	Yes

Appends zero or more function names to use as callbacks, in order. A function can be added more than once. The list of functions will be included in any fattened binary. Use this to register callbacks and plugins on first load of modules. This function should only be called from global scope or immediately inside the function _program(). functionsVariableName is created and initialised as an array if it doesn't exist.

`core_functions_execute`

Parameter	Value	Optional
`functionsVariableName`	A variable to use to store the function names	No
`…`	Zero or more arguments to pass to callbacks	Yes

This function executes all functions registered in functionsVariableName as callbacks, passing the value of … as arguments to them.

Namespace `core_init`

These are mostly functions used internally.

To use in code

This namespace is included by default. No additional actions are required.

Functions

`core_init_defines`

Parameter	Value	Optional
`…`	Zero or more functions or global constants	Yes

This function should only be called from global scope or immediately inside the function _program(). It is used to document that a function or global constant, that should be found inside a functions file, is actually to be found in init.functions.

`core_init_verbosity`

Parameter	Value	Optional

No parameters.

This function writes to standard out a positive integer number representing the current verbosity level set by --verbose for a shellfire application. It is typically used to decide if to output debugging information, eg

if [ "$(core_init_verbosity)" -gt 2 ]; then
	cat "/path/to/some/file" 1>&2
fi

If using core_message(), then this is checked internally - there's no need to guard calls to core_message() with this function.

`core_init_language`

Parameter	Value	Optional

No parameters.

This function writes to standard out the current language setting, or en_US.UTF-8 if there isn't one. Useful if you need to manipulate language sensitive programs which don't use LC_* or LANG variables (which we set automatically).

Namespace `core_path`

These are utility functions that check for certain properties of paths (files, folders, that sort of thing). Unlike the core_validate namespace functions, these ones do not exit the application. Instead, they return an exit code, and so are intended to be used from constructs such as if statements or &&.

For example, to check if a path is a readable, searchable folder path:-

if ! core_path_isReadableAndSearchableFolderPath '/etc'; then
	core_message WARN "'/etc' is not a readable, searchable folder path. This is very odd."
fi

To use in code

This namespace is included by default. No additional actions are required.

Functions

`core_path_isReadableFilePath()`

Parameter	Value	Optional
`filePath`	Path to a potential file	No

Returns an exit code of 0 if filePath is a readable, extant file or 1 if it is not.

`core_path_isReadableNonEmptyFilePath()`

Parameter	Value	Optional
`filePath`	Path to a potential file	No

Returns an exit code of 0 if filePath is a readable, non-empty extant file or 1 if it is not.

`core_path_isReadableNonEmptyExecutableFilePath()`

Parameter	Value	Optional
`filePath`	Path to a potential file	No

Returns an exit code of 0 if filePath is a readable, non-empty executable extant file or 1 if it is not.

`core_path_isReadableAndSearchableFolderPath()`

Parameter	Value	Optional
`folderPath`	Path to a potential folder	No

Returns an exit code of 0 if folderPath is a readable, searchable extant folder or 1 if it is not.

`core_path_isReadableAndSearchableAndWritableFolderPath()`

Parameter	Value	Optional
`folderPath`	Path to a potential folder	No

Returns an exit code of 0 if folderPath is a readable, searchable and writable extant folder or 1 if it is not.

Namespace `core_pipefail`

These functions exist to allow non-bash shells to run pipelines as if set -o pipefail was used, and so:-

Get an exit code of 1 if any process in the pipeline fails;
Find the exit codes of each command in the pipeline

For example, to run three commands in a pipeline:-

set +e
core_pipefail_execute head -n 4 /path/to/file \| tail -n 2 | grep -q 'something'
exitCode=$?
set -e

if [ $exitCode -eq 1 ]; then
	printf '%s\n' 'Pipeline failed"
	printf 'Exit code for head: %s\n' $core_pipefail_PIPESTATUS_1
	printf 'Exit code for tail: %s\n' $core_pipefail_PIPESTATUS_2
	printf 'Exit code for grep: %s\n' $core_pipefail_PIPESTATUS_3
fi

Please note that the variables core_pipefail_PIPESTATUS_* are global. Nested pipelines, etc, will not work. Additionally, this function uses file descriptors 3 and 4.

To use in code

This namespace is included by default. No additional actions are required.

Functions

`core_pipefail_execute()`

Parameter	Value	Optional
`Pipeline`	Pipeline command, with pipes replaced by \\|	Yes

Returns an exit code of 0 if pipeline succeeded or 1 if it did not. Sets variables core_pipefail_PIPESTATUS_N, where N is one-based for each command in the pipeline.

Namespace `core_snippet`

This namespace allows you to embed plain text and binary snippets inside shell scripts - a bit like shell archives. Snippets can be retrieved and used as temporary files, variable values or even interpreted as here docs (they make excellent simple templates)! Snippets let you separate functionality from data, and so source control and version data separately. They also let you deploy unusual compiled binaries with your code - just make sure you've statically linked everything…

Binary snippets are base64 encoded. Plain text snippets are embedded as is (they are called raw), although trailing line feeds are stripped on retrieval due to fundamental shell limitations - watch out! For advanced use cases, it is possible to define your own storage codec. Perhaps Ascii85? If so, please contribute a patch if others could benefit.

Snippets are stored in lib/shellfire/${_program_name} as files ending in .snippet. File names can consist only of A-Z, a-z, 0-9 and underscores to maximise cross-shell compatibility. For a _program_name=example, you might use a snippet as follows:-

core_snippet_embed raw my_snippet
example_useMySnippet()
{
	# Extract to a file
	# 'no' here means do not append. Specify 'yes' to append to the end of /path/to/extract/to
	core_snippet_retrieve my_snippet 'no' /path/to/extract/to
	
	# Extract to a temporary file
	local TMP_FILE
	core_temporaryFiles_newFileToRemoveOnExit
	local mySnippetTemporaryFilePath="$TMP_FILE"
	core_snippet_retrieve my_snippet 'no' "$mySnippetTemporaryFilePath"
	
	# Use as a heredoc - make sure any functions $(somefunc) and variables ${somevar} are defined, or your program will exit!
	# This allows you to store templates as snippets
	core_snippet_retrieveAndSourceAsHereDoc my_snippet
}

By convention, the embed statement 'decorates' the function that'll use the snippet. The codec here is raw - it could also be base64.

To use in code

This namespace is included by default. No additional actions are required.

Functions

`core_snippet_embed()`

Parameter	Value	Optional
`codecName`	`raw` or `base64`	No
`…`	Zero or more `snippetName` snippet names	Yes, although at least one is needed to be useful

You must call this function only from global scope or immediately inside the function _program(). If you define your own codec, then you must define the functions core_snippet_${codecName}_encoder and core_snippet_${codecName}_decoder. These do not take arguments, but the variables snippetName and snippetFilePath are in scope (and snippetAppend, a boolean, for core_snippet_${codecName}_decoder).

`core_snippet_retrieve()`

Parameter	Value	Optional
`snippetName`	Name of the snippet used in `core_snippet_embed()`	No
`snippetAppend`	Specify `no` to overwrite any existing file or `yes` to append to the end.	No
`snippetFilePath`	Where to write the snippet to.	No

Retrieves a snippet and writes it to snippetFilePath. If the raw codec was used when embedding the snippet with core_snippet_embed(), then the resultant data will have any trailing line feeds stripped and ASCII NUL characters will have been ignored.

`core_snippet_retrieveAndSourceAsHereDoc()`

Parameter	Value	Optional
`snippetName`	Name of the snippet used in `core_snippet_embed()`	No

Retrieves a snippet and treats it as if it was a here doc (ie <<EOF). If the raw codec was used when embedding the snippet with core_snippet_embed(), then the resultant data will have any trailing line feeds stripped and ASCII NUL characters will have been ignored.

Namespace `core_temporaryFiles`

This namespace exposes two simple functions to manage temporary files and folders. It abstracts away all the variability of different operating systems, BSDs and Linux distros. Under the covers, it handles signals and manages arrays so that temporary resources are always cleaned up, no matter what. Of course, you're still free to rm a file when you're done with it - a good idea if you need many temporary files.

To use in code

This namespace is included by default. No additional actions are required.

Functions

`core_temporaryFiles_newFileToRemoveOnExit()`

Parameter	Value	Optional

No parameters.

On return, sets the variable TMP_FILE to the path to an extant temporary file with no content. Uses the most secure temporary file approach available, with as random-as-possible names (securely so on Linux, Mac OS X and FreeBSD), and restricted file permissions (writable and readable only by the current user).

Ideally, you should make TMP_FILE local and then copy the value immediately, as a subsequent call to core_temporaryFiles_newFileToRemoveOnExit() will overwrite it (this can happen if using recursion, as local variables are inheritated in shell script). Example:-

local TMP_FILE
core_temporaryFiles_newFileToRemoveOnExit
ourTemporaryFilePath="$TMP_FILE"

`core_temporaryFiles_newFolderToRemoveOnExit()`

Parameter	Value	Optional

No parameters.

On return, sets the variable TMP_FOLDER to the path to an extant temporary folder. Uses the most secure temporary file approach available, with as random-as-possible names (securely so on Linux, Mac OS X and FreeBSD), and restricted file permissions (writable, readable and searchable only by the current user).

Ideally, you should make TMP_FOLDER local and then copy the value immediately, as a subsequent call to core_temporaryFiles_newFolderToRemoveOnExit() will overwrite it (this can happen if using recursion, as local variables are inheritated in shell script). Example:-

local TMP_FOLDER
core_temporaryFiles_newFolderToRemoveOnExit
ourTemporaryFolderPath="$TMP_FOLDER"

Namespace `core_terminal`

This namespace exposes functions to safely write coloured strings to a terminal on a file descriptor. If a terminal is not present, it outputs plain text without terminal escape sequences, so standard error (or whatever else) can be redirected to a file, logged, etc. It has a small list of known ANSI VT102 (ECMA-48) compliant terminals to which it outputs escape sequences directly. If your terminal isn't listed, it falls back to the tput program, and then to plain text. The odd reason for this counter-intuitive, apparently anti-best-practice behaviour is that tput is frequently missing (especially on BusyBox based systems), broken or incorrectly set up (as is commonly the case, especially on AIX and the BSDs).

If you are using a terminal that supports ANSI escape sequences and isn't supported, please submit a pull request.

To use in code

This namespace is included by default. No additional actions are required. Please note that currently all functions are defined in core/init.functions as they are used during shellfire application bootstrapping.

Functions

`core_terminal_ansiSupported()`

Parameter	Value	Optional

No parameters.

Used to determine if a terminal supports ANSI escape sequences. Only currently used if tput is not available. Modify this function to detect your ANSI terminal type if we don't support it. Returns an exit code of 0 is supported and 1 otherwise.

`core_terminal_tput()`

Parameter	Value	Optional
`…`	One or more tput arguments	No

Defensive wrapping of tput in case it is missing, broken or incompletely installed.

`core_terminal_colour()`

Parameter	Value	Optional
`fd`	File descriptor to check is a terminal. Use `1` for standard out and `2` for standard error.	No
`ground`	What to colour: `foreground` or `background`	No
`colour`	File descriptor to check is a terminal	No

ground may be one of:-

Value	Description
`foreground`	Colour the foreground
`background`	Colour the background

colour may be one of:-

Value
`black`
`red`
`green`
`yellow`
`blue`
`magenta`
`cyan`
`white`
`default`

This functions write an escape sequence to standard out. Either redirect it the file descriptor specified in fd, or capture it in a string for latter use, eg

coloured="My $(core_terminal_colour 2 foreground red)red$(core_terminal_reset) string"
printf '%s\n' "$coloured" 1>&2

`core_terminal_effect()`

Parameter	Value	Optional
`fd`	File descriptor to check is a terminal. Use `1` for standard out and `2` for standard error.	No
`…`	Zero or more effect names	Yes, but specify at least one for to be useful!

ground may be one of:-

Value	Description
`foreground`	Colour the foreground
`background`	Colour the background

effect may be one of:-

Value
`bold`
`dim`
`blink`
`reversed`
`invisible`

This functions write an escape sequence to standard out. Either redirect it the file descriptor specified in fd, or capture it in a string for latter use, eg

affected="My $(core_terminal_effect 2 bold blink)bold, blinking$(core_terminal_reset) string"
printf '%s\n' "$affected" 1>&2

`core_terminal_reset()`

Parameter	Value	Optional
`fd`	File descriptor to check is a terminal. Use `1` for standard out and `2` for standard error.	No

Resets all terminal escape sequences by writing to standard out. Use this as in the examples above.

Namespace `core_trap`

This namespace exposes functions to manage signals (traps). It also sets up and registers internal logic to make clean up of common resources straightforward and error-free.

To use in code

This namespace is included by default. No additional actions are required.

Functions

`core_trap_addOnCleanUp()`

Parameter	Value	Optional
`handler`	The name of a handler function to callback	No

Use this function to register a callback for signals that cause the application to exit, and where a clean up action is needed. There is no need to use this method for temporary file clean up; that's internally registered.

`core_trap_addHandler()`

Parameter	Value	Optional
`handler`	The name of a handler function to callback	No
`…`	Zero or more signal names	Yes, but specify at least one for to be useful!

Use this function to register a callback for zero or more signals. Signal names that are cross-platform are:-

Signal Name
`EXIT`
`HUP`
`INT`
`QUIT`
`ABRT`
`PIPE`
`TERM`
`TSTP`
`USR1`
`USR2`
`INFO`

Namespace `core_umask`

This namespace is used internally to ensure a sensible default umask of 0022 is set.

To use in code

This namespace is included by default. No additional actions are required.

Functions

There are no public functions.

Namespace `core_validate`

This namespace exposes functions that validate a supplied value. If the validation fails, the program exits. The intended use case is for validating command line and configuration arguments at program initialisation. Some shellfire applications supplement these with additional validate functions in their own namespaces; by convention, this is done as ${_program_name}_validate_FUNCTIONNAME in a file ${_program_name}/validate.functions.

All of the functions for validation follow the same pattern, and take the same four arguments: code, category, name and value. For example, to validate that the --output-path command line switch was given a valid folder path, you might do this inside your shellfire application, example:-

_program_commandLine_processOptionWithArgument()
{
	case "$optionName" in
		
		…
		
		o|output-path)
			core_validate_folderPathIsReadableAndSearchableAndWritableOrCanBeCreated $core_commandLine_exitCode_USAGE 'option' "$optionNameIncludingHyphens" "$optionValue"
			example_outputPath="$optionValue"
		;;
		
		…
		
	esac
}

Of course, it'd be nice if users could specify a default --output-path in a configuration file. They can do this by creating, say /etc/example/rc (one of many possibilities) with the content:-

example_outputPath='/var/tmp/example'

You'd then want to be able to validate that the configured value or the value specified on the command line was right. Inside your shellfire application, you could add the following:-

_program_commandLine_validate()
{
	…

	if core_variable_isSet example_outputPath; then
		core_validate_folderPathIsReadableAndSearchableAndWritableOrCanBeCreated $core_commandLine_exitCode_CONFIG 'configuration setting' 'example_outputPath' "$example_outputPath"
	else
		core_message INFO "Defaulting --output-path to current working directory"
		example_outputPath="$(pwd)"
	fi
	
	…
}

This works because we follow a convention: a command-line long option is named ${_program_name}_lowercaseUppercase. If your program was overdrive and you had a long option --number-of-gears, then the option would be overdrive_numberOfGears.

Conventions

`code`

Conventions for code are:-

Never use 0
Avoid 1 and 2
Always use the most apt value from the commandLine interface exit codes (core_commandLine_exitCode_*)
Never use values above 127
Use core_commandLine_exitCode_USAGE if category is option
Use core_commandLine_exitCode_CONFIG if category is configuration setting

`category`

Conventions for category are:-

option for command line options (switches) such as -o and --output-path above
configuration setting for configuration settings such as example_outputPath
argument if being used for validating function arguments (not a common use case)
defaulted value for if being used for a default when no configuration or command line option was set, and the default isn't valid (eg when an option might be a folder path, and the default folder path doesn't exist)

To use in code

This namespace is included by default. No additional actions are required.

Functions

`core_validate_pathNotEmpty()`

Parameter	Value	Optional
`code`	An exit code (should not be zero). Ideally use one of the constants `core_commandLine_exitCode_*`.	No
`category`	What you are validating.	No
`name`	Name of what you are validating, eg `--output-path` or `example_outputPath`	No
`value`	Raw, supplied value of what you are validating	No

Use this function to validate that a path (value) is not empty ('') . Exits with code if validation fails.

`core_validate_folderPathReadableAndSearchable()`

Parameter	Value	Optional
`code`	An exit code (should not be zero). Ideally use one of the constants `core_commandLine_exitCode_*`.	No
`category`	What you are validating.	No
`name`	Name of what you are validating, eg `--output-path` or `example_outputPath`	No
`value`	Raw, supplied value of what you are validating	No

Use this function to validate that a path (value) is a readable, searchable (and so extant) folder. Exits with code if validation fails.

`core_validate_folderPathReadableAndSearchableAndWritable()`

Parameter	Value	Optional
`code`	An exit code (should not be zero). Ideally use one of the constants `core_commandLine_exitCode_*`.	No
`category`	What you are validating.	No
`name`	Name of what you are validating, eg `--output-path` or `example_outputPath`	No
`value`	Raw, supplied value of what you are validating	No

Use this function to validate that a path (value) is a readable, searchable, writable (and so extant) folder. Exits with code if validation fails.

`core_validate_parentFolderPathReadableAndSearchable()`

Parameter	Value	Optional
`code`	An exit code (should not be zero). Ideally use one of the constants `core_commandLine_exitCode_*`.	No
`category`	What you are validating.	No
`name`	Name of what you are validating, eg `--output-path` or `example_outputPath`	No
`value`	Raw, supplied value of what you are validating	No

Use this function to validate that the parent of the path (value) is a readable, searchable (and so extant) folder. Exits with code if validation fails.

`core_validate_parentFolderPathReadableAndSearchableAndWritable()`

Parameter	Value	Optional
`code`	An exit code (should not be zero). Ideally use one of the constants `core_commandLine_exitCode_*`.	No
`category`	What you are validating.	No
`name`	Name of what you are validating, eg `--output-path` or `example_outputPath`	No
`value`	Raw, supplied value of what you are validating	No

Use this function to validate that the parent of the path (value) is a readable, searchable, writable (and so extant) folder. Exits with code if validation fails. Useful for checking that a path that might be created will succeed.

`core_validate_folderPathIsReadableAndSearchableAndWritableOrCanBeCreated()`

Parameter	Value	Optional
`code`	An exit code (should not be zero). Ideally use one of the constants `core_commandLine_exitCode_*`.	No
`category`	What you are validating.	No
`name`	Name of what you are validating, eg `--output-path` or `example_outputPath`	No
`value`	Raw, supplied value of what you are validating	No

Use this function to validate that the the path (value) is a readable, searchable, writable (and so extant) folder or it can be created. Exits with code if validation fails. Useful for things like output paths.

`core_validate_filePathReadable()`

Parameter	Value	Optional
`code`	An exit code (should not be zero). Ideally use one of the constants `core_commandLine_exitCode_*`.	No
`category`	What you are validating.	No
`name`	Name of what you are validating, eg `--output-path` or `example_outputPath`	No
`value`	Raw, supplied value of what you are validating	No

Use this function to validate that the the path (value) is a readable (and so extant) file. Exits with code if validation fails.

`core_validate_filePathReadableAndExecutable()`

Parameter	Value	Optional
`code`	An exit code (should not be zero). Ideally use one of the constants `core_commandLine_exitCode_*`.	No
`category`	What you are validating.	No
`name`	Name of what you are validating, eg `--output-path` or `example_outputPath`	No
`value`	Raw, supplied value of what you are validating	No

Use this function to validate that the the path (value) is a readable, executable (and so extant) file. Exits with code if validation fails.

`core_validate_filePathReadableAndExecutableAndNotEmpty()`

Parameter	Value	Optional
`code`	An exit code (should not be zero). Ideally use one of the constants `core_commandLine_exitCode_*`.	No
`category`	What you are validating.	No
`name`	Name of what you are validating, eg `--output-path` or `example_outputPath`	No
`value`	Raw, supplied value of what you are validating	No

Use this function to validate that the the path (value) is a readable, executable (and so extant), not empty file (as most executables should be). Exits with code if validation fails.

`core_validate_socketPathReadableAndWritable()`

Parameter	Value	Optional
`code`	An exit code (should not be zero). Ideally use one of the constants `core_commandLine_exitCode_*`.	No
`category`	What you are validating.	No
`name`	Name of what you are validating, eg `--output-path` or `example_outputPath`	No
`value`	Raw, supplied value of what you are validating	No

Use this function to validate that the the path (value) is a readable, writable (and so extant) Unix domain socket. Exits with code if validation fails.

`core_validate_characterDeviceFileReadableAndWritable()`

Parameter	Value	Optional
`code`	An exit code (should not be zero). Ideally use one of the constants `core_commandLine_exitCode_*`.	No
`category`	What you are validating.	No
`name`	Name of what you are validating, eg `--output-path` or `example_outputPath`	No
`value`	Raw, supplied value of what you are validating	No

Use this function to validate that the the path (value) is a readable, writable (and so extant) character device. Exits with code if validation fails.

`core_validate_isBoolean()`

Parameter	Value	Optional
`code`	An exit code (should not be zero). Ideally use one of the constants `core_commandLine_exitCode_*`.	No
`category`	What you are validating.	No
`name`	Name of what you are validating, eg `--output-path` or `example_outputPath`	No
`value`	Raw, supplied value of what you are validating	No

Use this function to validate that the the value is a Boolean. Exits with code if validation fails.

Of course, there are no such things as booleans in shell script, but we can model booleans as strings. Booleans are used in many places in shellfire, as they document behaviour much better.

A value is a boolean and true if it is

true, TRUE, True or T
yes, YES, Yes, Y
on, ON, On
1

A value is a boolean and false if it is

false, FALSE, False or F
no, NO, No, N
off, OFF, Off
0

`core_validate_isUnsignedInteger()`

Parameter	Value	Optional
`code`	An exit code (should not be zero). Ideally use one of the constants `core_commandLine_exitCode_*`.	No
`category`	What you are validating.	No
`name`	Name of what you are validating, eg `--output-path` or `example_outputPath`	No
`value`	Raw, supplied value of what you are validating	No

Use this function to validate that the the value is an unsigned integer. Exits with code if validation fails. An unsigned integer is considered to be any unprefix decimal value greater than or equal to 0.

`core_validate_nonDynamicPort()`

Parameter	Value	Optional
`code`	An exit code (should not be zero). Ideally use one of the constants `core_commandLine_exitCode_*`.	No
`category`	What you are validating.	No
`name`	Name of what you are validating, eg `--output-path` or `example_outputPath`	No
`value`	Raw, supplied value of what you are validating	No

Use this function to validate that the the value is a non-dynamic port. Exits with code if validation fails. A non-dynamic port is one between 1 and 65535 inclusive.

Namespace `core_variable`

This namespace provides a wide range of helper functions to work with variables and values.

To use in code

This namespace is included by default. No additional actions are required.

Functions

`core_variable_isSet()`

Parameter	Value	Optional
`variableName`	A variable name	No

Returns a 0 exit code if the variable is set (defined) (but not empty or null), or a non-zero one if it isn't. Be aware that there is a bug in bash 3.2 on Mac OS X when invoked as sh that treats a defined, but not set local variable, as if it is empty.

`core_variable_isUnset()`

Parameter	Value	Optional
`variableName`	A variable name	No

Returns a 0 exit code if the variable is unset (undefined) (but not empty or null), or non-zero one if it is. Be aware that there is a bug in bash 3.2 on Mac OS X when invoked as sh that treats a defined, but not set local variable, as if it is empty.

`core_variable_indirectValue()`

Parameter	Value	Optional
`variableName`	A variable name	No

Sets the variable core_variable_indirectValue_result to the value pointed to by variableName. Exists because most shells don't provide a way to do this simply. Deliberately does not write to standard out, even though that would have been easier, as the caller, needs to capture standard out would then strip trailing line feeds and so end up with a different variable value.

`core_variable_unset()`

Parameter	Value	Optional
`variableName`	A variable name	No

Unsets (undefines) the variable pointed to by variableName. This method exists because the normal unset built-in misbehaves on pdksh when trying to unset non-existent variables (even if set +u is used)! One could argue that these shells are more correct, but, since the common ones (bash included) don't behave this way, this is the only cross-shell way to be consistent. We do not redefine the unset built in as this doesn't work in pdksh (which, despite ceasing development in 1999, is still widely used in the BSDs).

`core_variable_setVariable()`

Parameter	Value	Optional
`variableName`	A variable name	No
`value`	A value to set it to	No

Sets the variable pointed to by variableName to value.

`core_variable_setAndExportVariable()`

Parameter	Value	Optional
`variableName`	A variable name	No
`value`	A value to set it to	No

Sets and exports the variable pointed to by variableName to value.

`core_variable_isUnsetOrEmpty()`

Parameter	Value	Optional
`variableName`	A variable name	No

Returns a 0 exit code if the variable is unset (undefined) or empty (null in Shell speak!), or non-zero one if it is. Be aware that there is a bug in bash 3.2 on Mac OS X when invoked as sh that treats a defined, but not set local variable, as if it is empty.

`core_variable_setVariableIfUnset()`

Parameter	Value	Optional
`variableName`	A variable name	No
`variableDefaultValue`	A value to set it to	No

Sets the variable pointed to by variableName to value if, and only if, it is unset.

`core_variable_contains()`

Parameter	Value	Optional
`value`	A value	No
`contains`	A value to check for	No

Returns a 0 exit code if the value contains contains, or non-zero one otherwise.

`core_variable_matches()`

Parameter	Value	Optional
`value`	A value	No
`match`	A value to match	No

Returns a 0 exit code if the the shell glob expression match matches the value, or non-zero one otherwise. Avoid using bashisms.

`core_variable_startsWith()`

Parameter	Value	Optional
`value`	A value	No
`startsWith`	A value to match	No

Returns a 0 exit code if the value starts with startsWith, or non-zero one otherwise.

`core_variable_doesNotStartWith()`

Parameter	Value	Optional
`value`	A value	No
`startsWith`	A value to match	No

Returns a 0 exit code if the value does not start with startsWith, or non-zero one otherwise.

`core_variable_endsWith()`

Parameter	Value	Optional
`value`	A value	No
`endsWith`	A value to match	No

Returns a 0 exit code if the value ends with endsWith, or non-zero one otherwise.

`core_variable_doesNotEndWith()`

Parameter	Value	Optional
`value`	A value	No
`endsWith`	A value to match	No

Returns a 0 exit code if the value does not end with endsWith, or non-zero one otherwise.

`core_variable_firstCharacter()`

Parameter	Value	Optional
`value`	A value	No

Writes to standard out the first character of value.

`core_variable_lastCharacter()`

Parameter	Value	Optional
`value`	A value	No

Writes to standard out the last character of value.

`core_variable_trimSpaceAndHorizontalTab()`

Parameter	Value	Optional
`value`	A value	No

Writes to standard out value with any leading or trailing spaces and horizontal tabs removed (POSIX character class [:blank:]).

`core_variable_trimWhitespace()`

Parameter	Value	Optional
`value`	A value	No

Writes to standard out value with any leading or trailing whitespace removed (POSIX character class [:space:]).

`core_variable_allButLastN()`

Parameter	Value	Optional
`value`	A value	No
`numberToOmit`	Number of characters to remove	No

Writes to standard out value with the trailing numberToOmit characters removed.

`core_variable_allButLast()`

Parameter	Value	Optional
`value`	A value	No

Writes to standard out value with the final character removed. Useful for stripping a trailing / from path prefixes.

`core_variable_allButFirstN()`

Parameter	Value	Optional
`value`	A value	No
`numberToOmit`	Number of characters to remove	No

Writes to standard out value with the leading numberToOmit characters removed.

`core_variable_allButFirst()`

Parameter	Value	Optional
`value`	A value	No

Writes to standard out value with the leading character removed. Useful for stripping a leading / from path prefixes.

`core_variable_characterByCharacter()`

Parameter	Value	Optional
`value`	A value	No
`callback`	A callback (function name)	No

Calls callback for each character character of value. Inefficient. Useful for building any manner of parsers in shell script, though.

`core_variable_isTrue()`

Parameter	Value	Optional
`value`	A value	No

Returns an exit code of:-

0 if value is a boolean true (T, t, true, True, TRUE, Y, y, yes, Yes, YES, on, On, ON, 1)
1 if value is a boolean false (F, f, false, False, FALSE, N, n, no, No, NO, off, Off, OFF, 0)
2 if value is not valid

`core_variable_isFalse()`

Parameter	Value	Optional
`value`	A value	No

Returns an exit code of:-

0 if value is a boolean false (F, f, false, False, FALSE, N, n, no, No, NO, off, Off, OFF, 0)
1 if value is a boolean true (T, t, true, True, TRUE, Y, y, yes, Yes, YES, on, On, ON, 1)
2 if value is not valid

`core_variable_isInvalidBoolean()`

Parameter	Value	Optional
`value`	A value	No

Returns an exit code of:-

0 if value is not valid
1 if value is a boolean false (F, f, false, False, FALSE, N, n, no, No, NO, off, Off, OFF, 0)
1 if value is a boolean true (T, t, true, True, TRUE, Y, y, yes, Yes, YES, on, On, ON, 1)

`core_variable_escapeForSingleQuotedLiteralInHeredoc`

Parameter	Value	Optional
`value`	A value	No

Writes a value to standard out suitable for use in a here doc that will itself become a bash script as a single quoted literal. In essence, escapes any ' single quotes as '"'"'. For example, it could be used as:-

my_value="hello ' world"
cat <<-EOF
	# Note the SINGLE quotes surrounding $()
	my_heredoc_value='$(core_variable_escapeForSingleQuotedLiteralInHeredoc "$my_value")'
EOF

Namespace `core_variable_array`

This namespace provides a wide range of helper functions to pretend variables are arrays. It uses a bespoke separator character. There is experimental support to make bash arrays appear to work under this pretence. This module works hard to overcome serious bugs in bash 3.2 when run as sh (unset variables are empty; most control code separate characters don't work). Do not use arrays with file paths if there's any chance they could contain \r.

To use arrays in global scope, one does:-

# Optional, core_variable_array_append initialises if necessary
core_variable_array_initialise myArray

# Append values - spaces are correctly handled
core_variable_array_append myArray 'value1' 'value 2'
core_variable_array_append myArray 'etc'

length=$(core_variable_array_length myArray)
if core_variable_array_isEmpty myArray; then
	echo Empty!
fi

# will be `value 2`
itemAtIndex1="$(core_variable_array_at myArray 1)"

# will be 'value1, value2, etc'
echo "$(core_variable_array_string myArray ', ')"

# Use to call an executable or function with all arguments
# Will do  cat 'value1' 'value 2' 'etc'
core_variable_array_passToFunctionAsArguments myArray cat

# Iterate
index=0
some_callback()
{
	echo "array element at index $index in $core_variable_array_element"
	index=$((index+1))
}
core_variable_array_iterate myArray some_callback

# Iterate as if members were callbacks
core_variable_array_iterate myArray 'arg to pass to every callback'

To use in code

This namespace is included by default. No additional actions are required.

Functions

`core_variable_array_initialise()`

Parameter	Value	Optional
`variableName`	An variable name	No

Initialises an array. variableName will then exist. If a local array is desired, then a little more work is needed to work-around shell interpreter bugs; one needs to define a second variable ${variableName}_initialised. This is because under the covers two variables are used to manage arrays (primarily because some versions of bash can't differentiate between undefined and defined local variables)

local myArray
local myArray_initialised
core_variable_array_initialise myArray

`core_variable_array_append()`

Parameter	Value	Optional
`variableName`	An variable name	No
`…`	Zero or more elements to append	Yes, but specify at least one for to be useful!

Appends elements … to an array. If the array doesn't yet exist, it is initialised if … has one or more elements. Caveats as above about local arrays.

core_variable_array_append myArray 'value1' 'value 2'

`core_variable_array_appendUniquely()`

Parameter	Value	Optional
`variableName`	An variable name	No
`…`	Zero or more elements to append	Yes, but specify at least one for to be useful!

Appends elements … to an array if they haven't already been added. If the array doesn't yet exist, it is initialised if … has one or more elements. Caveats as above about local arrays.

core_variable_array_append myArray 'value1' 'value 2'

`core_variable_array_unset()`

Parameter	Value	Optional
`variableName`	An variable name	No

Unsets an array.

`core_variable_array_isSet()`

Parameter	Value	Optional
`variableName`	An variable name	No

Returns a 0 exit code if the array variableName is set or a non-zero one if it isn't.

`core_variable_array_isUnset()`

Parameter	Value	Optional
`variableName`	An variable name	No

Returns a 0 exit code if the array variableName is unset or a non-zero one if it isn't.

`core_variable_array_isEmpty()`

Parameter	Value	Optional
`variableName`	An variable name	No

Returns a 0 exit code if the array variableName is empty or a non-zero one if it isn't.

`core_variable_array_length()`

Parameter	Value	Optional
`variableName`	An variable name	No

Writes the length of the array variableName to standard out.

`core_variable_array_at()`

Parameter	Value	Optional
`variableName`	An variable name	No
`index`	A zero-based index	No

Set the variable core_variable_array_index_element to the element value at index index in the array variableName.

`core_variable_array_string()`

Parameter	Value	Optional
`variableName`	An variable name	No
`separator`	A string to separate the array elements	No

Writes the concatenation of the elements of the array variableName, separated by separator, to standard out.

`core_variable_array_contains()`

Parameter	Value	Optional
`variableName`	An variable name	No
`containsValue`	An element value to find	No

Returns a 0 exit code if the array variableName contains an element matching the value containsValue or a non-zero one if it isn't.

`core_variable_array_iterate()`

Parameter	Value	Optional
`variableName`	An variable name	No
`callback`	A function name	No
`…`	Zero or more elements to pass to `callback`	Yes

Iterates over each element of the array, setting the variable core_variable_array_element before calling callback with ….

`core_variable_array_iterateShortcut()`

Parameter	Value	Optional
`variableName`	An variable name	No
`callback`	A function name	No
`…`	Zero or more elements to pass to `callback`	Yes

Iterates over each element of the array, setting the variable core_variable_array_element before calling callback with …. Expects the callback to return an exit code. If the exit code if 0, then iteration is shortcircuited and the function returns 0. If all callbacks are called and none return 0, then the function returns an exit code of 1.

`core_variable_array_iterateAsCallbacks()`

Parameter	Value	Optional
`variableName`	An variable name	No
`…`	Zero or more elements to pass to `callback`	Yes

Iterates over each element of the array, treating it as the name of function to callback. Calls each callback with ….

`core_variable_array_iterateAsCallbacksShortcut()`

Parameter	Value	Optional
`variableName`	An variable name	No
`…`	Zero or more elements to pass to `callback`	Yes

Iterates over each element of the array, treating it as the name of function to callback. Calls each callback with …. Expects callbacks to return an exit code. If the exit code if 0, then iteration is shortcircuited and the function returns 0. If all callbacks are called and none return 0, then the function returns an exit code of 1.

`core_variable_array_passToFunctionAsArguments()`

Parameter	Value	Optional
`variableName`	An variable name	No
`function`	Either a function name or an executable on the path	No

Passes all the elements of the array variableName to a function function. Useful for building up complex command lines, eg for curl:-

local optionsForCurl
local optionsForCurlInitialised
core_variable_array_initialise optionsForCurl

if [ "$(core_init_verbosity)" -gt 0 ]; then
	core_variable_array_append optionsForCurl --verbose
fi
core_variable_array_append optionsForCurl --url 'http://www.stormmq.com/'
if [ -n "$myUserCredentials" ]; then
	core_variable_array_append optionsForCurl --user "$myUserCredentials"
fi

# calls curl with, perhaps, curl --verbose --url 'http://www.stormmq.com/'
# Spaces in arguments are correctly handled
core_variable_array_passToFunctionAsArguments optionsForCurl curl

Name		Name	Last commit message	Last commit date
Latest commit History 454 Commits
base64		base64
dependency		dependency
variable		variable
COPYRIGHT		COPYRIGHT
LICENSE		LICENSE
README.md		README.md
children.functions		children.functions
commandLine.functions		commandLine.functions
compatibility.functions		compatibility.functions
configuration.functions		configuration.functions
core.functions		core.functions
file.functions		file.functions
functions.functions		functions.functions
init.functions		init.functions
licences.snippet		licences.snippet
path.functions		path.functions
pipefail.functions		pipefail.functions
snippet.functions		snippet.functions
temporaryFiles.functions		temporaryFiles.functions
trap.functions		trap.functions
umask.functions		umask.functions
validate.functions		validate.functions

License

shellfire-dev/core

Folders and files

Latest commit

History

Repository files navigation

core: functions module for shellfire

Compatibility

Overview

Overridding a module's function(s)

Namespaces

User Definable Definitions

A few criticial functions aren't where you'd expect

core_compatibility

core_variable

core_dependency

core_terminal

core

Importing

Namespaceless

User-Definable Constants

User-Definable Functions

_program()

Namespace core

To use in code

Functions

core_message

core_exitError

core_TODO

core_uses

core_uses

Namespace core_base64_decode

To use in code

Functions

core_base64_decode_string

core_base64_decode_file

Namespace core_commandLine

To use in code

Global Constants

Exit Codes

Functions

User-Definable Functions

_program_commandLine_parse()

_program_commandLine_parseInitialise()

_program_commandLine_helpMessage()

_program_commandLine_optionExists()

_program_commandLine_processOptionWithoutArgument()

_program_commandLine_processOptionWithArgument()

_program_commandLine_processOptionWithArgument()

_program_commandLine_handleNonOptions()

_program_commandLine_validate()

Namespace core_children

To use in code

Functions

core_children_killOnExit()

Namespace core_compatibility

To use in code

Functions

core_compatibility_basename()

core_compatibility_dirname()

core_compatibility_which()

core_compatibility_whichNoOutput()

core_compatibility_echo()

core_compatibility_sleepSupportsFractionalSeconds()

Namespace core_configuration

Configuration Locations

To use in code

Functions

core_configuration_blacklist

Namespace core_dependency

To use in code

Functions

core_dependency_declares()

core_dependency_declaresAsArray()

core_dependency_requires()

core_dependency_fallback()

core_dependency_oneOf()

Namespace core_file

To use in code

Functions

`core`: functions module for shellfire

`core_compatibility`

`core_variable`

`core_dependency`

`core_terminal`

`core`

`_program()`

Namespace `core`

`core_message`

`core_exitError`

`core_TODO`

`core_uses`

`core_uses`

Namespace `core_base64_decode`

`core_base64_decode_string`

`core_base64_decode_file`

Namespace `core_commandLine`

`_program_commandLine_parse()`

`_program_commandLine_parseInitialise()`

`_program_commandLine_helpMessage()`

`_program_commandLine_optionExists()`

`_program_commandLine_processOptionWithoutArgument()`

`_program_commandLine_processOptionWithArgument()`

`_program_commandLine_processOptionWithArgument()`

`_program_commandLine_handleNonOptions()`

`_program_commandLine_validate()`

Namespace `core_children`

`core_children_killOnExit()`

Namespace `core_compatibility`

`core_compatibility_basename()`

`core_compatibility_dirname()`

`core_compatibility_which()`

`core_compatibility_whichNoOutput()`

`core_compatibility_echo()`

`core_compatibility_sleepSupportsFractionalSeconds()`

Namespace `core_configuration`

`core_configuration_blacklist`

Namespace `core_dependency`

`core_dependency_declares()`

`core_dependency_declaresAsArray()`

`core_dependency_requires()`

`core_dependency_fallback()`

`core_dependency_oneOf()`

Namespace `core_file`

`core_file_characterByCharacter`

`core_file_characterByCharacterCreate`

Namespace `core_functions`

`core_functions_exist`

`core_functions_register`

`core_functions_execute`

Namespace `core_init`

`core_init_defines`

`core_init_verbosity`

`core_init_language`

Namespace `core_path`

`core_path_isReadableFilePath()`

`core_path_isReadableNonEmptyFilePath()`

`core_path_isReadableNonEmptyExecutableFilePath()`

`core_path_isReadableAndSearchableFolderPath()`

`core_path_isReadableAndSearchableAndWritableFolderPath()`

Namespace `core_pipefail`

`core_pipefail_execute()`

Namespace `core_snippet`

`core_snippet_embed()`

`core_snippet_retrieve()`

`core_snippet_retrieveAndSourceAsHereDoc()`

Namespace `core_temporaryFiles`

`core_temporaryFiles_newFileToRemoveOnExit()`

`core_temporaryFiles_newFolderToRemoveOnExit()`

Namespace `core_terminal`

`core_terminal_ansiSupported()`

`core_terminal_tput()`

`core_terminal_colour()`

`core_terminal_effect()`

`core_terminal_reset()`

Namespace `core_trap`

`core_trap_addOnCleanUp()`

`core_trap_addHandler()`

Namespace `core_umask`

Namespace `core_validate`