Skip to content
This repository has been archived by the owner on Apr 29, 2021. It is now read-only.

WIP: Allow sourcing of helper files from BATS_LIB_PATH #216

Open
wants to merge 47 commits into
base: master
Choose a base branch
from

Conversation

ntnn
Copy link

@ntnn ntnn commented Mar 22, 2017

The existing functionality of load() is preserved:
1. Load helper files in the same directory the current testfile
resides in
2. Load helper files by absolute path

Additionally an environment variable BATS_LIB_PATH can be defined to
be source helper files from.

If BATS_LIB_PATH is empty the following locations are used:
1. $HOME/.bats/lib
Allows users to manage helper libraries themselves, similar to
python/ruby/...
2. /usr/lib/bats
Allows to install helper libraries via a package manager.

Example usage:

  • bats installed via package manager
  • bats-support files installed via package manager to /usr/lib/bats
$ tree /usr/lib/bats
/usr/lib/bats
├── bats-support/
│   ├── error.bash
│   ├── lang.bash
│   └── output.bash
└── bats-support.bash

1 directory, 4 files
$ cat /usr/lib/bats/bats-support
source "$(dirname "${BASH_SOURCE[0]}")/bats-support/output.bash"
source "$(dirname "${BASH_SOURCE[0]}")/bats-support/error.bash"
source "$(dirname "${BASH_SOURCE[0]}")/bats-support/lang.bash"
$ cat example.bats
#!/usr/bin/env bats

load bats-support
@test "This will fail" {
    fail "This is an error message with error.bash loaded from /usr/lib/bats/error.bash"
}
$ bats example.bats
1..1
not ok 1 This will fail
# (from function `fail' in file /usr/lib/bats/bats-support/error.bash, line 40,
#  in test file example.bats, line 6)
#   `fail "This is an error message with error.bash loaded from /usr/lib/bats/error.bash"' failed
# This is an error message with error.bash loaded from /usr/lib/bats/error.bash

The bats-support here is different than the load.bash found in the repository.

Currently a loading file is required, which sources the files in the directory, unless the library is a single file.
To work around this I'll add a mechanism which sources all files in a directory, if the directory exists but no loading file is found.

Also, the last line of the error message shows the wrong location for error.bash.

  • Fix error message with wrong path
  • Source directory if no loading file exists

Tim Harsch and others added 2 commits January 20, 2016 17:14
Some implementations of head (e.g. the one in Busybox)
can't handle options like "-1". Replace them with "-n1".

Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
@ntnn
Copy link
Author

ntnn commented Mar 22, 2017

@ztombol You've written three libraries for bats so far, what do you think of a load mechanism like this?

Additionally I've read quite some issues and comments about loading helper files, but none seem to have fruited into anything other than the current load(), which only sources in the same dir or absolute paths.

@ntnn
Copy link
Author

ntnn commented Mar 22, 2017

Additionally I'd like to know how heavy posix compliance weighs in. Since many features of bats are based on non-posix functionality (e.g. set +-E/set +-T) it doesn't make sense to use the slower [ builtin vs. the [[ keyword.

If POSIX-compliance is a long term goal I'll gladly push that to the branch.

btamayo and others added 26 commits September 19, 2017 14:55
FullSemver: 0.4.1+9
Branch: master
Minor fix in the installation snippet
This follows the example from https://www.appveyor.com/docs/lang/ruby/
except that it doesn't need `install` or `before_test` steps.
This enables us to get an idea of the performance impact of a change.
As with the update to .appveyor.yml in the previous commit, this enables
us to get an idea of the performance impact of a change.
Also sets the `language` to `bash`.
The following build is demonstrating failures I can't reproduce on my
own macOS system:

  https://travis-ci.org/bats-core/bats-core/jobs/281719290
When running under Bash 3.2.57(1)-release on macOS, the following tests
would fail because `BATS_ERROR_STACK_TRACE` would be empty, and hence no
information about the actual error would get printed:

- one failing test
- failing test with significant status
- failing test file outside of BATS_CWD

This is because each of these cases use `FIXTURE_ROOT/failing.bats`, and
the `ERR` trap would not fire for its `eval "( exit ${STATUS:-1} )"`
line. Changing it to `exit ${STATUS:-1}` produced the same effect, and
changing it to `return ${STATUS:-1}` would cause the output to point to
the previous line, which executes `true`.

However, the correct status would be reported to the `EXIT` trap, so now
we call `bats_error_trap` at the very beginning of `bats_teardown_trap`.

All the existing tests now pass under Bash 3.2.57(1)-release, Bash
4.2.25(1)-release (the version from the default Ubuntu 12.04.5/Precise
image on Travis CI), and Bash 4.4.12(1)-release.
Add Appveyor configuration, update Travis CI configuration, fix macOS bug
Preserves existing behavior. Next step will be to take the target
variable name as the second argument.
This is part of the effort to improve performance by reducing the number
of command substitutions/subshells spawned by `bats_debug_trap`.

Under Bash 3.2.57(1)-release on a MacBook Pro with a 2.9GHz Intel Core
i5 CPU and 8GB 1867MHz DDR3 RAM, this makes `bin/bats test/` go from:

  44 tests, 0 failures

  real    0m7.565s
  user    0m3.664s
  sys     0m3.368s

to:

  real    0m6.449s
  user    0m3.290s
  sys     0m2.665s
Also eliminates a subshell.
Also replaces `sed` invocation with a `while` loop, saving a subprocess.
Also, `bats-preprocess` now converts DOS/Windows CRLF line endings.
Somehow this is ever-so-slightly faster.
This is part of the effort to improve performance by reducing the number
of command substitutions/subshells spawned by `bats_debug_trap`.

Under Bash 3.2.57(1)-release on a MacBook Pro with a 2.9GHz Intel Core
i5 CPU and 8GB 1867MHz DDR3 RAM, this makes `bin/bats test/` go from the
following for the previous commit:

  44 tests, 0 failures

  real    0m5.293s
  user    0m2.853s
  sys     0m2.087s

to:

  real    0m4.319s
  user    0m2.559s
  sys     0m1.454s
This is in anticipation of refactoring away the `$(eval echo
"$quoted_name")` command substitution.
mbland and others added 19 commits September 30, 2017 15:12
This is part of the effort to improve performance by reducing the number
of command substitutions/subshells.

Under Bash 3.2.57(1)-release on a MacBook Pro with a 2.9GHz Intel Core
i5 CPU and 8GB 1867MHz DDR3 RAM, this shaves off O(0.15s) from the test
suite at the previous commit, but I anticipate this effect being
magnified on Windows platforms.
Under Bash 3.2.57(1)-release and 4.4.12(1)-release on a MacBook Pro with
a 2.9GHz Intel Core i5 CPU and 8GB 1867MHz DDR3 RAM, this shaves off
O(0.4s) from the current test suite.

Before the change:

  46 tests, 0 failures

  real    0m4.392s
  user    0m2.489s
  sys     0m1.467s

After the change:

  real    0m3.980s
  user    0m2.312s
  sys     0m1.233s
Under Bash 3.2.57(1)-release and 4.4.12(1)-release on a MacBook Pro with
a 2.9GHz Intel Core i5 CPU and 8GB 1867MHz DDR3 RAM, this shaves off
O(0.1s) from the current test suite.

Before the change:

  46 tests, 0 failures

  real    0m3.983s
  user    0m2.320s
  sys     0m1.241s

After the change:

  real    0m3.861s
  user    0m2.276s
  sys     0m1.174s
Under Bash 3.2.57(1)-release and 4.4.12(1)-release on a MacBook Pro with
a 2.9GHz Intel Core i5 CPU and 8GB 1867MHz DDR3 RAM, this shaves off
O(0.25s) from the current test suite.

Before the change:

  46 tests, 0 failures

  real    0m3.851s
  user    0m2.273s
  sys     0m1.166s

After the change:

  real    0m3.595s
  user    0m2.171s
  sys     0m1.048s
Under Bash 3.2.57(1)-release and 4.4.12(1)-release on a MacBook Pro with
a 2.9GHz Intel Core i5 CPU and 8GB 1867MHz DDR3 RAM, this shaves off
O(0.04-0.05s) from the current test suite. Very minor, but it's a
straightforward change that may provide a minor-yet-noticeable effect on
Windows.

Before this change:

  46 tests, 0 failures

  real    0m3.588s
  user    0m2.171s
  sys     0m1.046s

After this change:

  real    0m3.538s
  user    0m2.119s
  sys     0m0.941s
Under Bash 3.2.57(1)-release and 4.4.12(1)-release on a MacBook Pro with
a 2.9GHz Intel Core i5 CPU and 8GB 1867MHz DDR3 RAM, this shaves off
O(0.16s) from the current test suite.

Before this change:

  46 tests, 0 failures

  real    0m3.541s
  user    0m2.125s
  sys     0m0.937s

After this change:

  real    0m3.372s
  user    0m2.031s
  sys     0m0.894s
While the performance impact of these changes are in the noise under
macOS 10.12.3 on a on a MacBook Pro with a 2.9GHz Intel Core i5 CPU and
8GB 1867MHz DDR3 RAM, eliminating these subshells makes the code more
consistent.

I did try removing `buffer` to eliminate yet more subshells, but the
flickering of the output did prove annoying, so I'm not removing it.
The four test cases updated in this commit were failing on my Windows
virtual machine because my username contains a space. Quoting the file
paths containing "$FIXTURE_ROOT" solved the problem.
Fix macOS/Bash 3.2 breakage; eliminate subshells from exec-test, preprocess
Transfer bats#163 - Avoid undocumented usage of the head utility.
Separate "skip" from the test name (description) to be distinct from "skip" as part of the TAP Directive.
…_test_fix

Transfer bats#142 - This commit corrects the problem of incorrect TAP format for skipped tests
The test cases that use the `-p` or `--pretty` flag would fail if `tput`
wasn't available (installed as part of the `ncurses` package). On
Alpine, `ncurses` isn't installed by default. This change ensures the
tests validate the expected behavior regardless of whether `tput` is
available.
The existing functionality of load() is preserved:
    1. Load helper files in the same directory the current testfile
       resides in
    2. Load helper files by absolute path

Additionally an environment variable BATS_LIB_PATH can be defined to
be source helper files from.

If BATS_LIB_PATH is empty the following locations are used:
    1. $HOME/.bats/lib
       Allows users to manage helper libraries themselves, similar to
       python/ruby/...
    2. /usr/lib/bats
       Allows to install helper libraries via a package manager.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants