Skip to content
Permalink
Browse files

Add option to disable */shcomp test variants

We always expect the ksh */shcomp unit test variants to behave like the
non-shcomp variants. So there is normally no reason to run the shcomp
variants. Provide an option to disable those tests in order to reduce
the cost of testing ksh script behavior.
  • Loading branch information
krader1961 committed Dec 22, 2019
1 parent b6be7e7 commit 0d76ee4bd7eef046f1c0aad28fc5c3cece51eaf7
Showing with 49 additions and 11 deletions.
  1. +32 −2 CONTRIBUTING.md
  2. +1 −1 meson.build
  3. +9 −5 meson_options.txt
  4. +6 −2 src/cmd/ksh93/meson.build
  5. +1 −1 src/cmd/ksh93/tests/meson.build
@@ -284,12 +284,42 @@ The `--setup=malloc` will enable malloc integrity features provided by your
system's malloc implementation if it supports such things via environment
variables. That flag can be omitted but its use is recommended.

### Disabling some tests.

You can minimize total test time by changing some build time defaults. This
can be useful to minimize test run time on your local system(s) since:

a) We expect API tests to never fail since the core APIs (e.g., SFIO) rarely
change and thus should never fail their unit tests.

b) We expect shcomp and non-shcomp variants to pass/fail in lock step. So
normally there is no reason to run the `*/shcomp` variants.

That is done by configuring the build thusly:

```
meson -Dbuild-api-tests=false -Dbuild-shcomp-tests=false
```

Obviously, all CI environments should enable both sets of tests, at least
some, if not all, of the time, and it would be a good idea for everyone
else to occassionally enable both sets of tests. Which is why the default
is for both sets to be enabled. You have to deliberately disable those
sets of tests if you care about the overhead they add.

You can also enable just the API tests, thus skipping all the ksh script
tests, by doing

```
meson -Dbuild-api-tests-only=true
```

### Testing with ASAN -- AddressSanitizer

At the moment this only works on Linux using gcc.

Configure with `meson -DASAN=true -Dbuild-api-tests=false`. Then build with
`ninja` as usual. Run the tests with `meson test --setup=asan`.
Configure with `meson -DASAN=true`. Then build with `ninja` as usual. Run
the tests with `meson test --setup=asan`.

You will need to install the `llvm-symbolizer` tool if the gcc version is less
than 4.9.3. For example, on OpenSuse 42.3 you'll need to run `sudo zypper
@@ -297,7 +297,7 @@ run_command('scripts/libast_prereq.sh')
subdir('src/lib/libast')
subdir('src/lib/libdll')

# Check if api tests should be built
# Check if API tests should be built.
if get_option('build-api-tests') == true
subdir('src/lib/libast/tests/')
endif
@@ -15,7 +15,7 @@ option('unittest', type : 'boolean', value : false)
# TODO: Use the following definition when we can count on Meson 0.45 being
# available on the platforms we care about. For the moment we're stuck with
# Meson 0.44.
# option('read-timeout', type : 'integer', min : 0, value : 0)
# option('read-timeout', type : 'integer', min : 0, value : 0)
#
option('read-timeout', type : 'string', value : '0')

@@ -25,14 +25,18 @@ option('read-timeout', type : 'string', value : '0')
#
option('audit-file', type : 'string', value : '/etc/ksh_audit')

# To disable building api tests, set build-api-tests option to false:
# meson -Dbuild-api-tests=false
option('build-api-tests', type : 'boolean', value : true)

# To build api tests only, set build-api-tests-only option to true:
# meson -Dbuild-api-tests-only=true
option('build-api-tests-only', type : 'boolean', value : false)

# To disable running API tests, set build-api-tests option to false:
# meson -Dbuild-api-tests=false
option('build-api-tests', type : 'boolean', value : true)

# To disable running */shcomp tests, set build-shcomp-tests option to false:
# meson -Dbuild-shcomp-tests=false
option('build-shcomp-tests', type : 'boolean', value : true)

# This number is used when git is not available
# meson -Dfallback-version-number='x.y.z'
option('fallback-version-number', type : 'string', value : '2020.99.99')
@@ -61,8 +61,12 @@ ld_library_path = 'LD_LIBRARY_PATH=' + ':'.join(
# Sample loadable builtin will be loaded from this path in builtins test cases
libsample_path = 'LIBSAMPLE_PATH=' + libsample.full_path()

subdir('tests/api')
# Check if only api tests should be built
# Check if API tests should be built.
if get_option('build-api-tests') == true
subdir('tests/api')
endif

# Check if only API tests should be built.
if get_option('build-api-tests-only') == false
subdir('tests')
endif
@@ -139,7 +139,7 @@ foreach testspec : all_tests
system_var, timeout_var])

# The shcomp variants are only applicable to the non-interactive tests.
if not test_name.endswith('.exp')
if get_option('build-shcomp-tests') == true and not test_name.endswith('.exp')
# Run the test after compiling the script with `shcomp`.
test(test_name + '/shcomp', ksh93_exe, timeout: timeout, is_parallel: parallel,
args: ['-p', test_driver, 'shcomp', test_name],

0 comments on commit 0d76ee4

Please sign in to comment.
You can’t perform that action at this time.