Skip to content

Latest commit

 

History

History
100 lines (69 loc) · 2.99 KB

testing.rst

File metadata and controls

100 lines (69 loc) · 2.99 KB
Raw

Testing

Serac has two levels of tests, unit and integration. Unit tests are used to test individual components of code, such as a class or function. While integration tests are for testing the code as a whole. For example, testing the serac driver with an input file against blessed answers.

Unit Tests

Unit Tests can be ran via the build target test after building the code.

Integration Tests

Note

Integration testing is in development and not fully featured.

Requirements:

  • Installed ATS
  • ATS_EXECUTABLE defined in the host-config (added automatically to Spack generated host-configs) or on command line via -DATS_EXECUTABLE=/path/to/ats.

  • If using a personal machine, check the ats-config directory in the serac repo and create a json file <your_machine_name>.json if you haven't already. Your machine's name can be found by running the following lines of code:

    $ python3
>>> import socket
>>> socket.gethostname().rstrip('1234567890')
>>> exit()

    Currently, there are configuration json files for Toss3 and BlueOS which can be used as reference.

  1. Build the code. Build code with the normal steps. More info in the quickstart-label. This generates a script in the build directory called ats.sh.

  2. Run integration tests. Run the corresponding command for the system you are on:

    # BlueOS
$ lalloc 2 ./ats.sh

# Toss3
$ salloc -N2 ./ats.sh

# Personal Machine (currently runs subset of tests)
$ ./ats.sh

    Append --help to the command to see the current options.

  3. View results. ATS gives a running summary and the final results. ATS also outputs the following helpful files in the platform and timestamp specific created log directory:

    • ats.log - All output of ATS
    • atss.log - Short summary of the run
    • atsr.xml - JUnit test summary

    ATS also outputs both a .log and .log.err for each test and checker that is run.

Installing ATS

ATS can be installed via the normal devtools install process. More info on devtools-label. This method is useful because it builds all development tools in one process.

If you want to install ATS by itself, ATS provides multiple methods to install in their Getting Started section.

ATS Test Helper Functions

We provide the following test helper functions to make defining integration tests easier in tests/test.ats.

  • tolerance_test

    ../../../../tests/integration/test.ats