ver2.03a

Library changelog

• Implemented a notion of forwarders that allow clients to listen and react to the TestingBatch events in a composable way
• Added a new type of error called StringWarning that can be used by users to differentiate between soft and hard errors
• Improved error diagnostics and fixed the behavior of PositiveDuration parsing
• NotValueOfType now may wrap inner errors

The main feature introduced are definitely forwarders that will allow to develop complex and modular logic in the client.

CLI changelog

• Fixed bad color formatting on Windows cmd.exe console
• Added support for parser warnings
• Deprecated duration being expressed without a unit suffix in inputs.txt
• Fixed unknown configuration fields being treated as errors. Now they produce warnings
• Negative exit codes are detected and printed as such (instead of numbers like 4294967295)
• Upon parsing errors and internal error (IE) verdicts, scold exits with non-zero exit code
• Colored text and progress bar are not printed if environment doesn't support color (though the behavior can be overridden with --force-colors and --force-progress)

ver2.02a

This release introduces a much more rigorous validation of the inputs.txt file. In particular, we implemented a deserialization routine that uses go's reflection to parse a set of key-value pairs into an arbitrary structure with arbitrarily typed fields. This proved so flexible, in fact, that we are now able to parse durations for tl option of forms 5s, 0.1ms, 12m, or 1ns! Old format (without time suffix) still works and assumed to specify seconds.

With the error generation, we also introduce a "compiler-like" error messages. Each error specifies the file, line, error description, and a code snippet where error was found. Not only the test suite option errors are printed this way, but old ones too.

CHANGELOG:

• Added type-checking for the inputs.txt.
• Added pretty and descriptive error messages.
• Fixed: crashes on very small time limits.
• Added: stdout and stderr are printed (if contain anything) on TL.
• Added: prevent test suite from running, if executable is a directory.

ver2.01a

This release introduces support for worker pools. Unlike the previous versions that executed all tests simultaneously, now you can specify how many instances of your application do you want to test in parallel. We also have a progress bar now!

Changelog:

• Added -j flag to specify the number of worker threads to use
• Added: -j flag's default value is the number of available cores on user's CPU
• Added progress bar
• Added --no-progress option to disable progress bar
• Added ability to pass command-line arguments to the target executable (handy for interpreted languages)
• Added target executable look-up in PATH (disable by writing ./my.exe instead of my or my.exe)
• Fixed: spawned processes are killed if exceed time limit

ver2.00b

Fix the cptest --version version.

ver2.00a

This release bumps up the version of cptest to the next major, as it should've have when releasing ver1.01a.

This release introduces:

• Colors on Windows
• --no-colors option
• Stderr is printed on WA
• Non-zero exit codes are printed
• Mismatched newlines are rendered as \n rather than a block character

With stderr being printed on WA, it is possible to non-intrusively debug code.

ver1.01a Backend rewritten

Compared to the old version, this release introduces a new output comparison engine that compares respective lexemes, instead of raw text. This way the comparison becomes insensitive to the white-spaces and tolerates minor differences. Next, cptest now is able to behave correctly with floating point numbers by correctly ignoring too precise digits. Finally, with the new engine a support for mismatch highlighting was implemented.

• Lexeme-based comparison
• Floating-point number support
• Mismatch highlighting
• Excessive line feeds don't screw up the rest of the output comparison anymore.

ver1.00b

Automatic executable searching on Windows is currently unavailable.

For help on flags and the syntax of the inputs files, run cptest -h.

Features:

• Lenient checking of the program's output against expected output
• Specify a time limit for execution
• Runtime errors are handled

Please note, that the input, expected output, and program's output is space-trimmed line-wise and empty lines are skipped.

CHANGELOG:

* Fixed incorrect path construction on Windows. 

ver1.00a

Automatic executable searching on Windows is currently unavailable.

For help on flags and the syntax of the inputs files, run cptest -h.

Features:

• Lenient checking of the program's output against expected output
• Specify a time limit for execution
• Runtime errors are handled

Please note, that the input, expected output, and program's output is space-trimmed line-wise and empty lines are skipped.