checks integrity of test data in programming contests like the ACM ICPC
Switch branches/tags
Clone or download
eldering Improve Makefile in developer/maintainer mode:
- Don't unnecessarily rebuild the lexer/parser after rerunning configure.
- Add 'maintainer-clean' target to regenerate a new version.
Latest commit 9703d55 Nov 19, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
debian Happy new year. Jan 12, 2017
doc After REPI and WHILEI, the loop variable contains the number of itera… Nov 19, 2018
examples Move example files to examples directory, add from NWERC 2008. Apr 22, 2018
haskell_edsl [haskell_edsl] Add examples Jun 14, 2017
m4 Update autoconf M4 and install-sh scripts. Feb 12, 2017
tests After REPI and WHILEI, the loop variable contains the number of itera… Nov 19, 2018
.gitignore Add Makefile rules to configure and build for Coverity scan. Jun 5, 2016
.travis.yml Re-enable using C++11 regex instead of LibBoost. Jun 3, 2016
COPYING Happy new year. Jan 12, 2017
Makefile Improve Makefile in developer/maintainer mode: Nov 19, 2018
README.md Move example files to examples directory, add from NWERC 2008. Apr 22, 2018
bootstrap Allow silent execution of bootstrap script and Makefile. Jun 2, 2016
checktestdata.cc Fix 096c0cd for older GCC versions. Apr 22, 2018
checktestdata.l Add FLOATP command with specified number of digits. Closes #5 Oct 20, 2016
checktestdata.y Add FLOATP command with specified number of digits. Closes #5 Oct 20, 2016
config.guess Make checktestdata compile standalone. Nov 12, 2014
config.mk.in Re-enable using C++11 regex instead of LibBoost. Jun 3, 2016
config.sub Make checktestdata compile standalone. Nov 12, 2014
configure.ac Add autoconf-archive macro to check for C++11 compiler support. Aug 26, 2016
databuffer.hpp Fix performance regression in 62119e5. Nov 18, 2018
install-sh Update autoconf M4 and install-sh scripts. Feb 12, 2017
libchecktestdata.cc Fix performance regression in 62119e5. Nov 18, 2018
libchecktestdata.hpp Also set float precision when generating data, raise to Feb 26, 2015
parser.h Add option to run parser with a different start symbol, Feb 26, 2015
parser.ih Fix compile error in checktestdata. Feb 26, 2014
parsetype.cc Move value_t to parsetype code. Apr 27, 2018
parsetype.hpp Fix compilation error in clang++ failing const expr cast to std::string. Nov 19, 2018
scanner.h Add option to run parser with a different start symbol, Feb 26, 2015
scanner.ih Replace flex++ by more modern flexc++. Oct 10, 2013

README.md

Checktestdata

Build Status Coverage Status Coverity Scan Status

Checktestdata is a tool to verify the syntactical integrity of test cases in programming contests like the ACM ICPC.

It allows you to specify a simple grammar for your testdata input files, according to which the testdata is checked. In the examples directory you find two sample scripts hello.ctd and fltcmp.ctd from the DOMjudge sample problems hello and fltcmp, and under examples/nwerc2008 the scripts used for the NWERC 2008 problemset.

Grammar specification

Checktestdata language specification

We also have a Haskell implementation of the checktestdata program, which supports this specification as well as a Haskell embedded domain-specific language.

Installation and use

Requirements:

If you don't have flexc++ and/or bisonc++ available, you may use the release branch where we've pre-generated the scanner/parser files.

Command line for installing the build dependencies on Debian or Ubuntu, when using the release branch:

apt-get install make g++ libboost-dev libgmp-dev autotools-dev automake

For Redhat-like distributions try:

yum install make g++ boost-devel gmp-devel automake

To compile checktestdata, run:

./bootstrap
make dist
make

Leave out the make dist step if you use the pre-generated scanner/parser files on the release branch.

Finally, you can run

make check

to run a number of tests. Note that a few of the tests are non-deterministic and may sometimes return an error. This is ok, but the make check command should run successfully more than 90% of the time.

For usage instructions run:

checktestdata --help

Library

The commandline program is built upon the separate library libchecktestdata.hpp (see checktestdata.cc as an example for how to use this library) that can be used to write the syntax checking part of special compare scripts. It can easily handle the tedious task of verifying that a team's submission output is syntactically valid, leaving just the task of semantic validation to another program.

When you want to support presentation error as a verdict, also in variable output problems, the option whitespace-ok can be useful. This allows any non-empty sequence of whitespace (no newlines though) where the SPACE command is used, as well as leading and trailing whitespace on lines (when using the NEWLINE command). Please note that with this option enabled, whitespace matching is greedy, so the script code

INT(1,2) SPACE SPACE INT(1,2)

does not match 1__2 (where the two underscores represent spaces), because the first SPACE command already matches both, so the second cannot match anything.

Copyright & Licencing

Checktestdata is Copyright © 2008 - 2018 by the checktestdata developers and all respective contributors. The current checktestdata developers are Jeroen Bransen, Jaap Eldering, Jan Kuipers, and Tobias Werth; see the git commits for a complete list of contributors.

Checktestdata, including its documentation, is free software; you can redistribute it and/or modify it under the terms of the two-clause BSD license. See the file COPYING.

The M4 autoconf macros are licenced under all-permissive and GPL3+ licences; see the respective files under m4/ for details.

Contact

The developers can be reached through the mailinglist domjudge-devel@domjudge.org. You need to be subscribed before you can post. Information, subscription and archives are available at: https://www.domjudge.org/mailman/listinfo/domjudge-devel

Some developers and users of checktestdata linger on the IRC channel dedicated to DOMjudge on the Freenode network: server irc.freenode.net, channel #domjudge