Portable Testsuite to check a C / C++ Compiler and the standard runtime library

Many compiler and the related standard library provide tests, but they are not easy usable as portable/standalone conformance check for other compiler or runtime library implementations.

Goals for stdtests

• A reusable license
• Only needs a Compiler and a posix shell to run the testsuite
• Portable: Works on many systems with many compiler and runtime libraries
• Check C Compiler and the C Runtime library
• Check C++ Compiler and the C++ Runtime library
• Create a nice Table from multiple Results

Expected usecase

• Extract the tests
• configure the testsuite to test a compiler / runtime library for a given standard
• make: run the testsuite and generate a result file
• make table: generate a table for multiple results

Example usage

• std: newest
• Compiler: tcc
• runtime-library: system-default

mkdir results
cd results
CC=tcc ../configure --disable-c++
make

• Now there should be an document with the results

Portability

• Current tested environments

  Target OS	tested Compiler
  Linux	tcc / gcc / g++ / owcc / clang / zig cc / zig c++
  Windows	x86_64-win32-tcc / x86_64-w64-mingw32-gcc / x86_64-w64-mingw32-g++

• Hints / Links for later use:

  • With many years delay, a command line switch arrived in MSVC 2015 Update 3 for selecting the c++ version

Checking C Compiler and the C Runtime library

check	C23	C17	C11	C99	C95	C89/C90 / ANSI-C

Checking C++ Compiler and the C++ Runtime library

check	C++2c	C++23	C++20	C++17	C++14	C++11	C++03	C++98

Some examples of other tables with Test-Results

• gcc Status of C99 features in GCC

• gcc C11 Status in GCC

• gcc C++ Standards support in GCC

• clang C++ Support in Clang

• msvc C/C++ Support in MSVC

• cppreference.com What's new in C99

• cppreference.com C99 Implementation status

• cppreference.com C23 Implementation status

• cppreference.com C++ Implementation status

Some links to compiler / library tests

Name	URL
clang	test top dir
gcc	test top dir
mcpp	preprocessor
musl	libc
glibc	libc

---

C Reference Documentation

• ISOC SC22/WG14 C standards commitee
• ISO SC22/WG14 docs
• ISO SC22/WG14 document log
• C23 state at iso.org: 40:Enquiry (40.60 since 07. Dec. 2023), waiting for 50:Approval

std	__STDC_VERSION__	Latest Draft	Release
C23	202311	N3096	ISO/IEC 9899:2023
C17	201710	N2176	ISO/IEC 9899:2018
C11	201112	N1570	ISO/IEC 9899:2011
C99	199901	N1256	ISO/IEC 9899:1999
C95	199409	c89_na1: c94	ISO/IEC 9899:1990/AMD1:1995
C90	not defined	Missing	ISO/IEC 9899:1999
C89	not defined	c89 draft	ANSI X3.159-1989

---

C++ Reference Documentation

• ISOCPP SC22/WG21 C++ standards commitee
• C++ standard core language issues list
• C++ standard library issues list
• C++26 First draft was N4958
• C++23 state at iso.org: 50:Approval (50.00 since 17. Jan. 2024), waiting for 60:Publication

std	__cplusplus	Latest Draft	Release
C++26	N/A	N4971	N/A
C++23	202302	N4950	ISO/IEC 14882:2023
C++20	202002	N4849	ISO/IEC 14882:2020
C++17	201703	N4659	ISO/IEC 14882:2017
C++14	201402	N3797	ISO/IEC 14882:2014
C++11	201103	N3242	ISO/IEC 14882:2011
C++03	200300	c++03 final	ISO/IEC 14882:2003
C++98	199711	nr. unknown	ISO/IEC 14882:1998

---

Maintainer Mode hints

• use 'bootstrap' to regenerate the build system files
• use 'configure' with "--enable-maintainer-mode" to rebuild all programs all tests groups are enabled by default:
  • C
  • C++ (alias names: cxx / cplusplus)
  • libc
  • libc++ (alias name: libcxx)

---