New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a test tool that will allow comparison between releases to identify regressions #33

Closed
jeking3 opened this Issue Oct 18, 2017 · 0 comments

Comments

Projects
None yet
1 participant
@jeking3
Collaborator

jeking3 commented Oct 18, 2017

In general, Boost.Format is a stable and long-lived component and used by many consumers. Changes to the behavior of Boost.Format could cause widespread issues. A mechanism is needed that will allow comparison of one edition of Boost.Format to another to ensure there haven't been any regressions.

Proposal

Create a matrix tester that will attempt to generate every possible format syntax string expression, and test against a variety of inputs appropriate to that type, and output the results to a text file which will be considered the results file. The results file format is:

# glibc.version = n
{format specification}<tab>OK |ERR<tab>string result or exception message
...

The tool will also allow for results to be generated against the platform's snprintf call, so that boost::format and snprintf on the same inputs and format specifications can be compared.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment