Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace testspec with TOML #3459

Closed
thisismiller opened this issue Jul 8, 2020 · 0 comments
Closed

Replace testspec with TOML #3459

thisismiller opened this issue Jul 8, 2020 · 0 comments
Assignees
Milestone

Comments

@thisismiller
Copy link
Contributor

"testspec" is the name given to the INI-like file format used to write *.txt tests in the tests/ directory.

As proposed on the forums, I'll be working to replace testspec with TOML, which will look very similar, but be an actual format with a specification and a parser.

I believe that this work will come in a series of four PRs:

  1. Make testspec more restrictive in terms of what it accepts. The parser does horrible things with regards to parsing arbitrary lines within a workload as a global setting, that I will not be able to convince any other parser to be equally terrible. Instead, I'll be making testspec enforce that global settings are set globally, test-level settings are set at the test level, and workload settings stay within workloads. This change should be entirely backwards compatible if I understand the testspec parser right.

  2. TOML is a typed configuration file format, and testspec treats everything as a string. This means that getOption() returns a string for all workload options, and that will need to change. getOption will need to become a templated function that specifies (or is able to deduce) the type that is being read. A large refactoring/code rewriting will be needed to push the type expectations from how workloads handle the string output to the getOption() call itself.

  3. Introduce toml11 into the build system, and use it to build a parser that can convert a .toml file into the std::vector<TestSpec> that is expected of the parser. fdbserver should then be able to correctly load and run .txt files or .toml files, and all .txt files that are not restarting tests will be converted to TOML.

  4. A small script will be added and integrated into the test harness, so that I can produce .txt files out of .toml files, as this will be necessary for older fdbserver binaries to be able to "read" .toml files. The restarting tests will be converted to .toml, and then the "testspec" parser will be removed from the codebase.

Probably in (3) I'll also include a writeup of testspec and the format expected by its TOML replacement.

Any future changes to the section names, key names, or general layout of the TOML file will require similar changes to the script added in (4) to be able to generate a TOML file that a previous version of FoundationDB would understand.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants