YAML Test Framework Matrix
Its output is currently saved in the gh-pages branch of yaml-test-suite.
It shows which frameworks pass which test cases:
|Test ID||Framework 1 Events||FW2 Events||FW1 JSON||...|
It only shows results of valid YAML files currently.
YAML Test Suite
The Test Suite is a collection of over 250 YAML test cases.
Every test case contains some of the following files:
- in.yaml: The input YAML
- test.event: Parser events in a specific format
- in.json: The corresponding JSON that matches the loaded data structure
- out.yaml, emit.yaml
- error: Marks YAML files that are invalid
The editor consists of code to create a Docker image that has currently 16 different YAML frameworks installed.
For each framework there exist scripts to output the yaml-test-suite parsing event format, the loaded data structure in native format and in JSON.
It also contains a script that will open Vim in the Docker container with some fancy mappings, so you can test YAML input for as manu frameworks at the same time.
Now the matrix takes all test cases from yaml-test-suite and runs them through the various frameworks in the yaml-editor Docker container.
It checks if the program died, or if the result matches the expected output. It does some transformations as not all frameworks support all features.