Compliance and testing

Hoedown attempts to keep compliance with the CommonMark spec.

Targeting

Because there are many versions of the spec, a CommonMark submodule points to the targeted version. This is the only version of CommonMark that should be consulted when implementing CommonMark features, submitting issues or anything else regarding compliance. hoedown_commonmark_version can be used at runtime to know which version of CommonMark is being targeted by the parser.

Testing

make test will use CommonMark's official test runner to test Hoedown against the 500+ examples in the spec. Instead of using the unmodified spec.txt in the submodule, we employ a local version of spec.txt with a few changes to account for small differences between Hoedown and CommonMark.

make test will also use the pathological tester to look for possible bottlenecks, inefficient parser or loops in Hoedown.

Upgrading to newer spec versions

The submodule is regularily updated to target newer versions of the spec. There are no clear rules on when to do this, but when it happens, the maintainer that did the update must:

• Checkout the appropiate version tag in the CommonMark module.
• Update the fields in src/version.h accordingly.
• Carefully review the differences introduced in spec.txt between the two versions, and note those which require changes in Hoedown's code.
• Replace the local spec.txt with the one in the newly checked out CommonMark, and [re]apply the necessary changes according to (and only to) Spec differences.
• Commit the new submodule, src/version.h and the local spec.txt. The usage of :sparkles: in the commit message is completely optional. All this must be committed directly on the develop branch, and then pushed.
• Start a new issue to track the changes required for the parser to be compliant with the new spec. If they are all trivial, the maintainer can just make a second commit with them.

The submodule won't be updated again until the issue is closed and Hoedown tests successfully against the new spec. No new versions should be published until Hoedown is compliant to the targeted spec again.

Exceptions

In case of problems with the spec itself (contradictory statements or examples, bad wording, ambiguity, visible bugs with the syntax), the affected code should be left unchanged, an issue started in jgm/CommonMark, and a FIXME should be added to the code mentioning the created issue. The problematic tests can then be removed from spec.txt, and Hoedown can be considered compliant to the new spec.