Why were actual data validations removed?

[marcj]

Currently, only data types itself are tested. This has been introduced in 61b353c. Validation benchmarks without basic data content validation is way too shallow to be called a validation benchmark. At best it is a type guard, but that has nothing to do with data validation. Almost all real-life validation code does use features like enums, negative number checks, string length limits, or email checks, etc. If this is not part of this benchmark, then this benchmark is highly misleading.

If a validator library is not able to validate the actual content for something like number ranges, or string length etc, then it should not be called data validator and not be part of this benchmark suite.

If you focus only on type guards only, without data content checks, then please add this to the README. Then we can remove the benchmark links from Marshal since comparing apple with oranges it the last people want.

[marcj]

Why is it misleading? Because it appears ts-json-validator is extremely fast since this change, and much faster like everyone else. But that's only the case for the synthetic benchmark of testing only for string, boolean, and number data types. Zero actual data content validation involved here. Once you add a simple minLength: 100 or maximum: 0 to a certain field, the speed of this validation library collapses from 33m o/s to 300k o/s. This is a decrease in performance of about 110x as soon as you add one of the cheapest data validator. On the other side, Marshal keeps the performance up: 25m o/s for data type only and 25m o/s with data validators. Do you think this is a fair benchmark and gives people a good intuition when they look at the result diagram?

A benchmark should always aim to cover a broad range of use-cases and real-life code, not very narrow synthetic code that nobody uses in a serious environment.

[moltar]

Validation benchmarks without basic data content validation is way too shallow to be called a validation benchmark.

I guess our opinions differ.

Wikipedia entry for data validation includes data-type check as a subset of data validation.

If you focus only on type guards only, without data content checks, then please add this to the README.

The README does say:

These packages are capable of validating the data for type correctness.

---

I am going to close this issue, because I don't feel there is anything actionable here.

I will open a separate issue that will discuss adding constraint checking back.

[moltar]

Also, want to note that yesterday I already started the work to refactor this code base to allow for additional benchmark suites.

The first additional benchmark suite will be to exclude ts-json-validator from tests as it is an outlier.

[moltar]

@marcj see #152

[marcj]

Ok, I missed that you even stated in the README that this library is not supposed to be a validation benchmark, but just a type checker. I've meanwhile removed our link to this benchmark. Please let me know when you've added a full validation benchmark so I can add it back. Would be a shame if we lost that valuable information.