Update schema benchmarks to use s/checker over s/validate #6

Merged
merged 1 commit into from Apr 19, 2016

Conversation

Projects
None yet
2 participants
@w01fe
Contributor

w01fe commented Apr 19, 2016

For schema, I think using s/validate is not a fair comparison with the other libraries -- as you mention, it throws an exception on error, and it also has to parse the schema each time.

I think s/checker is the better construct to use here -- it returns an error or nil for success (no exceptions), and only parses the schema once. (This is what's used by s/defn, for example).

@w01fe

This comment has been minimized.

Show comment
Hide comment
@w01fe

w01fe Apr 19, 2016

Contributor

Here are the :quick results I get after this change:

+-------------------------------------+-----------+------------+
| Test name                           | Library   | Mean (ns)  |
+-------------------------------------+-----------+------------+
| [:associative-collections :valid]   | :annotate |  19163.169 |
| [:associative-collections :valid]   | :herbert  |  16818.439 |
| [:associative-collections :valid]   | :schema   |   4908.251 |
| [:associative-collections :invalid] | :annotate |  27222.983 |
| [:associative-collections :invalid] | :herbert  |  13004.246 |
| [:associative-collections :invalid] | :schema   |   9286.991 |
| [:atomic-values :valid]             | :annotate |   1186.785 |
| [:atomic-values :valid]             | :herbert  |   6093.672 |
| [:atomic-values :valid]             | :schema   |   1344.102 |
| [:atomic-values :invalid]           | :annotate |  10511.393 |
| [:atomic-values :invalid]           | :herbert  |   6188.306 |
| [:atomic-values :invalid]           | :schema   |   1099.927 |
| [:custom-predicate :valid]          | :annotate |  10745.051 |
| [:custom-predicate :valid]          | :herbert  |  26008.728 |
| [:custom-predicate :valid]          | :schema   |   3116.197 |
| [:custom-predicate :invalid]        | :annotate |  10997.276 |
| [:custom-predicate :invalid]        | :herbert  |   8876.375 |
| [:custom-predicate :invalid]        | :schema   |   3679.138 |
| [:nil-allowed :valid]               | :annotate |   6446.034 |
| [:nil-allowed :valid]               | :herbert  |  10330.387 |
| [:nil-allowed :valid]               | :schema   |   3030.199 |
| [:nil-allowed :invalid]             | :annotate |  42792.417 |
| [:nil-allowed :invalid]             | :herbert  |  29767.560 |
| [:nil-allowed :invalid]             | :schema   |   3265.008 |
| [:sequential-collections :valid]    | :annotate |  21566.404 |
| [:sequential-collections :valid]    | :herbert  |  32325.145 |
| [:sequential-collections :valid]    | :schema   |   5708.073 |
| [:sequential-collections :invalid]  | :annotate |  32586.399 |
| [:sequential-collections :invalid]  | :herbert  |  21720.297 |
| [:sequential-collections :invalid]  | :schema   |   5752.001 |
+-------------------------------------+-----------+------------+
Contributor

w01fe commented Apr 19, 2016

Here are the :quick results I get after this change:

+-------------------------------------+-----------+------------+
| Test name                           | Library   | Mean (ns)  |
+-------------------------------------+-----------+------------+
| [:associative-collections :valid]   | :annotate |  19163.169 |
| [:associative-collections :valid]   | :herbert  |  16818.439 |
| [:associative-collections :valid]   | :schema   |   4908.251 |
| [:associative-collections :invalid] | :annotate |  27222.983 |
| [:associative-collections :invalid] | :herbert  |  13004.246 |
| [:associative-collections :invalid] | :schema   |   9286.991 |
| [:atomic-values :valid]             | :annotate |   1186.785 |
| [:atomic-values :valid]             | :herbert  |   6093.672 |
| [:atomic-values :valid]             | :schema   |   1344.102 |
| [:atomic-values :invalid]           | :annotate |  10511.393 |
| [:atomic-values :invalid]           | :herbert  |   6188.306 |
| [:atomic-values :invalid]           | :schema   |   1099.927 |
| [:custom-predicate :valid]          | :annotate |  10745.051 |
| [:custom-predicate :valid]          | :herbert  |  26008.728 |
| [:custom-predicate :valid]          | :schema   |   3116.197 |
| [:custom-predicate :invalid]        | :annotate |  10997.276 |
| [:custom-predicate :invalid]        | :herbert  |   8876.375 |
| [:custom-predicate :invalid]        | :schema   |   3679.138 |
| [:nil-allowed :valid]               | :annotate |   6446.034 |
| [:nil-allowed :valid]               | :herbert  |  10330.387 |
| [:nil-allowed :valid]               | :schema   |   3030.199 |
| [:nil-allowed :invalid]             | :annotate |  42792.417 |
| [:nil-allowed :invalid]             | :herbert  |  29767.560 |
| [:nil-allowed :invalid]             | :schema   |   3265.008 |
| [:sequential-collections :valid]    | :annotate |  21566.404 |
| [:sequential-collections :valid]    | :herbert  |  32325.145 |
| [:sequential-collections :valid]    | :schema   |   5708.073 |
| [:sequential-collections :invalid]  | :annotate |  32586.399 |
| [:sequential-collections :invalid]  | :herbert  |  21720.297 |
| [:sequential-collections :invalid]  | :schema   |   5752.001 |
+-------------------------------------+-----------+------------+

@muhuk muhuk merged commit 35771e6 into muhuk:master Apr 19, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment