Performance is getting worse between 0.1.11 and 0.1.12

[stevehu]

I did another performance test with json-schema-validator-perftest by upgrade to 0.1.15 from 0.1.9 and see performance dropped significantly in percentage. Our customers are using this library in production runtime to do schema validation and we have to find out why and get it fixed. I tried to bump version one by one from 0.1.9 and found the changes in between 0.1.11 and 0.1.12 introduced some overhead. I am wondering if someone can help to figure out where is the root cause and get it fixed. Thanks.

0.1.11

Iteration 0 (in 5 ms)
Iteration 20 (in 34 ms)
Iteration 40 (in 51 ms)
Iteration 60 (in 66 ms)
Iteration 80 (in 82 ms)
Iteration 100 (in 100 ms)
Iteration 120 (in 118 ms)
Iteration 140 (in 136 ms)
Iteration 160 (in 152 ms)
Iteration 180 (in 169 ms)
Iteration 200 (in 185 ms)
Iteration 220 (in 198 ms)
Iteration 240 (in 207 ms)
Iteration 260 (in 220 ms)
Iteration 280 (in 226 ms)
Iteration 300 (in 232 ms)
Iteration 320 (in 237 ms)
Iteration 340 (in 242 ms)
Iteration 360 (in 247 ms)
Iteration 380 (in 252 ms)
Iteration 400 (in 258 ms)
Iteration 420 (in 263 ms)
Iteration 440 (in 268 ms)
Iteration 460 (in 273 ms)
Iteration 480 (in 278 ms)
END -- time in ms: 283

0.1.12

Iteration 0 (in 7 ms)
Iteration 20 (in 40 ms)
Iteration 40 (in 60 ms)
Iteration 60 (in 80 ms)
Iteration 80 (in 99 ms)
Iteration 100 (in 120 ms)
Iteration 120 (in 141 ms)
Iteration 140 (in 165 ms)
Iteration 160 (in 183 ms)
Iteration 180 (in 195 ms)
Iteration 200 (in 207 ms)
Iteration 220 (in 219 ms)
Iteration 240 (in 230 ms)
Iteration 260 (in 250 ms)
Iteration 280 (in 259 ms)
Iteration 300 (in 267 ms)
Iteration 320 (in 275 ms)
Iteration 340 (in 283 ms)
Iteration 360 (in 291 ms)
Iteration 380 (in 299 ms)
Iteration 400 (in 307 ms)
Iteration 420 (in 315 ms)
Iteration 440 (in 323 ms)
Iteration 460 (in 331 ms)
Iteration 480 (in 338 ms)
END -- time in ms: 346

[stevehu]

I have upgrade json-schema-validator-perftest to version 0.1.12 so that other people can try it. If someone can upgrade this benchmark with JMH framework, that would be great. I can also make the build process to run the benchmark from light-bot so that we can find out if there are performance issue on every version update.

[stevehu]

With so many features added to the library, it seems nobody cares about the slight performance degradation.