pytest.benchmark

[RSabet]

do you want something like this?

[mre]

Yeah exactly!
Seems like we just need to port the remaining tests, right? If that is true I would merge it and we can port the tests one by one.
Can you reintegrate the master? Seems like the Travis build fails for the exact same reason as yesterday and I've merged @wdv4758h's fix in #17 now. 😅

[RSabet]

had to rename benchmark to benchmarks, hope this doesn't break anything and rename benchmark.py to test_benchmark.py

Tests are working fine: pytest tests

benchmarks can be invoked:

• just hyperjson: pytest benchmarks
• compare to other libraries: pytest benchmarks --compare-other-engines

for some reason '100 arrays dict' - will open an issue since it is too late

[mre]

Great progress. 😊
Tested on my machine and it works beautifully. Just a few nits before we can merge this:
Could you also adjust the Makefile to use pytest commands instead of my previous, hacky approach?
Something like this:

.PHONY: test
test:
	pytest tests

.PHONY: bench
bench:
	pytest benchmarks

.PHONY: bench-all
bench-all:
	pytest benchmarks --compare-other-engines	

[mre]

I'd prefer to have a separate block per test object.

doubles = [sys.maxsize * random.random() for x in range(256)]

unicode_strings = [ "نظام  لولاية ا...رعيا لابوين عمانيين " for x in range(256)]

and so on.
The advantage would be that it's easier to reason about what the final test object might look like and adding/removing a test object just means adding/removing a block.

The disadvantage is, that there's some redundancy, but I think it's worth it.
Also, we could also extract that magic number 256 to some variable.

[wdv4758h]

Here comes a result on one of my machine. hyperjson is at 3rd or 4th in many benchmarks, but it's at the 1st for:

• test_loads[hyperjson-256 doubles array]
• test_dumps[hyperjson-256 doubles array]
• test_loads[hyperjson-256 unicode array]
• test_dumps[hyperjson-256 unicode array]

For those benchmarks we are winning, we usually have clear margin. If this number is right, I think it's impressive for a super young project 😃

[wdv4758h]

I think it will be good to have standard library's json module in the benchmark too, since we won't want to be slower than the default one. And 3rd implementation are actually easy to be slower for composite objects 😛

[wdv4758h]

In my test, we are slower than standard library's json in these benchmarks:

• test_loads[hyperjson-100 dicts array]
• test_loads[hyperjson-256 Trues array]
• test_dumps[hyperjson-256 Trues array]
• test_loads[hyperjson-256 ascii array]
• test_loads[hyperjson-complex object]
• test_dumps[hyperjson-complex object]
• test_loads[hyperjson-composite object]
• test_dumps[hyperjson-composite object]

[RSabet]

although tests and benchmarks work locally, i can't make them pass in travis-ci, can someone help?

[mre]

Hum... it looks like you ran into a pytest bug. 😅
If I move the benchmarks/conftest.py to the project root folder, it works for me.
Here's the issue: pytest-dev/pytest#1596
Have nothing against keeping the conftest.py in the project root for now. 😉

[mre]

Thanks for this @RSabet 🎉