performance with large source base

[tarpas]

There is a lot of repetition and json array in the .testmondata sqlite database, let's make it more efficient.
@boxed you mentioned you have 20mb per commit Db, so this might interest you.

[tarpas]

ff6fb2b should produce .testmondata with around 25% of previous size. (tested just on pytest source code)
@boxed could you test on your project and report?

[boxed]

Looking at the .testmondata files now, they're ~34MB.

11MB with this change. I like it!

[boxed]

It looks like it's a bit faster, but I have only run two tests, so take it with a grain of salt:

• 28.8s for a run with no changes with this change.
• 38.5s with the old code

[tarpas]

Wow, that's still a long time, I hope it's not all testmon overhead. How long does pytest --collectonly take?
How long does a single shortest unit test take with and without --testmon please?

I'm testing on pytest (https://github.com/pytest-dev/pytest) code base and no changes run takes 3 seconds.

[boxed]

--collectonly takes 8 seconds. We've optimized that a bit to make it only look in specific folders.

I'll get back to you with the unit test run time...

[boxed]

This is all with the old version of testmon we use at work, not the new code from master, and it's with a 38MB .testmon database:

pytest --suite=unit --testmon: ~29s, zero tests run
pytest --suite=unit: ~23s, 2482 tests

So not very encouraging!

If I delete .testmondata and rerun pytest --suite=unit --testmon: ~53s, 2482 tests
Run pytest --suite=unit --testmon again (now with a 6.5MB .testmondata): ~4s

[tarpas]

Ok, thanks a lot for your reports. I'll need to set-up and do performance tuning on some large opensource project with pytest compatible test suite.

[tarpas]

Unfortunately there doesn't seem to be a python open source project with comparable size. Maybe django?
pypa/warehouse is probably still quite small
I assume CPython or Pypy test suites will never run under pytest.

[tarpas]

Bingo, It seems https://github.com/zzzeek/sqlalchemy fits the bill! And I discovered a where do we loose o lot of time. @boxed commit comming!

[tarpas]

This has been fixed in version 1.x