Benchmarking framework for Cyrus

[brong]

It would be very valuable to have a benchmarking tool to allow us to see how performance has changed over time. This would be something like Cassandane, but with fewer sets of data - maybe some accounts with a large amount of mail, and search indexes, etc.

It could be used to test different hardware layouts as well as changes between different versions of Cyrus, so it would be valuable to have similar commands and ratios of command types to "production" traffic, whatever that counts as. Both IMAP and JMAP.

[rsto]

Cassandane would be a solid base to build on. E.g. We could tar a "golden" set of mailbox folders that get installed during initialization, then run a Cassandane test suite with special metrics hooks on top of it.

That would be enough for blackbox testing. If we want to profile inside Cyrus we could use Cassandane's existing valgrind integration.

[elliefm]

Cassandane already has some tarred mailbox functionality, as part of the implementation ofhttps://github.com/cyrusimap/cassandane/issues/2 -- could possibly build upon this!

On Fri, Sep 8, 2017, at 05:05 PM, Robert Stepanek wrote: Cassandane would be a solid base to build on. E.g. We could tar a "golden" set of mailbox folders that get installed, then run a Cassandane test suite with special metrics hooks on top of it.> That would be enough for blackbox testing. If we want to profile inside Cyrus we could use Cassandane's existing valgrind integration.> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub[1], or mute the thread[2].>

Links: 1. #2139 (comment) 2. https://github.com/notifications/unsubscribe-auth/AKeeryfqhko_tUX6VrVDZ1FoRrSKZ7n-ks5sgOcbgaJpZM4PPsBv