Performance metrics and regression tracking

[TzviPM]

From the October 2018 Roadmap in #937:

At some point we will want to start more aggressively measuring and tracking performance. This is a vague, open-ended problem; but ideally we would have something like a CI system for performance, that would let us run some set of real-world benchmarks against new commits, and allow us to visualize changes over time. A good first step for this would be to create a new xi-bench crate that starts putting together these sorts of real-world benchmarks.

[TzviPM]

From the README:

Incredibly high performance. All editing operations should commit and paint in under 16ms. The editor should never make you wait for anything.

[jansol]

What are you trying to say?

[TzviPM]

Sorry, @jansol, I am trying to say that I'd like to take a stab at some performance benchmarks. Unfortunately, there wasn't any issue for that, so I made one.

[jansol]

This issue is a bit too vague to be useful so I'm inclined to just close it. As a rule of thumb I'd say if it a) needs planning b) is somehow controversial or c) is complex enough to have a TODO list, then it needs an issue.

Performance work is definitely a massive enough project to warrant a TODO list but the current wording in the roadmap and readme is too vague to actually write one. If you have a more specific plan please open an issue for that. If you want to just add some metrics or benchmarks here and there feel free to just open a PR without creating an issue; I think most of the time those will be fairly trivial.

[TzviPM]

@jansol, I was thinking of implementing a thin wrapper around the xi-core-lib that spawns xi-editor in a thread and measures performance of commands in an e2e-style test suite. Does that make sense to you for an approach? I suppose we should just chat about this in zulip?

[jansol]

Yes.

[cmyr]

@TzviPM I think this is a great project, and am happy to chat about it!