Split dashboard is slow to load when you have a large number of experiments

[nfm]

As per #329, the Split dashboard can be slow (30+ seconds) to load if you have a lot of experiments (40 or more), which can cause timeouts on Heroku.

From my previous testing, it seemed like the #calc_alternative_probabilities method was the bottleneck, partly caused by the large default value for beta_probability_simulations (the default is currently 10,000).

I'm not really across the statistics used to calculate this stuff. Does anyone know if it's possible to speed this up at all (using another algorithm), or reduce the default number of iterations (to something small enough so that the page loads with a couple of hundred tests) without compromising the result?

I'm currently working around this by setting beta_probability_simulations to 1, and just not using that feature.

/cc @caser

[gfried]

I'm a designer, not a dev, so forgive the ignorance, but as far as you know, has anyone written any mods that would allow customization of the split dashboard so you could, for example, group (and/or hide) closed tests, sort by date, etc...? My company uses and loves Split but finding a given test on the dashboard is a chore.

And if there's someone else better suited for this question, please point me to them!

Gabe

[nfm]

@gfried I haven't come across this, but I think the features you mentioned would be useful to everyone, so I expect a PR that added them would be accepted.

It looks like it wouldn't be straightforward to customize the dashboard page yourself - it's not currently written to be extensible. You could replace the whole thing, but that would be sub-optimal.

I think the best approach would be to fork the repo and add this functionality, and then submit a PR. I think you'd only need to touch the view files (https://github.com/splitrb/split/tree/master/lib/split/dashboard/views) and the dashboard.js (https://github.com/splitrb/split/blob/master/lib/split/dashboard/public/dashboard.js) to add features like sorting and filtering.

[gfried]

@nfm Great, thanks for the input. I'll dig in.

[mezis]

@nfm: this seems to have been fixed in v1.3.0 (which fixed the caching of winning alternatives). Our long dashboard still isn't fast, but it's bearable now 😉