vbench for statsmodels #936

Closed
josef-pkt opened this Issue Jul 2, 2013 · 11 comments

Projects

None yet

4 participants

@josef-pkt
Member

statsmodels still doesn't have a vbench for performance monitoring.

One target for statsmodels 0.6 could be to look for performance improvements, and check whether some previous changes caused serious performance regressions.

some references
https://github.com/pydata/vbench/tree/master/vbench
https://github.com/pydata/pandas/tree/master/vb_suite
https://github.com/yarikoptic/numpy-vbench

@vincentarelbundock
Member

I can look into vbench. Will need a bit of time.

@TomAugspurger
Contributor

I could also give it a shot in a few weeks. Do you have any experience with it @vincentarelbundock? I started to setup a version for a project I'm working on and gave up after an hour or so.

If I do take it on I'll write up some docs for future statsmodels devs and perhaps push those upstream to vbench. They are sorely sparse/nonexistent :-)

Either way, the project could be broken into parts. There's this issue of setting up the framework, and then there's the actual writing of the benchmarks. I'd be happy to help in either pace.

@vincentarelbundock
Member

No experience here. Go ahead if you feel like it. Little experience is better than no experience.

@TomAugspurger
Contributor

Ok thanks. Like I said, it will be a few weeks.

@josef-pkt
Member

@vincentarelbundock @TomAugspurger Thank you, that would be great.

I don't know anything about setting up a vbench, I was just following Yarik a bit who set up a vbench for numpy and it looks very useful.

I can also help with the benchmarks.
I would also like to try to set it up on Windows, but I'd rather have a version that I know works on Linux before I get into Windows specific problems (if there are any).

@TomAugspurger
Contributor

Ok. As I'm going through I'll note down the unix/windows idiosyncrasies.

@josef-pkt
Member

I think we can try to host the results from the vbench runs on gh-pages.

I started a new repo and a dummy page
https://github.com/statsmodels/vbench_results/tree/gh-pages
http://statsmodels.github.io/vbench_results/index.html

I still haven't tried to run the vbench on Windows.

@jseabold
Member
jseabold commented Dec 9, 2013

Why not use sourceforge for hosting with the rest of the docs?

@josef-pkt
Member

no special reason for not using sourceforge, except I thought it will be easier to push

The only problem I see is that we will have to purge the git history sometimes to prevent it from getting too large.

@jseabold
Member
jseabold commented Dec 9, 2013

Push is just a scp to sourceforge. Easier than using version control I think.

@jseabold jseabold closed this Apr 2, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment