Mep19 continuous integration hangout

Michael Droettboom edited this page Aug 15, 2013 · 1 revision

CI Infrastructure

  • We like Travis and it will probably remain part of our arsenal in any event. The reliability issues are being looked into.

  • Enable Amazon S3 uploads of testing products on Travis. This will help with post-mortem of failures (@mdboom is looking into this now).

  • We want Mac coverage. The best bet is probably to push Travis to enable it for our project by paying them for a Pro account (since they don't otherwise allow testing on both Linux and Mac).

  • We want Windows coverage. Shining Panda is an option there.

  • Investigate finding or building a tool that would collect and synthesize test results from a number of sources and post it to Github using the Github API. This may be of general use to the Scipy community.

  • For both Windows and Mac, we should document (or better yet, script) the process of setting up the machine for a build, and how to build binaries and installers. This may require getting information from Russel Owen and Christoph Gohlke. This is a necessary step for doing automated builds, but would also be valuable for a number of other reasons.

The test framework itself

  • We should investigate ways to make it take less time

    • Eliminating redundant tests, if possible

    • General performance improvements to matplotlib will help

  • We should be covering more things, particularly more backends

  • We should have more unit tests, fewer integration tests, if possible