Roadmap Shogun 2015 hack

Heiko Strathmann edited this page Feb 28, 2015 · 25 revisions
Clone this wiki locally

Shogun hack 2015 Ideas

This is a list of topics to address when all Shogun developers get together. We aim to have a structured list of items, with links to existing or newly created github issues. Feel free to merge this with content from the other Wiki pages or discussions.

Usability.

Efficiency & Clean-ups

Computing Backends.

  • Parallelism backend interface
    • Batch cluster backend (PBS/SGE,SLURM,etc) #1622
    • Pthread/openMP backend #1623
    • Director classes to use Shogun as scheduler from modular interface #1624
    • Add dependencies between jobs #2524
    • Graphlab backend #2525
    • Zookeeper coordination?
  • Populate internal Linear algebra interface
    • Add matrix factorisations #2526
    • Add linear solves #2527
  • Stan for autodiff and MCMC #1875 #1998 #1929
  • Vowpal Wabbit update

SHOGUN goes large-scale

  • Investigate how non-toy examples scale in SHOGUN.
  • Increase awareness of benchmarking and profiling:
    • Identify hotspots using selected examples and right tools (gprof?).
    • Create minimal example(s!) which covers hotspot and benchmark.
    • Start optimizing minimal example(s!) and benchmark again.
  • Developers should know how to benchmark and profile: Small document to describe this?
  • If you know examples that scale better in other frameworks, please document them here, so we can investigate:
    • ...
  • Better support of sparse-datastructures and online-algorithms.
  • Interface with MOE for better model selection