Manifold one year roadmap

Publish date: 08/15/2019; last modified at: 02/22/2020

Time scope: 02/2020 - 02/2021

Features:

• Support more model types
  • √ Regression
  • √ Binary classification
  • [P1] Multi-class classification
  • [P2] Ranking / recommendation model types
• Support more feature types
  • √ Numeric feature
  • √ Categorical feature
  • √ Geospatial feature
  • [P2] Image feature
  • [P2] Textual feature
• Advanced data slicing logic
  • [P0] Allow customizing per-instance model performance metric
  • [P0] Allow customizing data segmentation column(s)
    • √ Automatic segmentation by all model performance scores
    • √ Manual segmentation by feature value
    • [p1]Segmentation by difference between models' performance
    • √ Allowing picking single / multiple models' performance for automatic segmentation
• Architecture improvement for performance, scalability and flexibility
  • √ Allow data joining to support customized or derived model comparison metrics
  • [P1] Use async instead of sync for intensive data computation
  • [P2] Support tensor-based internal data representation
• Metadata user customization
  • [P0] Allow users to change model names
  • [P1] Allow users to monitor and change feature types
• Platform integration
  • √ Integration with Jupyter notebook
    • √ Support dataframe / numpy array data type
  • [P2] Integration with Tensorboard
• User experience and user education
  • √ Add in-UI documentation
  • [P0] Improve error messages for input data
  • [P0] Add on-screen tooltips
  • [P1] Create practical examples using public datasets
• Visualization innovation
  • [P2] Clearer visual encoding for heatmap overlays in geo-spatial data
• Explorative topics
  • [P1] Ways to surface bias as well as variance for selected data slice (currently we focus on bias)
  • [P1] Ways to surface conditional feature distribution for selected data slice (currently only absolute feature distribution is shown)