Skip to content

Roadmap

Máté Gulyás edited this page Jan 6, 2020 · 2 revisions

0.1.0

Time 2019-12-18

Planned Features

  • Docker image
  • License
  • Dashboard
  • Workspace page
  • Clusters page
  • Users page

0.1.1

Time 2020 January

Planned Features

  • Codebase refactoring [Design doc needed]
  • Separate views and logic
  • Improved error handling in scraping

0.2.0

Time 2020 March

Planned Features

  • Improved error handling on UI
  • Pluggable settings backend [Design doc needed]
  • Scraping on-demand [Design doc needed]
  • Add CI / CD pipeline [Design doc needed]
  • Alerts page: [Design doc needed]
  • Thresholds defined for Workspaces
  • Outlier detection (windowed avg +/- n * std)
  • Improved monthly cost prediction
  • Based on usage patterns and average daily usage

0.3.0

Time 2020 May

Planned Features

  • Unit test coverage is at least 50% [Design doc needed]
  • Pluggable storage backend [Design doc needed]
  • Design facelift
  • Website

0.4.0

Time 2020 June

Planned Features

  • Tags: Clusters, Workspaces and Users can be tagged [Design doc needed]
  • Tag page: Data can be filtered and aggregated based on filters

1.0.0

Time 2020 Q3

Planned Features

  • Spark Driver metrics collection [Design doc needed]
  • Spark jobs in Clusters page [Design doc needed]
  • Warning for non-spot instance clusters
  • Failed jobs with errors on Workspace and Clusters level
  • Driver metrics are analysed for errors and suggestions
  • Predicted cost for the next 3-6 months
  • Based on the prophet library
  • Budgets for Workspaces, Users and Tags

1.1.0

Time 2020 Q4

Planned Features

  • Data export for further analyses
  • Integrations: Send Grid, Pagerduty [Design doc needed]

1.2.0

Time 2020 Q4

Planned Features

  • Insights for
    • Slow jobs
    • Underutilized clusters
    • Wrong cluster types and what cluster types should be used
    • Unused optimisations (arrow, delta caching and)
    • Skewed joins

1.3.0

Time Unknown

Planned Features

  • User automations
    • Add or remove users from multiple workspaces
    • Import notebook into multiple users and workspaces
    • Create isolated clusters easily for
Clone this wiki locally
You can’t perform that action at this time.