Klipfolio data aggregation

[eljhkrr]

Metrics required:

• Disabled/Enabled data sources
• Refresh fails

https://docs.google.com/spreadsheets/d/1-BMXlMWrpif3fwI4TN-1VlvJNcdbC4HU1fP3SGIARnQ/edit#gid=0

To be aggregated in #29

[eljhkrr]

• Accuracy check
  Muso shows 15 out of 15 data sources failing
  
  VHW has 37 failing data sources shown as 1%
  
• Use stacked bar chart with total data source count, failing data sources & % failing overlay
• Would be nice to get data on disabled data sources

[eljhkrr]

• Klipfolio doesn't strictly align datasource refresh with the time window setting. Currently the widget obtains data by comparing last refresh date with current time to determine if the widget is up to date. However, this results in false positives as seen below with musoapp datasources:
  
  
  This threshold can be standardized to 3 days to improve usability.

• Stacked bar chart for total datasource count is implemented by stacking metrics on top of each other:
  
  Reverted to the layout in the point above for a more natural look.

[eljhkrr]

Feedback from last check-in:

• Introduce a potentially stale datasource metric based on % of refresh interval and a higher % for failing data sources. In the data snippet above, refresh interval of 43200 vs refresh age of 134501 represents 311%. A cut off of 500% would indicate definite failure.

• Bars can be stacked and justified by getting the difference from total number of datasources

[eljhkrr]

I've added a potential failed category in addition to total & failed indicators. The potential failed is a buffer for incidents where klipfolio doesn't refresh according to the set interval, allowing about to 5 cycles without an update.
This is what it looks like for PIH:

Muso:

Brac:

[eljhkrr]

Interesting case in MoH Mali supervisor app where a file-based datasource is set to never refresh and shows as a refresh failure:


This is a false positive and datasources like these should be filtered out