Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proof of Concept: read locations, parameters and timeseries from databricks #90

Open
4 tasks done
d2hydro opened this issue Mar 3, 2023 · 1 comment
Open
4 tasks done
Labels
enhancement New feature or request

Comments

@d2hydro
Copy link
Collaborator

d2hydro commented Mar 3, 2023

Ultimate goal is to be able to read from Azure as a source for the WIK hydrodashboards app.

With this issue we purely test performance with a basic POC bokeh dashboard

Actions we take:

  • Read locations via pd DataFrame into bokeh Select object
  • Read parameters via pd DataFrame into bokeh Select object
  • Read timeseries via pd DataFrame into bokeh Column DataSource shown in a Bokeh figure
  • User can update data in Bokeh figure when selecting a location/parameter
@d2hydro d2hydro self-assigned this Mar 3, 2023
@d2hydro d2hydro added the enhancement New feature or request label Mar 3, 2023
@d2hydro
Copy link
Collaborator Author

d2hydro commented Mar 3, 2023

Required improvements on AAM side:

  • add a list of available parameters per object in silver_v2.dim_measurement_object
  • Include grouping of objects with their time-series (equivalent to FEWS filters)

Nice to haves from AAM side

  • SQL statement for getting the header of a databricks table, as we won't be able to construct it if the List[Row] result gives an empty list
  • sampling/filtering/shading of large timeseries (> 100000 values) for dashboarding (so we won't need to do that on the bokeh-side

@d2hydro d2hydro removed their assignment Mar 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: Review & Test
Development

No branches or pull requests

0 participants