Permalink
Browse files

Working NOAA dashboardd with urth-core-watch patch.

  • Loading branch information...
vinomaster committed Nov 20, 2015
1 parent 856085b commit 0060324e6618afa52c571278f86876a60cce2899
Showing with 153 additions and 90 deletions.
  1. +129 −90 noaa/hdtadash/dev_weather_dashboard.ipynb.yaml
  2. +24 −0 noaa/hdtadash/urth-core-watch.html
@@ -15,10 +15,8 @@ cells:
- cell_type: markdown
source: |-
>STATUS NOTES:
1. Work in progress
2. Code works when executed in notebook view manually evaluating cells after each user interaction.
3. Eventing for dashboarding view not work.
4. Map does not render even though code works using test notebook folium.ipynb
1. all works using urth-core-watch patch and a few hacks.
2. ToDo: revisit code after v0.1.1 is released.
metadata:
urth:
dashboard:
@@ -53,7 +51,7 @@ cells:
layout:
col: 0
height: 20
row: 95
row: 131
width: 12
- cell_type: markdown
source: |-
@@ -131,7 +129,10 @@ cells:
<link rel="import" href="urth_components/google-map/google-map-marker.html" is='urth-core-import' package='GoogleWebComponents/google-map'>
<link rel="import" href="urth_components/urth-viz-table/urth-viz-table.html" is='urth-core-import'>
<link rel="import" href="urth_components/urth-viz-chart/urth-viz-chart.html" is='urth-core-import'>
<!-- Add custom Polymer Widget for injecting raw HTML into a urth-core widget -->
<link rel="import" href="./urth-raw-html.html">
<!-- HACK: Use Property Watch patch for v0.1.0 of declarativewidgets; This can be removed for v0.1.1 -->
<link rel="import" href="./urth-core-watch.html">
metadata:
collapsed: false
urth:
@@ -156,19 +157,36 @@ cells:
dashboard: {}
execution_count: null
- cell_type: markdown
source: |2-
source: |-
### Prepare Filesystem
#### Data Preparation Options
1. Use the [NOAA data Munging](https://github.com/ibm-et/jupyter-samples/tree/master/noaa/etl) project to generate CSV files for the latest NOAA data.
2. Use the sample **March 16, 2015** snapshot provided in this repo. Open a terminal session and run these commands:
```
$ cd /home/main/notebooks/noaa/hdtadash/data/
$ tar -xvf station_summaries.tar
```
2. Use the sample **March 16, 2015** snapshot provided in this repo and do one of the following:
* Open a terminal session and run these commands:
```
cd /home/main/notebooks/noaa/hdtadash/data/
tar -xvf station_summaries.tar
```
* Enable, execute and then disable the following bash cell
metadata:
urth:
dashboard:
hidden: true
- cell_type: raw
source: |-
%%bash
cd /home/main/notebooks/noaa/hdtadash/data/
tar -xvf station_summaries.tar
metadata:
urth:
dashboard: {}
- cell_type: markdown
source: |-
#### Plot Storage
Earlier versions of this notebook stored chart images to disk. We used a specific directory to store plot images (*.png files). However, this approach does not work if the notebook user would like to deploy as a local application.
@@ -380,15 +398,6 @@ cells:
urth:
dashboard: {}
execution_count: null
- cell_type: markdown
source: |-
## Visualization
In this section of the notebook we will define the widgets and supporting functions for the construction of an interactive dashboard. See [Polymer Data Bindings](https://www.polymer-project.org/1.0/docs/devguide/data-binding.html) for more details.
metadata:
urth:
dashboard:
hidden: true
- cell_type: code
source: |-
# Use the Polymer Channel API to establish two-way binding between elements and data.
@@ -402,18 +411,28 @@ cells:
channel("noaaquery").set("cleanupToggleOptions", ["Yes","No"])
channel("noaaquery").set("cleanupPreference", "No")
channel("noaaquery").set("displayTypeOptions", ["Data","Map"])
channel("noaaquery").set("isNewQuery", "")
def reset_settings():
channel("noaaquery").set("isNarration", True)
channel("noaaquery").set("isMap", True)
channel("noaaquery").set("isNewQuery", True)
channel("noaaquery").set("stationResultsReady", "")
reset_settings()
metadata:
collapsed: true
urth:
dashboard: {}
execution_count: null
- cell_type: markdown
source: |-
## Visualization
In this section of the notebook we will define the widgets and supporting functions for the construction of an interactive dashboard. See [Polymer Data Bindings](https://www.polymer-project.org/1.0/docs/devguide/data-binding.html) for more details.
metadata:
urth:
dashboard:
hidden: true
- cell_type: markdown
source: |-
### Narration Widget
@@ -563,11 +582,11 @@ cells:
source: |-
def process_query(fname,stations,statecode,cleanuppref):
global DATA_STATE_STATION_LIST
reset_settings()
if cleanuppref == "Yes":
image_cleanup(IMAGE_DIRECTORY)
reset_settings()
DATA_STATE_STATION_LIST = process_station_detail_for_state(fname,stations,statecode)
channel("noaaquery").set("isNewQuery", True)
channel("noaaquery").set("stationResultsReady", True)
return DATA_STATE_STATION_LIST
# We can examine stations per state data.
@@ -719,9 +738,7 @@ cells:
map_obj = folium.Map(location=[centerpoint_latitude, centerpoint_longitude],zoom_start=6)
for index, row in df.iterrows():
map_obj.simple_marker([row.Latitude, row.Longitude], popup=row.QueryTag)
map_doc = display_map(map_obj)
channel("noaaquery").set("stationmap", map_doc)
return map_doc
return display_map(map_obj)
# We can examine the generated HTML for the dynamic map
#render_map()
@@ -731,25 +748,45 @@ cells:
urth:
dashboard: {}
execution_count: null
- cell_type: markdown
source: |-
##### HACK: urth-core-watch seems to misbehave when combined with output elements. The workaround is to split the widget into two.
metadata:
urth:
dashboard:
hidden: true
- cell_type: code
source: |-
%%html
<template id="station_detail_combo_widget" is="urth-core-bind" channel="noaaquery">
<template is="dom-if" if="{{isNewQuery}}">
<paper-card style="width: 100%;" heading="{{stateAbbrev}} Weather Stations" elevation="1">
<p>These are the weather stations monitoring local conditions. Select a station to explore historical record temperatures.</p>
<urth-viz-table datarows="{{ stationQueryResult.data }}" selection="{{station}}" columns="{{ stationQueryResult.columns }}" rows-visible=20>
</urth-viz-table>
</paper-card>
<template id="station_detail_combo_func" is="urth-core-bind" channel="noaaquery">
<urth-core-watch value="{{stationResultsReady}}">
<urth-core-function id="renderFoliumMapFunc"
ref="render_map"
result="{{foliumMap}}" auto>
</urth-core-function>
<div>
<template is="dom-if" if="{{isNewMap}}">
<urth-raw-html html="{{stationmap}}"/>
</template>
</div>
</urth-core-watch>
</template>
metadata:
collapsed: false
urth:
dashboard:
hidden: true
execution_count: null
- cell_type: code
source: |-
%%html
<template id="station_detail_combo_widget" is="urth-core-bind" channel="noaaquery">
<paper-card style="width: 100%;" heading="{{stateAbbrev}} Weather Stations" elevation="1">
<p>These are the weather stations monitoring local conditions. Select a station to explore historical record temperatures.</p>
<urth-viz-table datarows="{{ stationQueryResult.data }}" selection="{{station}}" columns="{{ stationQueryResult.columns }}" rows-visible=20>
</urth-viz-table>
</paper-card>
<template is="dom-if" if="{{isNewQuery}}">
<template is="dom-if" if="{{isMap}}">
<div>
<urth-raw-html html="{{foliumMap}}"/>
</div>
</template>
</template>
</template>
metadata:
@@ -758,7 +795,7 @@ cells:
dashboard:
layout:
col: 0
height: 25
height: 39
row: 19
width: 12
execution_count: null
@@ -778,7 +815,7 @@ cells:
df_station_detail = fetch_station_data(station)
channel("noaaquery").set("yearsOfService", compute_years_of_station_data(df_station_detail))
DATA_STATION_DETAIL_RESULTS = df_station_detail
display(Javascript("stationRecordFreqFunc.invoke()"))
#display(Javascript("stationRecordFreqFunc.invoke()"))
return df_station_detail
metadata:
collapsed: true
@@ -809,8 +846,8 @@ cells:
dashboard:
layout:
col: 0
height: 18
row: 44
height: 22
row: 58
width: 12
execution_count: null
- cell_type: markdown
@@ -853,7 +890,6 @@ cells:
if os.path.isfile(fname):
os.remove(fname)
sns_plot.savefig(fname)
channel("noaaquery").set("recordQueryResults", df_new)
return sns_plot.fig
def compute_record_durations(df,rectype):
@@ -889,20 +925,11 @@ cells:
if os.path.isfile(fname):
os.remove(fname)
plt.savefig(fname)
channel("noaaquery").set("durationResults", df_durations)
return fig
metadata:
collapsed: false
urth:
dashboard:
hidden: true
execution_count: null
- cell_type: code
source: |-
def explore_record_temperature_frequency(rectype,recfreqfactor):
global DATA_FREQUENCY_RESULTS
channel("noaaquery").set("isAboveFreqFactor", True)
channel("noaaquery").set("frequencyResultsAvailable", "")
channel("noaaquery").set("numberRecordDays", 0)
if rectype == "High":
df_record_days = compute_tmax_record_quantity(DATA_STATION_DETAIL_RESULTS,recfreqfactor)
@@ -911,10 +938,9 @@ cells:
if not df_record_days.empty:
channel("noaaquery").set("numberRecordDays", len(df_record_days))
DATA_FREQUENCY_RESULTS = df_record_days
channel("noaaquery").set("frequencyResultsAvailable", True)
else:
channel("noaaquery").set("isAboveFreqFactor", "")
display(Javascript("stationRecordFreqFunc.invoke()"))
#display(Javascript("stationRecordFreqFunc.invoke()"))
return df_record_days
metadata:
collapsed: false
@@ -924,52 +950,65 @@ cells:
- cell_type: code
source: |-
%%html
<template id="station_synopsis_widget" is="urth-core-bind" channel="noaaquery">
<urth-core-function id="stationRecordFreqFunc"
ref="explore_record_temperature_frequency"
arg-rectype="[[recType]]"
arg-recfreqfactor="[[occurrenceFactor]]"
result="{{stationFreqRecordsResult}}" auto>
</urth-core-function>
<template id="station_synopsis_data_widget" is="urth-core-bind" channel="noaaquery">
<urth-core-watch value="{{station.0}}">
<urth-core-function id="stationRecordFreqFunc"
ref="explore_record_temperature_frequency"
arg-rectype="[[recType]]"
arg-recfreqfactor="[[occurrenceFactor]]"
result="{{stationFreqRecordsResult}}" auto>
</urth-core-function>
</urth-core-watch>
</template>
metadata:
collapsed: false
urth:
dashboard:
hidden: true
execution_count: null
- cell_type: code
source: |-
%%html
<template id="station_synopsis_chart_widget" is="urth-core-bind" channel="noaaquery">
<template is="dom-if" if="{{stationFreqRecordsResult}}">
<paper-card style="width: 100%;" heading="Temperature Record Analysis" elevation="1">
<p>This station has experienced {{numberRecordDays}} days of new {{recType}} records where a new record has been set more than {{occurrenceFactor}} times throughout the operation of the station.</p>
<urth-viz-table datarows="{{ stationFreqRecordsResult.data }}" selection="{{dayAtStation}}" columns="{{ stationFreqRecordsResult.columns }}" rows-visible=20>
</urth-viz-table>
<template is="dom-if" if="{{isAboveFreqFactor}}">
<urth-core-function id="stationRecordsFunc"
ref="plot_record_results"
arg-rectype="[[recType]]"
result="{{stationRecordsPlot}}" auto>
</urth-core-function>
<urth-core-function id="stationDurationsFunc"
ref="plot_duration_results"
arg-rectype="[[recType]]"
result="{{stationDurationsPlot}}" auto>
</urth-core-function>
<paper-card heading="Station {{station.0}} Records Per Day" elevation="0">
<p>The current {{recType}} temperature record for each day that has experienced more than {{occurrenceFactor}} new record events since the station has come online.</p>
<img src="{{stationRecordsPlot}}"/><br/>
</paper-card>
<paper-card heading="Duration of Station {{station.0}} Records Per Day" elevation="0">
<p>For each day that has experienced more than {{occurrenceFactor}} {{recType}} temperature records, some days have had records stand for a large portion of the life of the station.</p>
<img src="{{stationDurationsPlot}}"/>
</paper-card>
</template>
<template is="dom-if" if="{{!isAboveFreqFactor}}">
<p>This weather station has not experienced any days with greater than {{occurrenceFactor}} new {{recType}} records.</p>
</template>
</paper-card>
</template>
<template is="dom-if" if="{{isAboveFreqFactor}}">
<urth-core-function id="stationRecordsFunc"
ref="plot_record_results"
arg-rectype="[[recType]]"
result="{{stationRecordsPlot}}" auto>
</urth-core-function>
<urth-core-function id="stationDurationsFunc"
ref="plot_duration_results"
arg-rectype="[[recType]]"
result="{{stationDurationsPlot}}" auto>
</urth-core-function>
<paper-card heading="Station {{station.0}} Records Per Day" elevation="0">
<p>The current {{recType}} temperature record for each day that has experienced more than {{occurrenceFactor}} new record events since the station has come online.</p>
<img src="{{stationRecordsPlot}}"/><br/>
</paper-card>
<paper-card heading="Duration of Station {{station.0}} Records Per Day" elevation="0">
<p>For each day that has experienced more than {{occurrenceFactor}} {{recType}} temperature records, some days have had records stand for a large portion of the life of the station.</p>
<img src="{{stationDurationsPlot}}"/>
</paper-card>
</template>
<template is="dom-if" if="{{!isAboveFreqFactor}}">
<p>This weather station has not experienced any days with greater than {{occurrenceFactor}} new {{recType}} records.</p>
</template>
</template>
</template>
metadata:
collapsed: false
urth:
dashboard:
layout:
col: 0
height: 33
row: 62
height: 51
row: 80
width: 12
execution_count: null
- cell_type: markdown
@@ -993,7 +1032,7 @@ cells:
layout:
col: 0
height: 14
row: 115
row: 151
width: 12
metadata:
kernelspec:
Oops, something went wrong.

0 comments on commit 0060324

Please sign in to comment.