Weather Analytics and Extrema Monitoring
The demo can be found here.
At a Glance
- .NET application to query the DarkSky API on a daily cadence, record weather data of 100 U.S. cities, and calculate and persist lookback aggregations in AWS hosted SQL Server. Containerized in Docker for deployment.
- React front-end for city rankings by weather condition, geographic comparison, and drilling into outliers for further insights across lookback period.
What Is It For
- I've had silly questions about weather for a while - what cities are rainiest, what parts of the U.S. experience the most extreme weather, are there any correlations between weather conditions within areas of the U.S. (e.g. "Is there a relationship between most humid cities and rainiest cities?")
- The application provides a by-city rankings of various weather conditions (temperature, precipitation, humidity, wind speed, cloud cover), a bar chart for visualizing differences between cities, a trend chart to visualize weather history across time for some set of cities, and a map to show how weather phenomena plays out by geography.
Technical Stack: .NET Core / C#, React, SQL Server
- The front-end is built using React with heavy reliance on a personal component library.
- For state management, I was deciding between Redux, Apollo, and Context. I opted to just use Context, given there wasn't much state manipulation on the front-end.
- For the back-end, I'm using a .NET Core Web API deployed to Heroku using Docker. I opted for C# just to build familiarity, but there are no major constraints that would make this language better than others.
- For the database, I'm using SQL server. There's a good argument for using NoSQL, given the data isn't heavily relational (aside from
cityIdbinding many of the models together). That would allow for greater scalability and performance in case I wanted to add more cities or extend the lookback timeframe. In the end, I went with SQL Server just for ease-of-use.
Computation and Performance
- I didn't want to have to do the math on the fly to calculate average weather conditions for certain lookback periods (e.g. average precipitation for every city for the past 7, 30, 90, 180, and 365 days.
- Instead, there's a scheduled job that runs on a daily cadence to populate weather data for each city using DarkSky, and then aggregates that data across the aforementioned lookback periods across each city and weather condition. This way, no computation is performed when a user is interacting with the application, and it's solely passing information from the database to the frontend.
- I'm a fan of Redux, but I wanted to get comfortable with Context. Context was the right call for structuring how components would access the data, but I think it missed the mark for it's use in storing UI state like
selectedWeatherConditionName. It was annoying to update any values, but access was easy and straight-forward with React hooks.
What could be added to it
- Correlation coefficient matrix may be interesting to see whether weather conditions in an area are correlated - e.g. are cities in the midwest likely to have humidity trends in line with raininess trends?
- Alerting for particularly anomalous weather behavior.
What are the expectations of it
I fully intend for this to just be a hobby project that I check up on now and then. It may eventually evolve to be more useful for picking a place to vacation or live based on the weather, but for now will just be a simple dashboard.
How to run it
For the front-end:
npm run start
For the back-end:
Note that I use environment variables in Rider for the connection string to the database - that should be swapped out with information to your own database.