Update weather agent to support both Dark Sky and OpenWeather

- Updates the README clarifying support of OpenWeather, Dark Sky, and Wunderground. - Merges pull request huginn#2848's support for OpenWeather in, without removing support for Dark Sky as the original pull request did. (Thanks Ryan Waldron for this!) Choosing which weather provider to use is done via the `service` option which was previously used in the WeatherAgent to support Wunderground and Dark Sky simultaneously, until Wunderground ended support for their API, at which time it was removed. Since the WeatherAgent didn't require a `service` option for a while, this diff takes the approach of defaulting to "forecastio" if no `service` option is provided in order to avoid breaking configurations prior to this change. However, the default `service` value for a new configuration will be "openweather." Additionally, "forecastio" can be explicitly specified, again to ensure compatibility with even older configurations. A note on testing - I am not a Ruby developer, so I am not quite sure how best to add tests around the branching behavior that occurs in the `model` method. Even worse, since I do not have a Dark Sky API key, I cannot manually test it. I did manually test OpenWeather. It would also be ideal to have tests which verify that the translation of data occurs without error using fixtures from each provider. Tests of this behavior did not already exist in `weather_agent_spec.rb`, however OpenWeather is tested indirectly by `agent_spec.rb` among other specs. You can find these tests easily by searching for the OpenWeather data fixture's file name ("openweather.json"). If someone were interested in creating tests of this nature in `weather_agent_spec.rb`, you can also find a fixture file for Dark Sky at `/spec/data_fixtures/weather.json` in any commit prior to the commits from huginn#2848 (starting with a61924f.)
nighthawk · May 19, 2022 · cca884c · cca884c
1 parent d0a405c
commit cca884c
Show file tree

Hide file tree

Showing 10 changed files with 236 additions and 92 deletions.
diff --git a/README.md b/README.md
@@ -142,9 +142,9 @@ See [private development instructions](https://github.com/huginn/huginn/wiki/Pri
 
 #### Enable the WeatherAgent
 
-In order to use the WeatherAgent you need an [API key with OpenWeather](https://home.openweathermap.org/api_keys). Signup for one and then change the value of `api_key: your-key` in your seeded WeatherAgent and setting the service key to `openweather`.
+In order to use the WeatherAgent you need an [API key with OpenWeather](https://home.openweathermap.org/api_keys). Signup for one and then change the value of `api_key: your-key` in your seeded WeatherAgent and set the `service` key to `openweather`.
 
-Note, neither DarkSky nor Wunderground offers free API keys any more.
+Note: While Dark Sky is still supported in the Weather Agent, they no longer offer API keys. Support for Wunderground has been removed.
 
 #### Disable SSL
 

diff --git a/app/models/agents/weather_agent.rb b/app/models/agents/weather_agent.rb
@@ -5,19 +5,23 @@ module Agents
   class WeatherAgent < Agent
     cannot_receive_events!
 
+    gem_dependency_check { defined?(ForecastIO) }
+
     description <<-MD
       The Weather Agent creates an event for the day's weather at a given `location`.
 
-      You can specify for which day you would like the weather forecast using the `which_day` option, where the number 1 represents today, 2 represents tomorrow, and so on. The default is 1 (today). If you schedule this to run at night, you probably want 2 (tomorrow). Weather forecast inforation is only returned for at most one week at a time.
+      #{'## Include `forecast_io` in your Gemfile to use this Agent!' if dependencies_missing?}
 
-      The weather forecast information is provided by [OpenWeather](https://home.openweathermap.org).
+      You can specify for which day you would like the weather forecast using the `which_day` option, where the number 1 represents today, 2 represents tomorrow, and so on. The default is 1 (today). If you schedule this to run at night, you probably want 2 (tomorrow). Weather forecast inforation is only returned for at most one week at a time.
 
       The `location` must be a comma-separated string of map co-ordinates (longitude, latitude). For example, San Francisco would be `37.7771,-122.4196`.
 
-      You must set up an [API key for OpenWeather](https://home.openweathermap.org/api_keys) in order to use this Agent.
-
       Set `expected_update_period_in_days` to the maximum amount of time that you'd expect to pass between Events being created by this Agent.
 
+      The weather forecast information is provided by [OpenWeather](https://home.openweathermap.org) or Dark Sky - to choose which set `sevice` to either `openweather` or `forecastio` (Dark Sky). If `service` is not specified, it will default to `forecastio` to avoid breaking older configurations.
+
+      You must set up an [API key for OpenWeather](https://home.openweathermap.org/api_keys) or have an existing [API key from Dark Sky](https://darksky.net/dev/) in order to use this Agent.
+
     MD
 
     event_description <<-MD
@@ -57,6 +61,7 @@ def key_setup?
 
     def default_options
       {
+        'service' => 'openweather',
         'api_key' => 'your-key',
         'location' => '37.779329,-122.41915',
         'which_day' => '1',
@@ -71,14 +76,26 @@ def check
       end
     end
 
-    private
+    def dark_sky?
+      # Default to Dark Sky if the service is not defined, or set to the old forecastio value for backwards
+      # compatibility with old configurations.
+      interpolated["service"].nil? || interpolated["service"].downcase == "forecastio"
+    end
+
+    def wunderground?
+      interpolated["service"].present? && interpolated["service"].downcase == "wunderground"
+    end
+
+    def openweather?
+      interpolated["service"].present? && interpolated["service"].downcase == "openweather"
+    end
 
     def which_day
       (interpolated["which_day"].presence || 1).to_i
     end
 
     def location
-      interpolated["location"].presence || interpolated["zipcode"]
+      interpolated["location"].presence
     end
 
     def coordinates
@@ -89,13 +106,7 @@ def language
       interpolated["language"].presence || "en"
     end
 
-    def darksky?
-      interpolated["service"].presence && interpolated["service"].presence.downcase == "darksky"
-    end
-
-    def wunderground? 
-      interpolated["service"].presence && interpolated["service"].presence.downcase == "wunderground"
-    end
+    private
 
     def openweather_icon(code)
       "http://openweathermap.org/img/wn/#{code}@2x.png"
@@ -131,14 +142,19 @@ def validate_location
     end
 
     def validate_options
-      errors.add(:base, "The DarkSky API has been disabled since Aug 1st, 2020; please switch to OpenWeather.") if darksky?
-      errors.add(:base, "The Weather Underground API has been disabled since Jan 1st 2018; please switch to OpenWeather.") if wunderground?
       validate_location
       errors.add(:base, "api_key is required") unless interpolated['api_key'].present?
       errors.add(:base, "which_day selection is required") unless which_day.present?
+      errors.add(:base, "service must be one of: forecastio, openweather") unless
+        interpolated['service'].nil? or ['forecastio', 'openweather'].include? interpolated['service']
+
+      errors.add(
+        :base,
+        "The Weather Underground API has been disabled since Jan 1st 2018; please switch to OpenWeather."
+      ) if wunderground?
     end
 
-    def open_weather
+    def openweather
       if key_setup?
         onecall_endpoint = "http://api.openweathermap.org/data/2.5/onecall"
         lat, lng = coordinates
@@ -148,8 +164,93 @@ def open_weather
       end
     end
 
+    def dark_sky
+      if key_setup?
+        ForecastIO.api_key = interpolated['api_key']
+        lat, lng = coordinates
+        ForecastIO.forecast(lat, lng, params: {lang: language.downcase})['daily']['data']
+      end
+    end
+
     def model(which_day)
-      value = open_weather[which_day - 1]
+      if dark_sky?
+        # AFAIK, there is no warning-level log messages. In any case, I'd like to log this from validate_options but
+        # since the Agent doesn't exist yet, the Log record can't be created (no parent_id)
+        log "NOTICE: The DarkSky API will be disabled at the end of 2021; please switch to OpenWeather." if dark_sky?
+        dark_sky_model(which_day)
+      elsif openweather?
+        openweather_model(which_day)
+      end
+    end
+
+    def dark_sky_model(which_day)
+      value = dark_sky[which_day - 1]
+      if value
+        timestamp = Time.at(value.time)
+        day = {
+          'date' => {
+            'epoch' => value.time.to_s,
+            'pretty' => timestamp.strftime("%l:%M %p %Z on %B %d, %Y"),
+            'day' => timestamp.day,
+            'month' => timestamp.month,
+            'year' => timestamp.year,
+            'yday' => timestamp.yday,
+            'hour' => timestamp.hour,
+            'min' => timestamp.strftime("%M"),
+            'sec' => timestamp.sec,
+            'isdst' => timestamp.isdst ? 1 : 0 ,
+            'monthname' => timestamp.strftime("%B"),
+            'monthname_short' => timestamp.strftime("%b"),
+            'weekday_short' => timestamp.strftime("%a"),
+            'weekday' => timestamp.strftime("%A"),
+            'ampm' => timestamp.strftime("%p"),
+            'tz_short' => timestamp.zone
+          },
+          'period' => which_day.to_i,
+          'high' => {
+            'fahrenheit' => value.temperatureMax.round().to_s,
+            'epoch' => value.temperatureMaxTime.to_s,
+            'fahrenheit_apparent' => value.apparentTemperatureMax.round().to_s,
+            'epoch_apparent' => value.apparentTemperatureMaxTime.to_s,
+            'celsius' => ((5*(Float(value.temperatureMax) - 32))/9).round().to_s
+          },
+          'low' => {
+            'fahrenheit' => value.temperatureMin.round().to_s,
+            'epoch' => value.temperatureMinTime.to_s,
+            'fahrenheit_apparent' => value.apparentTemperatureMin.round().to_s,
+            'epoch_apparent' => value.apparentTemperatureMinTime.to_s,
+            'celsius' => ((5*(Float(value.temperatureMin) - 32))/9).round().to_s
+          },
+          'conditions' => value.summary,
+          'icon' => value.icon,
+          'avehumidity' => (value.humidity * 100).to_i,
+          'sunriseTime' => value.sunriseTime.to_s,
+          'sunsetTime' => value.sunsetTime.to_s,
+          'moonPhase' => value.moonPhase.to_s,
+          'precip' => {
+            'intensity' => value.precipIntensity.to_s,
+            'intensity_max' => value.precipIntensityMax.to_s,
+            'intensity_max_epoch' => value.precipIntensityMaxTime.to_s,
+            'probability' => value.precipProbability.to_s,
+            'type' => value.precipType
+          },
+          'dewPoint' => value.dewPoint.to_s,
+          'avewind' => {
+            'mph' => value.windSpeed.round().to_s,
+            'kph' =>  (Float(value.windSpeed) * 1.609344).round().to_s,
+            'degrees' => value.windBearing.to_s
+          },
+          'visibility' => value.visibility.to_s,
+          'cloudCover' => value.cloudCover.to_s,
+          'pressure' => value.pressure.to_s,
+          'ozone' => value.ozone.to_s
+        }
+        return day
+      end
+    end
+
+    def openweather_model(which_day)
+      value = openweather[which_day - 1]
       if value
         timestamp = Time.at(value.dt)
         day = {
@@ -174,30 +275,30 @@ def model(which_day)
           'period' => which_day.to_i,
           'high' => {
             'fahrenheit' => value.temp.max.round().to_s,
-            #'epoch' => value.temperatureMaxTime.to_s,
+            'epoch' => nil,
             'fahrenheit_apparent' => value.feels_like.day.round().to_s,
-            #'epoch_apparent' => value.apparentTemperatureMaxTime.to_s,
+            'epoch_apparent' => nil,
             'celsius' => ((5*(Float(value.temp.max) - 32))/9).round().to_s
           },
           'low' => {
             'fahrenheit' => value.temp.min.round().to_s,
-            #'epoch' => value.temperatureMinTime.to_s,
+            'epoch' => nil,
             'fahrenheit_apparent' => value.feels_like.night.round().to_s,
-            #'epoch_apparent' => value.apparentTemperatureMinTime.to_s,
+            'epoch_apparent' => nil,
             'celsius' => ((5*(Float(value.temp.min) - 32))/9).round().to_s
           },
           'conditions' => value.weather.first.description,
           'icon' => openweather_icon(value.weather.first.icon),
           'avehumidity' => (value.humidity * 100).to_i,
           'sunriseTime' => value.sunrise.to_s,
           'sunsetTime' => value.sunset.to_s,
-          #'moonPhase' => value.moonPhase.to_s,
+          'moonPhase' => nil,
           'precip' => {
-            'intensity' => value.rain.to_s,
-            #'intensity_max' => value.precipIntensityMax.to_s,
-            #'intensity_max_epoch' => value.precipIntensityMaxTime.to_s,
-            #'probability' => value.precipProbability.to_s,
-            'type' => figure_rain_or_snow(value.rain, value.snow),
+            'intensity' => value.rain.to_s.presence || '0',
+            'intensity_max' => nil,
+            'intensity_max_epoch' => nil,
+            'probability' => nil,
+            'type' => figure_rain_or_snow(value.rain, value.snow).presence,
           },
           'dewPoint' => value.dew_point.to_s,
           'avewind' => {
@@ -208,7 +309,7 @@ def model(which_day)
           'visibility' => value.visibility.to_s,
           'cloudCover' => value.clouds.to_s,
           'pressure' => value.pressure.to_s,
-          #'ozone' => value.ozone.to_s
+          'ozone' => nil,
         }
         return day
       end

diff --git a/data/default_scenario.json b/data/default_scenario.json
@@ -85,8 +85,9 @@
       "disabled": false,
       "guid": "bdae6dfdf9d01a123ddd513e695fd466",
       "options": {
-        "location": "42.3601,-71.0589",
-        "api_key": "put-your-key-here"
+        "service": "openweather",
+        "api_key": "put-your-key-here",
+        "location": "42.3601,-71.0589"
       },
       "schedule": "10pm",
       "keep_events_for": 0

diff --git a/spec/data_fixtures/weather.json → spec/data_fixtures/openweather.json b/spec/data_fixtures/weather.json → spec/data_fixtures/openweather.json
diff --git a/spec/fixtures/agents.yml b/spec/fixtures/agents.yml
@@ -58,7 +58,7 @@ bob_weather_agent:
   name: "SF Weather"
   guid: <%= SecureRandom.hex %>
   keep_events_for: <%= 45.days %>
-  options: <%= { :location => "37.779329,-122.41915", :api_key => 'test' }.to_json.inspect %>
+  options: <%= { :service => 'openweather', :location => "37.779329,-122.41915", :api_key => 'test' }.to_json.inspect %>
 
 bob_formatting_agent:
   type: Agents::EventFormattingAgent
@@ -75,7 +75,7 @@ jane_weather_agent:
   name: "SF Weather"
   guid: <%= SecureRandom.hex %>
   keep_events_for: <%= 30.days %>
-  options: <%= { :location => "37.779329,-122.41915", :api_key => 'test' }.to_json.inspect %>
+  options: <%= { :service => 'openweather', :location => "37.779329,-122.41915", :api_key => 'test' }.to_json.inspect %>
 
 jane_rain_notifier_agent:
   type: Agents::TriggerAgent

diff --git a/spec/fixtures/test_default_scenario.json b/spec/fixtures/test_default_scenario.json
@@ -45,8 +45,9 @@
       "disabled": false,
       "guid": "bdae6dfdf9d01a123ddd513e695fd466",
       "options": {
-        "location": "42.3601,-71.0589",
-        "api_key": "put-your-key-here"
+        "service": "openweather",
+        "api_key": "put-your-key-here",
+        "location": "42.3601,-71.0589"
       },
       "schedule": "10pm",
       "keep_events_for": 0

diff --git a/spec/models/agent_spec.rb b/spec/models/agent_spec.rb
@@ -283,7 +283,7 @@ def receive(events)
 
     describe ".receive!" do
       before do
-        stub_request(:any, /openweather/).to_return(:body => File.read(Rails.root.join("spec/data_fixtures/weather.json")), :status => 200)
+        stub_request(:any, /openweather/).to_return(:body => File.read(Rails.root.join("spec/data_fixtures/openweather.json")), :status => 200)
         # stub.any_instance_of(Agents::WeatherAgent).is_tomorrow?(anything) { true }
       end
 
@@ -926,7 +926,7 @@ def @agent.receive_webhook(params)
 
   describe ".drop_pending_events" do
     before do
-      stub_request(:any, /openweather/).to_return(body: File.read(Rails.root.join("spec/data_fixtures/weather.json")), status: 200)
+      stub_request(:any, /openweather/).to_return(body: File.read(Rails.root.join("spec/data_fixtures/openweather.json")), status: 200)
     end
 
     it "should drop pending events while the agent was disabled when set to true" do

diff --git a/spec/models/agents/email_agent_spec.rb b/spec/models/agents/email_agent_spec.rb
@@ -56,7 +56,7 @@ def get_message_part(mail, content_type)
     end
 
     it "can receive complex events and send them on" do
-      stub_request(:any, /openweather/).to_return(:body => File.read(Rails.root.join("spec/data_fixtures/weather.json")), :status => 200)
+      stub_request(:any, /openweather/).to_return(:body => File.read(Rails.root.join("spec/data_fixtures/openweather.json")), :status => 200)
       # stub.any_instance_of(Agents::WeatherAgent).is_tomorrow?(anything) { true }
       @checker.sources << agents(:bob_weather_agent)
 

diff --git a/spec/models/agents/email_digest_agent_spec.rb b/spec/models/agents/email_digest_agent_spec.rb
@@ -84,7 +84,7 @@ def get_message_part(mail, content_type)
     end
 
     it "can receive complex events and send them on" do
-      stub_request(:any, /openweather/).to_return(:body => File.read(Rails.root.join("spec/data_fixtures/weather.json")), :status => 200)
+      stub_request(:any, /openweather/).to_return(:body => File.read(Rails.root.join("spec/data_fixtures/openweather.json")), :status => 200)
       # stub.any_instance_of(Agents::WeatherAgent).is_tomorrow?(anything) { true }
       @checker.sources << agents(:bob_weather_agent)