IPP-06-DEV-004: Implement the WeatherServiceManager class

[labbenchstudios]

Description

• Using instructions from the in-class lecture, in this exercise, you'll create the WeatherServiceManager class, which will be the primary entry point for your application into all the weather service data collection processes.
  • This class will be implemented within the WeatherServiceManager.py module created in the previous exercise.
• This module will need to implement the functionality listed below under Actions.

Review the README

• Please see README.md for further information on, and use of, this content.
• License for embedded documentation and source codes: IPP-DOC-LIC

Estimated effort may vary greatly

• The estimated level of effort for this exercise shown in the 'Estimate' section below is a very rough approximation. The actual level of effort may vary greatly depending on your development and test environment, experience with the requisite technologies, and many other factors.

Actions

Reminders on System Configuration:

• Make Sure Your System is Setup for Python and This Course

  • See IPP-CFG-01-001

• Make Sure PYTHONPATH is Set Correctly

  • Whether running Python tests within your IDE or from the command line, you must set the PYTHONPATH environment variable in every execution environment (e.g., every terminal you launch) when attempting to run any of your scripts and their tests or the IPP test app from the command line. The IPP source and test paths will be as follows:
    • {your IPP source code path}
    • {your IPP source code path}/tests
  • As a reminder, you can include the PYTHONPATH environment variable for your environment within the activation script for your virtual environment, which will set PYTHONPATH automatically whenever your virtual environment is launched

• See IPP-DEV-01-001 for details.

Step 1: Implement the WeatherServiceManager class within the WeatherServiceManager.py module

• This class will define and implement the primary "state machine" used to asynchronously retrieve data from the configured weather service (using the WeatherServiceConnection sub-class implementation).
  • In this module, we'll introduce the concept of "concurrency", which technically means to do something at the same time as something else (or at least gives us the perception that multiple things are happening at the same time).
  • Implement the import statement, class definition, and class constructor (__init__() method):

import itertools

from apscheduler.schedulers.background import BackgroundScheduler

from ipp.common.ConfigUtil import ConfigUtil
from ipp.exercises.labmodule05.LocationData import LocationData
from ipp.exercises.labmodule05.WeatherData import WeatherData
from ipp.exercises.labmodule06.NoaaWeatherServiceConnector import NoaaWeatherServiceConnector
from ipp.exercises.labmodule06.WeatherDataListener import WeatherDataListener
from ipp.exercises.labmodule06.WeatherServiceConnector import WeatherServiceConnector

class WeatherServiceManager():
    def __init__(self):
        self.scheduler = BackgroundScheduler()
        self.isRunning = False
        self.dataListener = None

        self._initProperties()

Step 2: Create the _initProperties() method

• This will parse the requisite properties from the configuration file IppConfig.props:

    def _initProperties(self):
        self.configUtil = ConfigUtil()

        # TODO: make this configurable - for now, NOAA service is fine
        self.weatherSvc = NoaaWeatherServiceConnector()
        self.clientSession = None
        self.isConnected = False

        self.pollStationIDs = \
            self.configUtil.getProperty( \
                WeatherServiceConnector.WEATHER_SVC_SECTION_NAME, "pollStationIDs")
        
        self.pollStationList = [stationID.strip() for stationID in self.pollStationIDs.split(',')]
        self.pollStationCycle = None

        if self.pollStationIDs:
            print(f"Polling weather station ID's: {self.pollStationIDs}")

            self.pollStationCycle = itertools.cycle(self.pollStationList)
        else:
            # default to Boston (KBOS)
            self.pollStationIDs = "KBOS"

            print(f"No weather station ID's defined in config file. Using default: {self.pollStationIDs}")

Step 3: Create the internal method used for configuring the scheduler task.

• This method uses the apscheduler module to configure and instance a new task within the scheduler system.
  • This will be used by the start method to start the task.

    def _scheduleAndStartWeatherServiceJob(self):
        pollRate = self.weatherSvc.getPollRate()
        
        # IMPORTANT NOTE: You may need to experiment with these settings to ensure
        # you properly handle delayed responses, network timeouts, and data processing
        # delays once a response is received
        self.scheduler.add_job( \
            func = self.processWeatherData, trigger = 'interval', \
            id = self.pollStationIDs, replace_existing = True, seconds = pollRate, \
	    max_instances = 1, coalesce = True, misfire_grace_time = None)
        
        self.scheduler.start()

Step 4: Create the start and stop methods

• The start method will establish a scheduler task using the apscheduler library and start the scheduled task.
• The stop method will simply close the apscheduler and end the scheduled task.

    def startManager(self):
        success = False

        if not self.isRunning:
            print("Creating weather service client and connecting to service.")

            if not self.weatherSvc.isClientConnected():
                self.weatherSvc.connectToService()

            self._scheduleAndStartWeatherServiceJob()

            self.isRunning = True

            print("Weather station manager is now up and running!")

            success = True
        else:
            print("Client is already connected to weather service!")
            success = True

        return success
    
    def stopManager(self):
        success = False

        if self.isRunning:
            print("Disconnecting from weather service.")
            
            # disconnect here
            if self.weatherSvc.isClientConnected():
                self.weatherSvc.disconnectFromService()
                
            try:
                self.scheduler.shutdown(wait = False)
                self.isRunning = False
                success = True
            except:
                print("Failed to shutdown scheduler. Probably not running.")
        else:
            print("No weather service connection created! Call startManager() first.")

        return success

Step 5: Implement the weather station location parsing/generation logic.

• This internal method accepts a stationID string and attempts to create location data for that location.
  • NOTE: This could be part of the configuration file, or can even be a supplementary service that is used to retrieve location data from another public online source based on the weather station ID in the configuration. For now, these location data elements are hard coded for convenience and testing purposes.

    def _getLocationData(self, stationID: str = None):
        if stationID == "KJFK":
            # NYC (JFK airport)
            locData = LocationData()
            locData.name = "JFK International Airport"
            locData.city = "New York"
            locData.region = "NY"
            locData.country = "USA"
            locData.latitude = 40.63972
            locData.longitude = 73.77889

            return locData
        
        elif stationID == "KORD":
            # ORD (O'Hare airport)
            locData = LocationData()
            locData.name = "O'Hare International Airport"
            locData.city = "Chicago"
            locData.region = "IL"
            locData.country = "USA"
            locData.latitude = 40.978611
            locData.longitude = 73.904724

            return locData
        elif stationID == "KBOS":
            # BOS (Logan aiport)
            locData = LocationData()
            locData.name = "Logan International Airport"
            locData.city = "Boston"
            locData.region = "MA"
            locData.country = "USA"
            locData.latitude = 40.35843
            locData.longitude = 73.05977

            return locData
        else:
            # Unknown - just use stationID and zero out lat / lon
            locData = LocationData()
            locData.name = stationID
            locData.city = stationID
            locData.region = stationID
            locData.country = stationID
            locData.latitude = 0.0
            locData.longitude = 0.0

            return locData

Step 6: Implement the main action method - processWeatherData().

• This will use the weather service connection to request and obtain the latest weather data from the service for the next location in the configured list of weather station locations.

    def processWeatherData(self):
        stationID = next(self.pollStationCycle)
        print(f"Processing station ID: {stationID}")

        locData = self._getLocationData(stationID = stationID)

        rawData = self.weatherSvc.requestCurrentWeatherData(stationID = stationID, locData = locData)
        jsonData = self.weatherSvc.getLatestWeatherDataAsJson()
        wData = self.weatherSvc.getLatestWeatherData()

        # TODO: do some processing

        print(f"Just retrieved weather data for station ID: {stationID}\n{jsonData}\n\n")
        #print(f"Just retrieved weather data for station ID: {stationID}")

        if self.dataListener:
            self.dataListener.handleIncomingWeatherData(data = wData)

        return jsonData

Step 7: Implement the public-facing setter and helper methods used to set and retrieve properties of the weather service.

• For now, these properties are limited to setting the listener (which will receive updates from the weather service manager) and a boolean check to see if there's an actively connected session.

    def setListener(self, listener: WeatherDataListener = None):
        if listener:
            self.dataListener = listener
            
    def isClientConnected(self):
        return self.isConnected

Create Tests

• Create a new unittest module and class within this lab module's tests directory (IPP_HOME/tests).
  • In the ./tests/labmodule06 path, create a new module named test_WeatherServiceManager.py
  • Use the following template as the initial content for the module.

import datetime
import logging
import time
import unittest

from ipp.exercises.labmodule05.LocationData import LocationData
from ipp.exercises.labmodule05.TimeAndDateUtil import TimeAndDateUtil
from ipp.exercises.labmodule05.WeatherData import WeatherData

from ipp.exercises.labmodule06.WeatherServiceManager import WeatherServiceManager

class WeatherServiceManagerTest(unittest.TestCase):

	@classmethod
	def setUpClass(self):
		logging.basicConfig(format = '%(asctime)s:%(module)s:%(levelname)s:%(message)s', level = logging.DEBUG)
		logging.info("Testing WeatherServiceManager class...")
		
	def setUp(self):
		self.weatherSvcMgr = WeatherServiceManager()

	def tearDown(self):
		pass
	
	def testWeatherServiceManagerExecution(self):
		self.weatherSvcMgr.startManager()

		# let it run for ~2 min's
		time.sleep(120)

		self.weatherSvcMgr.stopManager()

Estimate

• Medium

Run Tests

• Testing using the unittest framework

  • In VS Code:
    • In the menu navigation on the left side, click on the beaker (mouseover the icons - hovering over the beaker will popup the name Testing)
    • All loaded tests will appear - click the Refresh Testing button to ensure they're all loaded.
      • NOTE: Tests must be in the top level tests path (within IPP_HOME) and each unittest module must begin with test_ (e.g., test_MyModule.py)
    • Using the IDE's controls, execute the desired test(s). For this lab module, run the following:
      • testWeatherServiceManagerExecution

• Testing the module directly

  • From within your IDE

    • Right click on your newly created module test_WeatherServiceManager.py and click run tests
    • You should see output similar to that discussed in class

  • From the command line

    • Open a terminal and cd to your IPP_HOME path
    • Start your virtual environment (if not already running)
    • Be sure your PYTHONPATH is set correctly
    • Run the module
      • python -m unittest ./tests/labmodule06/test_WeatherServiceManager.py
    • You should see output similar to that discussed in class

Sample output (yours may differ slightly)