module-content/docs/source/api_usage/3.6-Controlling-Simulation-API.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "2bc6eaf3",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "a8f99310",
   "metadata": {},
   "source": [
    "# Publishing and Subscribing with the Simulation API"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f905069f",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a8dbb390",
   "metadata": {},
   "source": [
    "## Introduction to the Simulation API\n",
    "\n",
    "\n",
    "The Simulation API is used for all actions related to a power system simulation. It is used to start, pause, restart, and stop a simulation from the command line or inside an application. It is all used to subscribe to measurements of equipment (such as line flows, loads, and DG setpoints) and the statuses of switches, capacitors, transformer taps, etc. It is also used to publish equipment control and other simulation input commands.\n",
    "\n",
    "In the Application Components diagram (explained in detail with sample code in [GridAPPS-D Application Structure](../overview/2.4-GridAPPS-D-Application-Structure.ipynb)), the PowerGrid Models API is used for controlling the simulation, subscribing to measurement data, and controlling equipment.\n",
    "\n",
    "This section covers only the portion of the API used for subscribing to measurements and publishing equipment control commands. Usage of the API for starting, stopping, and pausing simulations is covered in [Creating and Running Simulations with Simulation API](../api_usage/3.5-Creating-Running-Simulation-API.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb475001",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "592d49f3",
   "metadata": {},
   "source": [
    "## Processing Measurements & App Core Algorithm\n",
    "\n",
    "The central portion of a GridAPPS-D application is the measurement processing and core algorithm section. This section is built as either a class or function definition with prescribed arguments. Each has its advantages and disadvantages:\n",
    "\n",
    "* The function-based approach is simpler and easier to implement. However, any parameters obtained from other APIs or methods to be used inside the function currently need to be defined as global variables. \n",
    "\n",
    "* The class-based approach is more complex, but also more powerful. It provides greater flexibility in creating additional methods, arguments, etc."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6794a03d",
   "metadata": {},
   "source": [
    "### App Core Information Flow\n",
    "\n",
    "This portion of the application does not communicate directly with the GridAPPS-D platform. \n",
    "\n",
    "Instead, the next part of the GridAPPS-D application ([Subscribing to Simulation Output](#Subscribing-to-Simulation-Output)) delivers the simulated SCADA measurement data to the core algorithm function / class definition. The core algorithm processes the data to extract the desired measurements and run its optimization / control agorithm."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e5238dff",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0adf8abf",
   "metadata": {},
   "source": [
    "### Structure of Simulation Output Message\n",
    "\n",
    "The first part of the application core is parsing simulated SCADA and measurement data that is delivered to the application.\n",
    "\n",
    "The general format of the messages received by the Simulation API is a python dictionary with the following key-value pairs:\n",
    "\n",
    "```\n",
    "{         \n",
    "    \"simulation_id\" :           string,\n",
    "    \"message\" : {\n",
    "        \"timestamp\" :           epoch time number,\n",
    "        \"measurements\" : {\n",
    "            \"meas mrid 1\":{\n",
    "                                \"PNV measurement_mrid\": \"meas mrid 1\"\n",
    "                                \"magnitude\": number,\n",
    "                                \"angle\": number },\n",
    "            \"meas mrid 2\":{\n",
    "                                \"VA measurement_mrid\": \"meas mrid 2\"\n",
    "                                \"magnitude\": number,\n",
    "                                \"angle\": number },\n",
    "            \"meas mrid 3\":{\n",
    "                                \"Pos measurement_mrid\": \"meas mrid 3\"\n",
    "                                \"value\": number },\n",
    "                           .     \n",
    "                           .\n",
    "                           .\n",
    "            \"meas mrid n\":{\n",
    "                                \"measurement_mrid\": \"meas mrid n\"\n",
    "                                \"magnitude\": number,\n",
    "                                \"angle\": number },\n",
    "    }\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90fc0efd",
   "metadata": {},
   "source": [
    "### Format of Measurement Values\n",
    "\n",
    "In the message above, note the difference in the key-value pair structure of different types of measurements:\n",
    "\n",
    "__PNV Voltage Measurements__\n",
    "\n",
    "These are specified as `magnitude` and `angle` key-value pairs. \n",
    "\n",
    "* Magnitude is the RMS phase-to-neutral voltage. \n",
    "* Angle is phase angle of the voltage at the particular node.\n",
    "\n",
    "\n",
    "__VA Volt-Ampere Apparent Power Measurements__\n",
    "\n",
    "These are specified as `magnitude` and `angle` key-value pairs. \n",
    "* Magnitude is the apparent power. \n",
    "* Angle is complex power triangle angle (i.e. _acos(power factor)_)\n",
    "\n",
    "\n",
    "__Pos Position Measurements__\n",
    "\n",
    "These are specified as a `value` key-value pair. \n",
    "* Value is the position of the particular measurement\n",
    "    * For switch objects: value = 1 means \"closed\", value = 0 means \"open\"\n",
    "    * For capacitor objects, values are reversed: value = 1 means \"on\", value = 0 means \"off\"\n",
    "    * For regulator objects, value is the tap position, ranging from -16 to 16"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a4c916e",
   "metadata": {},
   "source": [
    "### Role of Measurement mRIDs\n",
    "\n",
    "The simulation output message shown above typically contains the measurement mRIDs for all available sensors for all equipment in the power system model. The application needs to filter the simulation output message to just the set of measurements relevant to the particular application (e.g. switch positions for a FLISR app or regulator taps for a VVO app).\n",
    "\n",
    "The equipment and measurement mRIDs are obtained in the first two sections of the application. See [Query for Power System Model](../overview/2.4-GridAPPS-D-Application-Structure.ipynb#Querying-for-the-Power-System-Model) and [Query for Measurement mRIDs](../overview/2.4-GridAPPS-D-Application-Structure.ipynb#Querying-for-Measurement-mRIDs) for examples of how these code sections fit in a sample app. \n",
    "\n",
    "API syntax details for the query messages to PowerGrid Models API to obtain equipment info and measurement mRIDs are given in [Query for Object Dictionary](../api_usage/3.3-Using-the-PowerGrid-Models-API.ipynb#Query-for-Object-Dictionary) and [Query for Measurements](../api_usage/3.3-Using-the-PowerGrid-Models-API.ipynb#Querying-for-Object-Measurements).\n",
    "\n",
    "These mRIDs will be needed to parse the simulation output message and filter it to just the desired set of measurements.\n",
    "\n",
    "For the example below, we will be interested in only the measurement associated with switches, so we will use the PowerGrid Models API to query for the set of measurements associated with the CIM Class `LoadBreakSwitch`. We then will filter those values to just the mRIDs associated with each type of measurement."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b66e3b51",
   "metadata": {},
   "outputs": [],
   "source": [
    "from gridappsd import topics as t\n",
    "\n",
    "# Create query message to obtain measurement mRIDs for all switches\n",
    "message = {\n",
    "    \"modelId\": model_mrid,\n",
    "    \"requestType\": \"QUERY_OBJECT_MEASUREMENTS\",\n",
    "    \"resultFormat\": \"JSON\",\n",
    "    \"objectType\": \"LoadBreakSwitch\"\n",
    "}\n",
    "\n",
    "# Pass query message to PowerGrid Models API\n",
    "response_obj = gapps.get_response(t.REQUEST_POWERGRID_DATA, message)\n",
    "measurements_obj = response_obj[\"data\"]\n",
    "\n",
    "# Switch position measurements (Pos)\n",
    "Pos_obj = [k for k in measurements_obj if k['type'] == 'Pos']\n",
    "\n",
    "# Switch phase-neutral-voltage measurements (PNV)\n",
    "PNV_obj = [k for k in measurements_obj if k['type'] == 'PNV']\n",
    "\n",
    "# Switch volt-ampere apparent power measurements (VA)\n",
    "VA_obj = [k for k in measurements_obj if k['type'] == 'VA']\n",
    "\n",
    "# Switch current measurements (A)\n",
    "A_obj = [k for k in measurements_obj if k['type'] == 'A']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "116de0e0",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4dd6d4b3",
   "metadata": {},
   "source": [
    "### App Core as a Function Definition\n",
    "\n",
    "The first approach used to build the application core is to define a function with the correct set of arguments that is then passed to the `.subscribe()` method associated with the `GridAPPPSD()` object.\n",
    "\n",
    "The function does not require a specific name, and is somewhat easier to define and use. However, the arguments of the function need to be named correctly for the GridAPPSD-Python library to process the simulation output correctly.\n",
    "\n",
    "The format for the function definition is \n",
    "\n",
    "```\n",
    "def mySubscribeFunction(header, message):\n",
    "    # do something when receive a message\n",
    "    # parse to get measurments\n",
    "    # do some calculations\n",
    "    # publish some equipment commands\n",
    "    # display some results\n",
    "```\n",
    "\n",
    "That function handle is then passed as an argument to the `.subscribe(topic, function_handle)` method when subscribing to the simulation in the next section."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "176c6e40",
   "metadata": {},
   "source": [
    "Note that the subscription function definition does not allow any additional parameters to be passed. The only allowed arguments are `header` and `message`. \n",
    "\n",
    "__Any other parameters, such as measurement mRIDs will need to be defined as global variables.__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b5ec4ef7",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define global python dictionary of position measurements\n",
    "global Pos_obj \n",
    "Pos_obj = [k for k in measurements_obj if k['type'] == 'Pos']\n",
    "\n",
    "# Define global python dictionary of phase-neutral-voltage measurements (PNV)\n",
    "global PNV_obj \n",
    "PNV_obj = [k for k in measurements_obj if k['type'] == 'PNV']\n",
    "\n",
    "# Define global python dictionary of volt-ampere apparent power measurements (VA)\n",
    "VA_obj = [k for k in measurements_obj if k['type'] == 'VA']\n",
    "\n",
    "# Current measurements (A)\n",
    "A_obj = [k for k in measurements_obj if k['type'] == 'A']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e862ba49",
   "metadata": {},
   "source": [
    "Below is the sample code for the core section of a basic application that tracks the number of open switches and the number of switches that are outaged."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d4f16111",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Only allowed arguments are `header` and `message`\n",
    "# message is simulation output message in format above\n",
    "def DemoAppCoreFunction(header, message):\n",
    "    \n",
    "    # Extract time and measurement values from message          \n",
    "    timestamp = message[\"message\"][\"timestamp\"]\n",
    "    meas_value = message[\"message\"][\"measurements\"]\n",
    "    \n",
    "    # Obtain list of all mRIDs from message\n",
    "    meas_mrid = list(meas_value.keys())\n",
    "    \n",
    "    # Example 1: Count the number of open switches\n",
    "    open_switches = []\n",
    "    for index in Pos_obj:\n",
    "        if index[\"measid\"] in meas_value: \n",
    "            mrid = index[\"measid\"]\n",
    "            power = meas_value[mrid]\n",
    "            if power[\"value\"] == 0: # Filter to measurements with value of zero\n",
    "                open_switches.append(index[\"eqname\"])\n",
    "    \n",
    "    # Print message to command line\n",
    "    print(\"............\")\n",
    "    print(\"Number of open switches at time\", timestamp, ' is ', len(set(open_switches)))\n",
    "    \n",
    "    \n",
    "    # Example 2: Count the number of outaged switches (voltage = 0)\n",
    "    dead_switches = []\n",
    "    for index in PNV_obj:\n",
    "        if index[\"measid\"] in meas_value:\n",
    "            mrid = index[\"measid\"]\n",
    "            voltage = meas_value[mrid]\n",
    "            if voltage[\"magnitude\"] == 0.0:\n",
    "                dead_switches.append(index[\"eqname\"])\n",
    "    \n",
    "    # Print message to command line\n",
    "    print(\"............\")\n",
    "    print(\"Number of outaged switches at time\", timestamp, ' is ', len(set(dead_switches)))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f4279da1",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8bb507f4",
   "metadata": {},
   "source": [
    "### App Core as a Class Definition\n",
    "\n",
    "The second approach used to build the app core and process measurements is to define a class containing two methods named `__init__` and `on_message`.\n",
    "\n",
    "These methods specify 1) how your app would initialize variables and attributes at the start of the simulation and 2) how your app behaves when it receives various messages. \n",
    "\n",
    "__IMPORTANT!__ The GridAPPS-D Platform uses the exact names and syntax for the methods:\n",
    "\n",
    "\n",
    "* `__init__(self, simulation_id, gapps_object, optional_objects)` -- This method requires the simulation_id and GridAPPS-D connection object. It is also possible add other user-defined arguments, such as measurement mRIDs or other information required by your application.\n",
    "\n",
    "\n",
    "* `on_message(self, headers, message)` -- This method allows the class to subscribe to simulation measurements. It also contains the core behavior of your application and how it responds to each type of message.\n",
    "\n",
    "It is also possible to use the same class definition to subscribe to other topics, such as Simulation Logs. This is done by creating additional user-defined methods and then passing those methods to the `.subcribe()` method associated with the GridAPPS-D connection object. An example of how this is done is provided for subcribing to simulation logs in [Logging with a Class Method](/api_usage/3.8-Using-the-Logging-API.ipynb#Create-Subscription-Class-Method).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7386ca5",
   "metadata": {},
   "source": [
    "```\n",
    "class YourSimulationClassName(object):\n",
    "    # Your documentation text here on what app does\n",
    "    \n",
    "    def __init__(self, simulation_id, gapps_obj, meas_obj, your_obj):\n",
    "        # Instantiate class with specific initial state\n",
    "        \n",
    "        # Attributes required by Simulation API\n",
    "        self._gapps = gapps_obj\n",
    "        self._simulation_id = simulation_id\n",
    "        \n",
    "        # Attributes to publish difference measurements\n",
    "        self.diff = DifferenceBuilder(simulation_id)\n",
    "        \n",
    "        # Custom attributes for measurements, custom info\n",
    "        self.meas_mrid = meas_obj\n",
    "        self.your_attribute1 = your_obj[\"key1\"]\n",
    "        self.your_attribute2 = your_obj[\"key2\"]\n",
    "        \n",
    "    def on_message(self, headers, message):\n",
    "        # What app should do when it receives a subscription message\n",
    "        \n",
    "        variable1 = message[\"message\"][\"key1\"]\n",
    "        variable2 = message[\"message\"][\"key2\"]\n",
    "\n",
    "        # Insert your custom app behavior here\n",
    "        if variable1 == foo:\n",
    "            bar = my_optimization_result\n",
    "            \n",
    "        # Insert your custom equipment commands here\n",
    "        if variable2 == bar:\n",
    "            self.diff.add_difference(object_mrid, control_attribute, new_value, old_value)\n",
    "            \n",
    "    def my_custom_method_1(self, headers, message):\n",
    "        # Use extra methods to subscribe to other topics, such as simulation logs\n",
    "        variable1 = message[\"key1\"]\n",
    "        variable2 = message[\"key2\"]\n",
    "        \n",
    "    def my_custom_method_2(self, param1, param2):\n",
    "        # Use extra methods as desired\n",
    "        variable1 = foo\n",
    "        variable2 = bar\n",
    "        \n",
    "        \n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "769b58f1",
   "metadata": {},
   "source": [
    "Below is the sample code for the core section of a basic application that tracks the number of open switches and the number of switches that are outaged."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b517f6f8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Application core built as a class definition\n",
    "class DemoAppCoreClass(object):\n",
    "    \n",
    "    # Subscription callback from GridAPPSD object\n",
    "    def __init__(self, simulation_id, gapps_obj, meas_obj):\n",
    "        self._gapps = gapps_obj # GridAPPS-D connection object\n",
    "        self._simulation_id = simulation_id # Simulation ID\n",
    "        self.meas_mrid = meas_obj # Dictionary of measurement mRIDs obtained earlier\n",
    "    \n",
    "    def on_message(self, headers, message):\n",
    "        \n",
    "        # Extract time and measurement values from message          \n",
    "        timestamp = message[\"message\"][\"timestamp\"]\n",
    "        meas_value = message[\"message\"][\"measurements\"]\n",
    "        \n",
    "        # Filter measurement mRIDs for position and voltage sensors\n",
    "        Pos_obj = [k for k in self.meas_mrid if k['type'] == 'Pos']\n",
    "        PNV_obj = [k for k in self.meas_mrid if k['type'] == 'PNV']\n",
    "        \n",
    "         # Example 1: Count the number of open switches\n",
    "        open_switches = []\n",
    "        for index in Pos_obj:\n",
    "            if index[\"measid\"] in meas_value: \n",
    "                mrid = index[\"measid\"]\n",
    "                power = meas_value[mrid]\n",
    "                if power[\"value\"] == 0: # Filter to measurements with value of zero\n",
    "                    open_switches.append(index[\"eqname\"])\n",
    "        \n",
    "        # Print message to command line\n",
    "        print(\"............\")\n",
    "        print(\"Number of open switches at time\", timestamp, ' is ', len(set(open_switches)))\n",
    "    \n",
    "    \n",
    "        # Example 2: Count the number of outaged switches (voltage = 0)\n",
    "        dead_switches = []\n",
    "        for index in PNV_obj:\n",
    "            if index[\"measid\"] in meas_value:\n",
    "                mrid = index[\"measid\"]\n",
    "                voltage = meas_value[mrid]\n",
    "                if voltage[\"magnitude\"] == 0.0:\n",
    "                    dead_switches.append(index[\"eqname\"])\n",
    "        \n",
    "        # Print message to command line\n",
    "        print(\"............\")\n",
    "        print(\"Number of outaged switches at time\", timestamp, ' is ', len(set(dead_switches)))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a036a9f",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "460e5301",
   "metadata": {},
   "source": [
    "## Subscribing to Simulation Output\n",
    "\n",
    "### Simulation Subscription Information Flow\n",
    "\n",
    "The figure below shows the information flow involved in subscribing to the simulation output.\n",
    "\n",
    "The subscription request is sent using `gapps.subscribe(topic, class/function object)` on the specific Simulation topic channel (explained in [API Communication Channels](../api_usage/3.1-API-Communication-Channels.ipynb)). No immediate response is expected back from the platform. However, after the next simulation timestep, the Platform will continue to deliver a complete set of measurements back to the application for each timestep until the end of the simulation."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d16c9a30",
   "metadata": {},
   "source": [
    "![subscribe-to-sim](images/3.6/06_subscribe_to_sim.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7861e07",
   "metadata": {},
   "source": [
    "__Application passes subscription request to GridAPPS-D Platform__\n",
    "\n",
    "The subscription request is perfromed by passing the app core algorithm function / class definition to the `gapps.subscribe` method. The application then passes the subscription request through the Simulation API to the topic channel for the particular simulation on the GOSS Message Bus. If the application is authorized to access simulation output, the subscription request is delivered to the Simulation Manager.\n",
    "\n",
    "__GridAPPS-D Platform delivers published simulation output to Application__\n",
    "\n",
    "Unlike the previous queries made to the various databases, the GridAPPS-D Platform does not provide any immediate response back to the application. Instead, the Simulation Manager will start delivering measurement data back to the application through the Simulation API at each subsequent timestep until the simulation ends or the application unsubscribes. The measurement data is then passed to the core algorithm class / function, where it is processed and used to run the app's optimization / control algorithms. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b142889",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0779ee65",
   "metadata": {},
   "source": [
    "### Subscription API Communication Channel\n",
    "\n",
    "This is a dynamic `/topic/` communication channel that is best implemented by importing the GriAPPSD-Python library function for generating the correct topic. This communication channel is used for all simulation subscription API calls."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2ae760f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "from gridappsd.topics import simulation_output_topic\n",
    "\n",
    "output_topic = simulation_output_topic(viz_simulation_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd3e62ad",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7207c434",
   "metadata": {},
   "source": [
    "### Comparison of Subscription Approaches\n",
    "\n",
    "Each approach has its advantages and disadvantages. \n",
    "\n",
    "* The function-based approach is simpler and easier to implement. However, any parameters obtained from other APIs or methods to be used inside the function currently need to be defined as global variables. \n",
    "\n",
    "\n",
    "* The class-based approach is more complex, but also more powerful. It provides greater flexibility in creating additional methods, arguments, etc.\n",
    "\n",
    "\n",
    "* The Simulation Library-based approach is easiest, but only works currently for parallel digital twin simulations started using the `simulation_obj.start_simulation()` method.\n",
    "\n",
    "The choice of which approach is used depends on the personal preferences of the application developer."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74271b3c",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1496211",
   "metadata": {},
   "source": [
    "### Subscription for Function-based App Core\n",
    "\n",
    "If the application core was created as a function definition as shown in [App Core as Function Definition](#App-Core-as-a-Function-Definition), then the function name is passed to the `.subscribe(output_topic, core_function)` method of the GridAPPS-D Connection object. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4b08eb84",
   "metadata": {},
   "outputs": [],
   "source": [
    "conn_id = gapps.subscribe(output_topic, DemoAppCoreFunction)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12e1b530",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8657774",
   "metadata": {},
   "source": [
    "### Subscription for Class-Based App Core\n",
    "\n",
    "If the application core was created as a class definition as shown in [App Core as Class Definition](#App-Core-as-a-Class-Definition), then the function name is passed to the `.subscribe(output_topic, object)` method of the GridAPPS-D connection object. \n",
    "\n",
    "After defining the class for the application core as shown above, we create another object that will be passed to the subscription method. The required parameters for this object are the same as those defined for the `__init__()` method of the app core class, typically the Simulation ID, GridAPPS-D connection object, dictionary of measurements needed by the app core, and any user-defined objects.\n",
    "\n",
    "`class_obj = AppCoreClass(simulation_id, gapps_obj, meas_obj, your_obj)` "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "51b9bda7",
   "metadata": {},
   "outputs": [],
   "source": [
    "demo_obj = DemoAppCoreClass(viz_simulation_id, gapps, measurements_obj)\n",
    "\n",
    "conn_id = gapps.subscribe(subscribe_topic, demo_obj)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06b73aea",
   "metadata": {},
   "source": [
    "If we wish to subscribe to an additional topic (such as the Simulation Logs, a side communication channel between two different applications, or a communication with a particular service), we can define an additional method in the class (such as my_custom_method_1 in the [example class definition](#App-Core-as-a-Class-Definition) above) and then pass it to to the `.subscribe(topic, object.method)` method associated with the GridAPPS-D connection object:\n",
    "\n",
    "`gapps.subscribe(other_topic, demo_obj.my_custom_method_1)`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10c06dec",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "64d97d6d",
   "metadata": {},
   "source": [
    "## Subscribing to Parallel Simulations\n",
    "\n",
    "Parallel simulations started using the Simulation API (as shown in [Starting a Simulation](../api_usage/3.5-Creating-Running-Simulation-API.ipynb#Starting-the-Simulation)) and the `Simulation` library in GridAPPSD-Python do not need to use the `gapps.subscribe` method.\n",
    "\n",
    "Instead, the GridAPPSD-Python library contains several shortcut functions that can be used. These methods currently cannot interact with a simulation started from the Viz. This functionality will be added in a future release.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97229aeb",
   "metadata": {},
   "source": [
    "The code block below shows how a parallel simulation can be started using a simulation start message stored in a JSON file. The simulation is started using the `.start_simulation()` method. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b76f6643",
   "metadata": {},
   "outputs": [],
   "source": [
    "import json, os\n",
    "from gridappsd import GridAPPSD\n",
    "from gridappsd.simulation import Simulation\n",
    "\n",
    "# Connect to GridAPPS-D Platform\n",
    "os.environ['GRIDAPPSD_USER'] = 'tutorial_user'\n",
    "os.environ['GRIDAPPSD_PASSWORD'] = '12345!'\n",
    "gapps = GridAPPSD()\n",
    "assert gapps.connected\n",
    "\n",
    "model_mrid =  \"_C1C3E687-6FFD-C753-582B-632A27E28507\"\n",
    "run123_config = json.load(open(\"Run123NodeFileSimAPI.json\")) # Pull simulation config from saved file   \n",
    "simulation_obj = Simulation(gapps, run123_config) # Create Simulation object\n",
    "simulation_obj.start_simulation() # Start Simulation\n",
    "\n",
    "print(\"Successfully started simulation with simulation_id: \", simulation_obj.simulation_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e1306747",
   "metadata": {},
   "outputs": [],
   "source": [
    "simulation_id = simulation_obj.simulation_id"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1231814",
   "metadata": {},
   "source": [
    "The Simulation library provides four methods that can be used to define how the platform interacts with the simulation:\n",
    "\n",
    "* `.add_ontimestep_callback(myfunction1)` -- Run the desired function on each timestep\n",
    "\n",
    "\n",
    "* `.add_onmesurement_callback(myfunction2)` -- Run the desired function when a measurement is received. \n",
    "\n",
    "\n",
    "* `.add_oncomplete_callback(myfunction3)` -- Run the desired function when simulation is finished\n",
    "\n",
    "\n",
    "* `.add_onstart_callback(myfunction4)` -- Run desired function when simulation is started\n",
    "\n",
    "\n",
    "__Note: method name `.add_onmesurement_callback` is misspelled in the library definition!!__"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52251e79",
   "metadata": {},
   "source": [
    "Note that the measurement callback method returns just the measurements and timestamps without any of the message formatting used in the messages received by using the `gapps.subscribe(output_topic, object)` approach. \n",
    "\n",
    "The python dictionary returned by the GridAPPS-D Simulation output to the `.add_onmesurement_callback()` method is always named `measurements` and uses the following key-value pairs format:\n",
    "\n",
    "```\n",
    "{\n",
    "    '_pnv_meas_mrid_1': {'angle': number,\n",
    "                         'magnitude': number,\n",
    "                         'measurement_mrid': '_pnv_meas_mrid_1'},\n",
    "    '_va_meas_mrid_2': { 'angle': number,\n",
    "                         'magnitude': number,\n",
    "                         'measurement_mrid': '_va_meas_mrid_2'}, \n",
    "    '_pos_meas_mrid_3': {'measurement_mrid': '_pos_meas_mrid_3',\n",
    "                         'value': 1},\n",
    "           .\n",
    "           .\n",
    "           .\n",
    "    '_pnv_meas_mrid_n': {'angle': number,\n",
    "                         'magnitude': number,\n",
    "                         'measurement_mrid': '_pnv_meas_mrid_1'}\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ef81bd3",
   "metadata": {},
   "source": [
    "To use use these methods, we define a set of functions that determine the behavior of the application for each of the four types of callbacks listed above. These functions are similar to those defined for the function-based app core algorithm. \n",
    "\n",
    "```\n",
    "def my_onstart_func(sim):\n",
    "    # Do something when the simulation starts\n",
    "    # Do something else when the sim starts\n",
    "\n",
    "simulation_obj.add_onstart_callback(my_onstart_func)\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc907365",
   "metadata": {},
   "source": [
    "```\n",
    "def my_onmeas_func(sim, timestamp, measurements):\n",
    "    # Do something when app receives a measurement\n",
    "    # Insert your custom app behavior here\n",
    "    if measurements[object_mrid] == foo:\n",
    "        bar = my_optimization_result\n",
    "        \n",
    "simulation_obj.add_onmesurement_callback(my_onmeas_func)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12841968",
   "metadata": {},
   "source": [
    "```\n",
    "def my_oncomplete_func(sim):\n",
    "    # Do something when simulation is complete\n",
    "    # example: delete all variables, close files\n",
    "    \n",
    "simulation_obj.add_oncomplete_callback(my_oncomplete_func)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99aa7b19",
   "metadata": {},
   "source": [
    "The code block below shows how the same app core algorithm can be used for a parallel simulation using the `.add_onmesurement_callback()` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e40fab5c",
   "metadata": {},
   "outputs": [],
   "source": [
    "def demo_onmeas_func(sim, timestamp, measurements):\n",
    "    \n",
    "    open_switches = []\n",
    "    for index in Pos_obj:\n",
    "        if index[\"measid\"] in measurements:\n",
    "            mrid = index[\"measid\"]\n",
    "            power = measurements[mrid]\n",
    "            if power[\"value\"] == 0:\n",
    "                open_switches.append(index[\"eqname\"])\n",
    "\n",
    "    print(\"............\")\n",
    "    print(\"Number of open switches at time\", timestamp, ' is ', len(set(open_switches)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "484155a7",
   "metadata": {},
   "outputs": [],
   "source": [
    "simulation_obj.add_onmesurement_callback(demo_onmeas_func)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8db46c42",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eff568a9",
   "metadata": {},
   "source": [
    "## Publishing Commands to Simulation Input\n",
    "\n",
    "The next portion of a GridAPPS-D App is publishing equipment control commands based on the optimization results or objectives of the app algorithm. \n",
    "\n",
    "Depending on the preference of the developer, this portion can be a separate function definition, or included as part of the main class definition as part of the [App Core as a Class Definition](#App-Core-as-a-Class-Definition) described earlier."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "933d87b1",
   "metadata": {},
   "source": [
    "### Equipment Command Information Flow\n",
    "\n",
    "The figure below outlines information flow involved in publishing equipment commands to the simulation input. \n",
    "\n",
    "Unlike the various queries to the databases in the app sections earlier, equipment control commands are passed to the GridAPPS-D API using the `gapps.send(topic, message)` method. No response is expected from the GridAPPS-D platform. \n",
    "\n",
    "If the application desires to verify that the equipment control command was received and implemented, it needs to do so by 1) checking for changes in the associated measurements at the next timestep and/or 2) querying the Timeseries Database for historical simulation data associated with the equipment control command."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad2940fd",
   "metadata": {},
   "source": [
    "![publish-commands](images/3.6/07_publish_commands.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30658434",
   "metadata": {},
   "source": [
    "__Application sends difference message to GridAPPS-D Platform__\n",
    "\n",
    "First, the application creates a difference message containing the current and desired future control point / state of the particular piece of power system equipment to be controlled. The difference message is a JSON string or equivalant Python dictionary object. The syntax of a difference message is explained in detail below in [Format of Difference Message](#Format-of-a-Difference-Message).\n",
    "\n",
    "The application then passes the query through the Simulation API to the GridAPPS-D Platform, which publishes it on the topic channel for the particular simulation on the GOSS Message Bus. If the app is authenticated and authorized to control equipment, the difference message is delivered to the Simulation Manager. The Simulation Manager then passes the command to the simulation through the Co-Simulation Bridge (either FNCS or HELICS).\n",
    "\n",
    "__No response from GridAPPS-D Platform back to Application__\n",
    "\n",
    "The GridAPPS-D Platform does not provide any response back to the application after processing the difference message and implementing the new equipment control setpoint."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b081f3b",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e790e18d",
   "metadata": {},
   "source": [
    "### Simulation Input API Channel\n",
    "\n",
    "This is a dynamic `/topic/` communication channel that is best implemented by importing the GriAPPSD-Python library function for generating the correct topic. \n",
    "\n",
    "* `from gridappsd.topics import simulation_input_topic`\n",
    "* `input_topic = simulation_input_topic(simulation_id)`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cb75b3e9",
   "metadata": {},
   "outputs": [],
   "source": [
    "from gridappsd.topics import simulation_input_topic\n",
    "\n",
    "input_topic = simulation_input_topic(viz_simulation_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f5cd9d1",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d9342ee",
   "metadata": {},
   "source": [
    "### Equipment Control mRIDs\n",
    "\n",
    "The mRIDs for controlling equipment are generally the same as those obtained using the `QUERY_OBJECT_DICT` key with the PowerGrid Models API, which was covered in [Query for Object Dicionary](../api_usage/3.3-Using-the-PowerGrid-Models-API.ipynb#Query-for-Object-Dictionary).\n",
    "\n",
    "However, the control attributes for each class of equipment in CIM use a different naming convention than those for the object types. Below is a list of `\"objectType\"` used to query for mRIDs and the associated control attribute used in a difference message for each category of power system equipment:\n",
    "\n",
    "* __Switches__\n",
    "    * CIM Class Key: `\"objectType\": \"LoadBreakSwitch\"`\n",
    "    * Control Attribute: `\"attribute\": \"Switch.open\"`\n",
    "    * Values: `1` is open, `0` is closed\n",
    "\n",
    "* __Capacitor Banks:__\n",
    "    * CIM Class Key: `\"objectType\": \"LinearShuntCompensator\"`\n",
    "    * Control Attribute: `\"attribute\": \"ShuntCompensator.sections\"`\n",
    "    * Values: `0` is off/open, `1` is on/closed\n",
    "    \n",
    "* __Inverter-based DERs:__\n",
    "    * CIM Class Key: `\"objectType\": \"PowerElectronicsConnection\"`\n",
    "    * Control Attribute: `\"attribute\": \"PowerElectronicsConnection.p\"`\n",
    "    * Control Attribute: `\"attribute\": \"PowerElectronicsConnection.q\"`\n",
    "    * Values: number in Watts or VArs (not kW)\n",
    "    \n",
    "* __Synchronous Rotating (diesel) DGs:__\n",
    "    * CIM Class Key: `\"objectType\": \"SynchronousMachine\"`\n",
    "    * Control Attribute: `\"attribute\": \"RotatingMachine.p\"`\n",
    "    * Control Attribute: `\"attribute\": \"RotatingMachine.q\"`\n",
    "    * Values: number in Watts or VArs (not kW)\n",
    "    \n",
    "* __Regulating Transformer Tap:__\n",
    "    * CIM Class Key: `\"objectType\": \"RatioTapChanger\"`\n",
    "    * Control Attribute: `\"attribute\": \"TapChanger.step\"`\n",
    "    * Values: integer value for tap step\n",
    "    \n",
    "__The query for RatioTapChanger is not supported in the PowerGrid Models API at the current time. A custom SPARQL query needs to be done using the sample query in [CIMHub Sample Queries](https://github.com/GRIDAPPSD/CIMHub/blob/master/queries.txt)__\n",
    "\n",
    "The example below shows a query to obtain the correct mRID for switch SW2 in the IEEE 123 node model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0234e408",
   "metadata": {},
   "outputs": [],
   "source": [
    "from gridappsd import topics as t\n",
    "\n",
    "message = {\n",
    "    \"modelId\": model_mrid,\n",
    "    \"requestType\": \"QUERY_OBJECT_DICT\",\n",
    "    \"resultFormat\": \"JSON\",\n",
    "    \"objectType\": \"LoadBreakSwitch\"\n",
    "}\n",
    "\n",
    "response_obj = gapps.get_response(t.REQUEST_POWERGRID_DATA, message)\n",
    "switch_dict = response_obj[\"data\"]\n",
    "\n",
    "# Filter to get mRID for switch SW2:\n",
    "for index in switch_dict:\n",
    "    if index[\"IdentifiedObject.name\"] == 'sw2':\n",
    "        sw_mrid = index[\"IdentifiedObject.mRID\"]\n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f20b177a",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d19de1d6",
   "metadata": {},
   "source": [
    "### Format of a Difference Message\n",
    "\n",
    "The general format for a difference message is a python dictionary or equivalent JSON string that specifies the reverse difference and the forward difference, in compliance with the CIM standard:\n",
    "\n",
    "The __reverse difference__ is the current status / value associated with the control attribute. It is a formatted as a list of dictionary constructs, with each dictionary specifying the equipment mRID associated with the CIM class keys above, the control attribute, and the current value of that control attribute. The list can contain reverse differences for multiple pieces of equipment.\n",
    "\n",
    "The __forward difference__ is the desired new status / value associated with the control attribute. It is a formatted as a list of dictionary constructs, with each dictionary specifying the equipment mRID associated with the CIM class keys above, the control attribute, and the current value of that control attribute. The list can contain foward differences for multiple pieces of equipment.\n",
    "\n",
    "```\n",
    "message = {\n",
    "  \"command\": \"update\",\n",
    "  \"input\": {\n",
    "      \"simulation_id\": \"simulation id as string\",\n",
    "      \"message\": {\n",
    "          \"timestamp\": epoch time number,\n",
    "          \"difference_mrid\": \"optional unique mRID for command logs\",\n",
    "          \"reverse_differences\": [{\n",
    "\n",
    "                  \"object\": \"first equipment mRID\",\n",
    "                  \"attribute\": \"control attribute\",\n",
    "                  \"value\": current value\n",
    "              },\n",
    "              {\n",
    "\n",
    "                  \"object\": \"second equipment mRID\",\n",
    "                  \"attribute\": \"control attribute\",\n",
    "                  \"value\": current value\n",
    "              }\n",
    "          ],\n",
    "          \"forward_differences\": [{\n",
    "\n",
    "                  \"object\": \"first equipment mRID\",\n",
    "                  \"attribute\": \"control attribute\",\n",
    "                  \"value\": new value\n",
    "              },\n",
    "              {\n",
    "\n",
    "                  \"object\": \"second equipment mRID\",\n",
    "                  \"attribute\": \"control attribute\",\n",
    "                  \"value\": new value\n",
    "              }\n",
    "              ]\n",
    "              }\n",
    "      }\n",
    "}\n",
    "\n",
    "```\n",
    "\n",
    "Note: The GridAPPS-D platform does not validate whether `\"reverse_differences\":` has the correct equipment control values for the current time. It is used just for compliance with the CIM standard."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97f7e1b9",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f05a790",
   "metadata": {},
   "source": [
    "### Using GridAPPSD-Python DifferenceBuilder"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73a579b5",
   "metadata": {},
   "source": [
    "The `DifferenceBuilder` class is a GridAPPSD-Python tool that can be used to automatically build the difference message with correct formatting. \n",
    "\n",
    "First, import DifferenceBuilder from the GridAPPSD-Python Library and create an object that will be used to create the desired difference messages. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4b0a74d6",
   "metadata": {},
   "outputs": [],
   "source": [
    "from gridappsd import DifferenceBuilder\n",
    "\n",
    "my_diff_build = DifferenceBuilder(viz_simulation_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f51efcc4",
   "metadata": {},
   "source": [
    "We then use two methods associated with the DifferenceBuilder object:\n",
    "\n",
    "\n",
    "* `.add_difference(self, object_mrid, control_attribute, new_value, old_value)` -- Generates a correctly formatted difference message.\n",
    "\n",
    "\n",
    "* `.get_message()` -- Saves the message as a python dictionary that can be published using `gapps.send(topic, message)`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "212896b9",
   "metadata": {},
   "outputs": [],
   "source": [
    "my_diff_build.add_difference(sw_mrid, \"Switch.open\", 1, 0) # Open switch given by sw_mrid\n",
    "\n",
    "message = my_diff_build.get_message()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b855b5b5",
   "metadata": {},
   "source": [
    "The difference message is then published to the GOSS Message Bus and the Simulation API using the `.send()` method associated with the GridAPPS-D connection object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ed016a2d",
   "metadata": {},
   "outputs": [],
   "source": [
    "gapps.send(input_topic, message)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1a955b3",
   "metadata": {},
   "source": [
    "## Unsubscribing from a Simulation\n",
    "\n",
    "To unsubscribe from a simulation, pass the connection ID to `.unsubscribe(conn_id)` method of the GridAPPS-D connection object. The connection ID was obtained earlier when subscribing using `conn_id = gapps.subscribe(topic, message)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "84810b3e",
   "metadata": {},
   "outputs": [],
   "source": [
    "gapps.unsubscribe(conn_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d9d20c4",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c18b7954",
   "metadata": {},
   "source": [
    "![GridAPPS-D_narrow.png](../images/GridAPPS-D_narrow.png)"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Tags",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}