module-content/docs/source/api_usage/3.5-Creating-Running-Simulation-API.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Running Simulations with the Simulation API\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction to the Simulation API\n",
    "\n",
    "\n",
    "The Simulation API is used for all actions related to a power system simulation. It is used to start, pause, restart, and stop a simulation from the command line or inside an application. It is all used to subscribe to measurements of equipment (such as line flows, loads, and DG setpoints) and the statuses of switches, capacitors, transformer taps, etc. It is also used to publish equipment control and other simulation input commands.\n",
    "\n",
    "In the Application Components diagram (explained in detail with sample code in [GridAPPS-D Application Structure](../overview/2.4-GridAPPS-D-Application-Structure.ipynb)), the PowerGrid Models API is used for controlling the simulation, subscribing to measurement data, and controlling equipment.\n",
    "\n",
    "This section covers only the portion of the API used for starting, stopping, and pausing simulations. Equipment control commands are covered in [Controlling Simulation with the Simulation API](../api_usage/3.6-Controlling-Simulation-API.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![subscribe-to-sim](images/3.5/06_subscribe_to_sim.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## API Syntax Overview\n",
    "\n",
    "__Application passes subscription request to GridAPPS-D Platform__\n",
    "\n",
    "The subscription request is perfromed by passing the app core algorithm function / class definition to the `gapps.subscribe` method. The application then passes the subscription request through the Simulation API to the topic channel for the particular simulation on the GOSS Message Bus. If the application is authorized to access simulation output, the subscription request is delivered to the Simulation Manager.\n",
    "\n",
    "__GridAPPS-D Platform delivers published simulation output to Application__\n",
    "\n",
    "Unlike the previous queries made to the various databases, the GridAPPS-D Platform does not provide any immediate response back to the application. Instead, the Simulation Manager will start delivering measurement data back to the application through the Simulation API at each subsequent timestep until the simulation ends or the application unsubscribes. The measurement data is then passed to the core algorithm class / function, where it is processed and used to run the app's optimization / control algorithms. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### API Communication Channel \n",
    "\n",
    "The Simulation API is used to communicate with a broad range of subscribers through both static `/queue/` (the type of channel used for passing databse queries) and dynamic `/topic/` communication channel names. \n",
    "\n",
    "Extreme care is needed to use the correct communication channel. For a review of GridAPPS-D topics, see [API Communication Channels](../api_usage/3.1-API-Communication-Channels.ipynb)\n",
    "\n",
    "The correct communication channel for each Simulation API call will be provided in the corresponding section for each API task below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Structure of a Simulation Message\n",
    "\n",
    "\n",
    "Due to the wide range of tasks accomplished by the Simualtion API, there is no single message format that is used across all API Calls. \n",
    "\n",
    "Each message takes the form of a python dictionary or equivalent JSON script wrapped as a python string. \n",
    "\n",
    "The structure of each Simulation API call message will be provided in the corresponding section for each API task below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Starting a Simulation\n",
    "\n",
    "This section explains the API call used to start simulations. For most application developers, typical use cases would be either \n",
    "\n",
    "* starting a digital twin simulation from within an app for \"what-if\" analysis\n",
    "* building an automated test script that would launch a simulation with exactly identical parameters each time.\n",
    "\n",
    "The API message structure and start command are also used by the GridAPPS-D Viz when starting a simulation using the steps in [Creating a Simulation with GridAPPS-D Viz](../installation/1.5-Using-GridAPPS-D-Viz.ipynb#Creating-a-Simulation). However, the API call and detail simulation start message are hidden from the user when using Viz.\n",
    "\n",
    "\n",
    "### GridAPPS-D Communication Channel\n",
    "\n",
    "This is a static `/queue/` communication channel used to quest the simulation to start. The base static string used is `goss.gridappsd.process.request.simulation`, which can be called using the `.REQUEST_SIMULATION` method from the topics library.\n",
    "\n",
    "When developing in python, it is recommended to use the `.REQUEST_SIMULATION` method. When using the STOMP client in GridAPPS-D VIZ, it is necessary to use the base static string."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from gridappsd import topics as t\n",
    "topic = t.REQUEST_SIMULATION"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Simulation Start Message Structure\n",
    "\n",
    "To start a simulation from the command line or from inside an application, it is necessary to pass a message with a structure similar to that used by the [Configuration File API](../api_usage/3.4-Using-the-Configuration-File-API.ipynb)) so that the GridAPPS-D platform can create a custom GridLab-D file from the CIM XML model stored in the Blazegraph Database.\n",
    "\n",
    "The accepted set of key-value pairs for the Simulation API call to start a new simulation is\n",
    "\n",
    "```\n",
    "message = {\n",
    "    \"power_system_config\": {\n",
    "        \"key1\": \"value1\",\n",
    "        \"key2\": \"value2\"},\n",
    "    \"simulation_config\": {\n",
    "        \"key1\": \"value1\",\n",
    "        \"key2\": \"value2\",\n",
    "        \"model_creation_config\": {\n",
    "            \"key1\": \"value1\",\n",
    "            \"key2\": \"value2\",\n",
    "            \"model_state\": {\n",
    "                \"key1\": \"value1\",\n",
    "                \"key2\": \"value2\"} \n",
    "            }\n",
    "         },\n",
    "    \"simulation_output\": {\n",
    "        \"key1\": \"value1\",\n",
    "        \"key2\": \"value2\"},\n",
    "    \"application_config\": [{\n",
    "        \"key1\": \"value1\",\n",
    "        \"key2\": \"value2\"}],\n",
    "    \"test_config\": {\n",
    "        \"events\": [{,\n",
    "            \"message\": {\n",
    "                \"forward_differences\": [{\n",
    "                    \"key1\": \"value1\",\n",
    "                    \"key2\": \"value2\"}]\n",
    "                \"reverse_differences\": [{\n",
    "                    \"key1\": \"value1\",\n",
    "                    \"key2\": \"value2\"}],\n",
    "            \"key1\": \"value1\",\n",
    "            \"key2\": \"value2\"}\n",
    "            }]       \n",
    "    },\n",
    "    \"service_configs\": {\n",
    "        \"key1\": \"value1\",\n",
    "        \"key2\": \"value2\"} \n",
    "}\n",
    "```\n",
    "\n",
    "The components of the message are identical to the options in the tabs in the GridAPPS-D. Each key-value pair is explained in detail below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  Power System Configuration\n",
    "\n",
    "This key is __required__ and specifies the CIM XML model to be imported and used for the simulation. Required key-value pairs are\n",
    "\n",
    "```\n",
    "\"power_system_config\": {   REQUIRED KEY                      REQUIRED VALUE\n",
    "                     \"GeographicalRegion_name\":           mRID as string ,\n",
    "                     \"SubGeographicalRegion_name\":        mRID as string ,\n",
    "                     \"Line_name\":                         mRID as string }\n",
    "                \n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As can be seen, the required key-value pairs are identical to those that are selected graphically using the GridAPPS-D Viz:\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![model-config](images/3.5/config_system_config.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Simulation Configuration\n",
    "\n",
    "The `\"simulation_config\":` key is __required__ and specifies the runtime parameters, such as simulation duration and time step. Required key-value pairs are\n",
    "\n",
    "```\n",
    "\"simulation_config\": {   REQUIRED KEY                      REQUIRED VALUE\n",
    "                    \"start_time\":                    epoch time string ,\n",
    "                    \"duration\":                      number (seconds) ,\n",
    "                    \"simulator\":                     \"GridLAB-D\" ,\n",
    "                    \"timestep_frequency\":            number (milliseconds) ,\n",
    "                    \"timestep_increment\":            number (milliseconds) ,\n",
    "                    \"run_realtime\":                  true OR false ,\n",
    "                    \"simulation_name\":               string ,\n",
    "                    \"power_flow_solver_method\":      \"NR\" ,\n",
    "                    \"model_creation_config\":         key-value pairs }\n",
    "```\n",
    "\n",
    "The key `\"model_creation_config\":` is one of the __optional__ key-value pairs that can specified. This specifies the load profile and changes to the base case used for the simulation. Key-value pairs are\n",
    "\n",
    "```\n",
    "    \"model_creation_config\": { KEY                       VALUE\n",
    "                        \"load_scaling_factor\":             number ,\n",
    "                        \"schedule_name\":                   string ,\n",
    "                        \"z_fraction\":                      number ,\n",
    "                        \"i_fraction\":                      number ,\n",
    "                        \"p_fraction\":                      number ,\n",
    "                        \"randomize_zipload_fractions\":     true OR false ,\n",
    "                        \"use_houses\":                      true OR false , \n",
    "                        \"model_state\":                     key-value pairs }\n",
    "                        \n",
    "```\n",
    "\n",
    "The key `\"model_state:\"` is one of the __optional__ key-value pairs used to specify changes to switch positions and DER setpoints from the base case stored in the Blazegraph database. Key-value pairs are\n",
    "\n",
    "```\n",
    "        \"model_state\": {  \n",
    "                        \"synchronousmachines\":[\n",
    "                                {\"name\": \"GLM/DSS name\",\"p\": number,\"q\": number},\n",
    "                                {\"name\": \"GLM/DSS name\",\"p\": number,\"q\": number}],\n",
    "                        \"switches\":[\n",
    "                                {\"name\":\"GLM/DSS name\", \"open\": true},\n",
    "                                {\"name\":\"GLM/DSS name\", \"open\": false}]\n",
    "            }\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As can be seen, the required key-value pairs are identical to those that are selected graphically using the GridAPPS-D Viz. By default, only the required key-value pairs are listed in the viz. The simulation launched in the Viz can be configured using any of the `\"model_creation_config\"` and `\"model_state\"` key-value pairs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![model-state-config](images/3.5/config_sim_config.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Application Configuration\n",
    "\n",
    "The `\"application_config\":` key is __optional__ and specifies which applications will run during the simulation. The associated value is a list that specifies the name and application-specific configuration parameters. \n",
    "\n",
    "```\n",
    "\"application_config\": {\n",
    "        \"applications\": [\n",
    "                {\n",
    "                \"name\": \"application1\",\n",
    "                \"config_string\": \"application-specific config string\"\n",
    "                }, \n",
    "                {\n",
    "                \"name\": \"application2\",\n",
    "                \"config_string\": \"application-specific config string\"\n",
    "                }]\n",
    "}              \n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As can be seen, the required key-value pairs are identical to those that are selected graphically using the GridAPPS-D Viz. Note: simulations started using the GridAPPS-D Viz can only run one app at a time. This is due to the GUI design and not a limitation of the Simulation API.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![app-config](images/3.5/config_app_config.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Test Manager Configuration\n",
    "\n",
    "The TestManager is used to create realistic operational events during the simulation run, including faults, communication outages, and other unexpected equipment actions. These types of events can be used to test robustness of applications in variable conditions similar to those of the real world.\n",
    "\n",
    "There are three types events supported by the TestManager and the GridAPPS-D Platform:\n",
    "\n",
    "1. CIM defined fault events, used when a line is down or for taking a piece of equipment out of service.  \n",
    "2. Communication outage events which simulates measurements or control message outages.\n",
    "3. Scheduled command at specific time which sends commands to a piece of equipment.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![event-classes](images/3.5/EventClassDiagram.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When starting a simulation, any combination of event categories can be specified using the `\"test_config\":` key-value pair structure. This key is __optional__ and specifies whether any pre-scripted events should be scheduled to occur during the simulation, such as faults, communication outages, and equipment malfunctions.  \n",
    "\n",
    "The TestManager configuration can be written within the text of the Simulation API call or imported from a JSON file. Each of the event categories are discussed in further detail below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    "\"test_config\": {\n",
    "    \"events\": [\n",
    "        {\n",
    "            \"event_type\": \"CommOutage\",\n",
    "            \"tag\": \"unique_tag\",\n",
    "            \"startDateTime\": \"YYYY-MM-DD HH:MM:SS\" OR epoch time number,\n",
    "            \"stopDateTime\": \"YYY-MM-DD HH:MM:SS\" OR epoch time number,\n",
    "            \"allInputOutage\": true or false,\n",
    "            \"inputList\": [\n",
    "                {\n",
    "                    \"name\": \"equipment name\",\n",
    "                    \"type\": \"CIM equipment type\",\n",
    "                    \"mRID\": \"equipment object mRID\",\n",
    "                    \"phases\": [\n",
    "                        {\n",
    "                            \"phaseLabel\": \"A\" or \"B\" or \"C\",\n",
    "                            \"phaseIndex\": number\n",
    "                        }\n",
    "                    ],\n",
    "                    \"attribute\": \"CIM Control Attribute\"\n",
    "                }\n",
    "            ],\n",
    "            \"allOutputOutage\": true or false,\n",
    "            \"outputList\": [\n",
    "                {\n",
    "                    \"name\": \"equipment name\",\n",
    "                    \"type\": \"CIM equipment type\",\n",
    "                    \"mRID\": \"equipment object mRID\",\n",
    "                    \"phases\": [ \"A\" or \"B\" or \"C\"  ],\n",
    "                    \"measurementTypes\": [ \"PNV\" or \"VA\" or \"POS\" ]\n",
    "                }\n",
    "            ]\n",
    "        },\n",
    "        {\n",
    "            \"event_type\": \"Fault\",\n",
    "            \"tag\": \"unique_tag\",\n",
    "            \"equipmentType\": \"CIM object type\",\n",
    "            \"equipmentName\": \"equipment name\",\n",
    "            \"mRID\": [\n",
    "                \"equipment object mRID\",\n",
    "                \"equipment object mRID\",\n",
    "                \"equipment objhect mRID\"\n",
    "            ],\n",
    "            \"phases\": [\n",
    "                {\n",
    "                    \"phaseLabel\": \"A\" or \"B\" or \"C\",\n",
    "                    \"phaseIndex\": number\n",
    "                }\n",
    "            ],\n",
    "            \"faultKind\": \"lineToGround\" or \"lineToLine\" or \"lineToLineToGround\",\n",
    "            \"faultImpedance\": {\n",
    "                \"rGround\": \"value\",\n",
    "                \"xGround\": \"value\"\n",
    "            },\n",
    "            \"startDateTime\": \"YYYY-MM-DD HH:MM:SS\" OR epoch time number,\n",
    "            \"stopDateTime\": \"YYYY-MM-DD HH:MM:SS\" OR epoch time number\n",
    "        },\n",
    "        {\n",
    "            \"message\": {\n",
    "                \"forward_differences\": [\n",
    "                    {\n",
    "                        \"object\": \"control object mRID\",\n",
    "                        \"attribute\": \"CIM Class Attribute\",\n",
    "                        \"value\": number\n",
    "                    }\n",
    "                ],\n",
    "                \"reverse_differences\": [\n",
    "                    {\n",
    "                        \"object\": \"control object mRID\",\n",
    "                        \"attribute\": \"CIM Class Attribute\",\n",
    "                        \"value\": number\n",
    "                    }\n",
    "                ]\n",
    "            },\n",
    "            \"event_type\": \"ScheduledCommandEvent\",\n",
    "            \"occuredDateTime\": \"YYYY-MM-DD HH:MM:SS\" OR epoch time number,\n",
    "            \"stopDateTime\": \"YYYY-MM-DD HH:MM:SS\" OR epoch time number\n",
    "        }\n",
    "    ]\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Fault Events\n",
    "\n",
    "Fault Events are defined in a Test Script and define the CIM Fault events that will be intialized and cleared at scheduled times.\n",
    "\n",
    "The phases string is all the combinations of the 3-phases plus neutral and secondary phases.\n",
    "Some examples are :\n",
    "\n",
    "1. \"A\"\n",
    "2. \"AN\" would be line to line.\n",
    "3. \"AB\" would be line to line.\n",
    "4. \"S12N\" both hot wires to ground\n",
    "5. \"S12\" both hot wires together.\n",
    "\n",
    "\n",
    "PhaseConnectedFaultKind is an enumeration with the following values:\n",
    "\n",
    "1.\tlineToGround\n",
    "2.\tlineToLine\n",
    "3.\tlineToLineToGround\n",
    "4.\tlineOpen\n",
    "\n",
    "\n",
    "The key-value pairs for specifying a fault event are:\n",
    "\n",
    "```\n",
    "{\n",
    "    \"PhaseConnectedFaultKind\": string,\n",
    "    \"FaultImpedance\": {\n",
    "        \"rGround\": float,\n",
    "        \"xGround\": float,\n",
    "        \"rLineToLine\":float,\n",
    "        \"xLineToLine\":float },\n",
    "    \"ObjectMRID\": [string],\n",
    "    \"phases\": string,\n",
    "    \"event_type\": string,\n",
    "    \"occuredDateTime\": long,\n",
    "    \"stopDateTime\": long\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Example 1: Fault Event Example in test script__\n",
    "\n",
    "```\n",
    "{   \n",
    "    \"command\": \"new_events\",\n",
    "    \"events\" : [{\n",
    "        \"PhaseConnectedFaultKind\": \"lineToGround\",\n",
    "        \"FaultImpedance\": {\n",
    "            \"rGround\": 0.001,\n",
    "            \"xGround\": 0.001 },\n",
    "        \"ObjectMRID\": [\"_9EF94B67-7279-21F4-5CEE-B2724E3C3FE6\"],\n",
    "        \"phases\": \"ABC\",\n",
    "        \"event_type\": \"Fault\",\n",
    "        \"occuredDateTime\": 1248130809,\n",
    "        \"stopDateTime\": 1248130816}\n",
    "    ]\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As can be seen, the required key-value pairs are identical to those that are selected graphically using the GridAPPS-D Viz. The Viz offers the equivalent options to create line faults, communications outages, or upload a custom difference message JSON file."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![test-config](images/3.5/config_test_fault.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Example 2: Commands sent from the Test Manager to the simulation__\n",
    "\n",
    "```\n",
    "{\n",
    "    \"command\": \"update\", \n",
    "    \"input\": {\n",
    "        \"timestamp\": 1553201000414, \n",
    "        \"reverse_differences\": [], \n",
    "        \"difference_mrid\": \"_ee4e4055-222f-4ccf-bed1-93063bd4392c\", \n",
    "        \"forward_differences\": [\n",
    "        {\n",
    "            \"ObjectMRID\": \"12344\", \n",
    "            \"FaultImpedance\": {\n",
    "                \"xLineToLine\": 0.0, \n",
    "                \"rGround\": 0.001, \n",
    "                \"rLineToLine\": 0.0, \n",
    "                \"xGround\": 0.001\n",
    "                }, \n",
    "            \"FaultMRID\": \"1233\", \n",
    "            \"PhaseCode\": \"AN\", \n",
    "            \"PhaseConnectedFaultKind\": \"lineToGround\"\n",
    "            }\n",
    "        ]\n",
    "    }\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Example 3: Clear a Fault Command example__\n",
    "\n",
    "```\n",
    "{\n",
    "    \"command\": \"update\", \n",
    "    \"input\": {\n",
    "        \"timestamp\": 1553201003561, \n",
    "        \"reverse_differences\": [{\n",
    "            \"ObjectMRID\": \"12344\", \n",
    "            \"FaultImpedance\": {\n",
    "                \"xLineToLine\": 0.0, \n",
    "                \"rGround\": 0.001, \n",
    "                \"rLineToLine\": 0.0, \n",
    "                \"xGround\": 0.001\n",
    "            }, \n",
    "            \"FaultMRID\": \"1233\", \n",
    "            \"PhaseCode\": \"AN\", \n",
    "            \"PhaseConnectedFaultKind\": \"lineToGround\"\n",
    "        }\n",
    "    ], \n",
    "    \"difference_mrid\": \"_00b4668d-8454-4f1c-aed9-42d1424af149\", \n",
    "    \"forward_differences\": []\n",
    "    }\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Communication Outage Events\n",
    "\n",
    "Communication Outage events are separate from the CIM events but have occuredDateTime and stopDateTime. \n",
    "\n",
    "1. The inputOutageList is the list of objectMRID and attribute pair.  The objectMRID is anything that can be controllable and specific control attribute i.e. \"RegulatingControl.mode\". \n",
    "2. The outputOutageList is the list of MRIDs for the measurement device that is associated to the . \n",
    "3. If allInputOutage is True the inputOutageList is not needed as all inputs to the simulator are blocked.\n",
    "4. If allOutputOutage is True the outputOutageList is not needed as all outputs from the simulator are blocked.\n",
    "\n",
    "The key-value pairs for communication outage events follow the format\n",
    "\n",
    "```\n",
    "{  \n",
    "    \"allOutputOutage\": boolean,\n",
    "    \"allInputOutage\": boolean,\n",
    "    \"inputOutageList\": [{\"objectMRID\":string, \"attribute\":string}],\n",
    "    \"outputOutageList\": [string],\n",
    "    \"event_type\": string,\n",
    "    \"occuredDateTime\": long,\n",
    "    \"stopDateTime\": long\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Example 1: JSON Communication Outage command for the TestManager__\n",
    "\n",
    "```\n",
    "{\n",
    "    \"command\": \"new_events\",\n",
    "    \"events\": [{\n",
    "        \"allOutputOutage\": false,\n",
    "        \"allInputOutage\": false,\n",
    "        \"inputOutageList\": [{\n",
    "            \"objectMRID\":\"_EF2FF8C1-A6A6-4771-ADDD-A371AD929D5B\",\n",
    "            \"attribute\":\"ShuntCompensator.sections\" },\n",
    "            {\"objectMRID\":\"_C0F73227-012B-B70B-0142-55C7C991A343\",\n",
    "            \"attribute\":\"ShuntCompensator.sections\"}],\n",
    "        \"outputOutageList\": [\"_5405BE1A-BC86-5452-CBF2-BD1BA8984093\"],\n",
    "        \"event_type\": \"CommOutage\",\n",
    "        \"occuredDateTime\": 1248130819,\n",
    "        \"stopDateTime\": 1248130824\n",
    "        }\n",
    "    ]\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As can be seen, the required key-value pairs are identical to those that are selected graphically using the GridAPPS-D Viz. The Viz offers the equivalent options to create line faults, communications outages, or upload a custom difference message JSON file."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![test-config](images/3.5/config_test_input_outage.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Example 2: Communication Event to the Simulation Bridge__\n",
    "\n",
    "```\n",
    "{\n",
    "    \"command\": \"CommOutage\",\n",
    "    \"input\": {\n",
    "        \"timestamp\": 1248130819,\n",
    "        \"forward_differences\": [\n",
    "        {\n",
    "            \"allOutputOutage\": false,\n",
    "            \"allInputOutage\": false,\n",
    "            \"inputOutageList\": [\n",
    "            {\n",
    "                \"objectMRID\": \"_EF2FF8C1-A6A6-4771-ADDD-A371AD929D5B\",\n",
    "                \"attribute\": \"ShuntCompensator.sections\"\n",
    "            },\n",
    "            {\n",
    "                \"objectMRID\": \"_C0F73227-012B-B70B-0142-55C7C991A343\",\n",
    "                \"attribute\": \"ShuntCompensator.sections\"\n",
    "            }\n",
    "            ],\n",
    "            \"outputOutageList\": [\n",
    "            \"_5405BE1A-BC86-5452-CBF2-BD1BA8984093\"\n",
    "            ],\n",
    "            \"faultMRID\": \"_ce5ee4c9-9c41-4f5e-8c5c-f19990f9cfba\",\n",
    "            \"event_type\": \"CommOutage\",\n",
    "            \"occuredDateTime\": 1248130819,\n",
    "            \"stopDateTime\": 1248130824\n",
    "        }\n",
    "        ],\n",
    "        \"reverse_differences\": []\n",
    "    }\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Scheduled Command Events\n",
    "\n",
    "Schedule Commands are events that can be scheduled for a specific point in time of the simulation.  This can be used to trigger a fault like behavior, such as changing the taps of a regular or mimicking behavior of protective devices.\n",
    "\n",
    "__Faults in topologically meshed networks need to created using a Scheduled Command event to open all switches that would be tripped by protective relaying due to a known bug in the GridLAB-D logical fault processor.__\n",
    "\n",
    "The set of key-value pairs for Scheduled Command events follow the format below. The message parameters follow the format of a difference message, which is explained in detail in [Format of Difference Messages](../api_usage/3.6-Controlling-Simulation-API.ipynb#Format-of-a-Difference-Message). \n",
    "\n",
    "```\n",
    "{   \n",
    "    \"command\": \"new_events\",\n",
    "    \"events\":[{\n",
    "        \"message\":{\n",
    "            \"forward_differences\": [{\n",
    "                \"object\": \"control object mRID\",\n",
    "                \"attribute\": \"CIM Class Attribute\",\n",
    "                \"value\": number }],\n",
    "            \"reverse_differences\": [{\n",
    "                \"object\": \"control object mRID\",\n",
    "                \"attribute\": \"CIM Class Attribute\",\n",
    "                \"value\": number }]\n",
    "            },\n",
    "        },\n",
    "        \"occuredDateTime\":long,\n",
    "        \"stopDateTime\":long,\n",
    "        }\n",
    "    ]\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Service Configuration\n",
    "\n",
    "The `\"service_configs\":` key is used to start a GridAPPS-D service, such as the Sensor Service, State Estimator, or DNP3 Service. The key-value pairs for starting the service depend on the particular service and will be covered in the documentation for each GridAPPS-D Service. Note that the syntax of this key is a plural noun, unlike the other config keys.\n",
    "\n",
    "* [DNP3 Service](https://gridappsd-training.readthedocs.io/en/develop/services/dnp3/source/index.html)\n",
    "* [Sensor Simulator Service](https://gridappsd-training.readthedocs.io/en/develop/services/sensor-simulator/source/index.html)\n",
    "* [State Estimator Service](https://gridappsd-training.readthedocs.io/en/develop/services/state-estimator/README.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Complete Simulation Start Message\n",
    "\n",
    "The complete simulation message that is passed to the Simulation API is a Python dictionary / JSON string that specified the `\"power_system_conf\":`, `\"application_config\":`, `\"simulation_config\":`, `\"test_config\":`, and `\"service_configs\":` key-value pairs.\n",
    "\n",
    "A sample simulation start message for the IEEE 123 bus system running the Sensor Simulator service and including one Scheduled Command event is shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    " run_config_123 = {\n",
    "    \"power_system_config\": {\n",
    "        \"GeographicalRegion_name\": \"_73C512BD-7249-4F50-50DA-D93849B89C43\",\n",
    "        \"SubGeographicalRegion_name\": \"_1CD7D2EE-3C91-3248-5662-A43EFEFAC224\",\n",
    "        \"Line_name\": \"_C1C3E687-6FFD-C753-582B-632A27E28507\"\n",
    "    },\n",
    "    \"application_config\": {\n",
    "        \"applications\": []\n",
    "    },\n",
    "    \"simulation_config\": {\n",
    "        \"start_time\": \"1570041113\",\n",
    "        \"duration\": \"120\",\n",
    "        \"simulator\": \"GridLAB-D\",\n",
    "        \"timestep_frequency\": \"1000\",\n",
    "        \"timestep_increment\": \"1000\",\n",
    "        \"run_realtime\": True,\n",
    "        \"simulation_name\": \"ieee123\",\n",
    "        \"power_flow_solver_method\": \"NR\",\n",
    "        \"model_creation_config\": {\n",
    "            \"load_scaling_factor\": \"1\",\n",
    "            \"schedule_name\": \"ieeezipload\",\n",
    "            \"z_fraction\": \"0\",\n",
    "            \"i_fraction\": \"1\",\n",
    "            \"p_fraction\": \"0\",\n",
    "            \"randomize_zipload_fractions\": False,\n",
    "            \"use_houses\": False\n",
    "        }\n",
    "    },\n",
    "    \"test_config\": {\n",
    "        \"events\": [{\n",
    "            \"message\": {\n",
    "                \"forward_differences\": [\n",
    "                    {\n",
    "                        \"object\": \"_6C1FDA90-1F4E-4716-BC90-1CCB59A6D5A9\",\n",
    "                        \"attribute\": \"Switch.open\",\n",
    "                        \"value\": 1\n",
    "                    }\n",
    "                ],\n",
    "                \"reverse_differences\": [\n",
    "                    {\n",
    "                        \"object\": \"_6C1FDA90-1F4E-4716-BC90-1CCB59A6D5A9\",\n",
    "                        \"attribute\": \"Switch.open\",\n",
    "                        \"value\": 0\n",
    "                    }\n",
    "                ]\n",
    "            },\n",
    "            \"event_type\": \"ScheduledCommandEvent\",\n",
    "            \"occuredDateTime\": 1570041140,\n",
    "            \"stopDateTime\": 1570041200\n",
    "        }]\n",
    "    },\n",
    "     \"service_configs\": [{\n",
    "        \"id\": \"gridappsd-sensor-simulator\",\n",
    "        \"user_options\": {\n",
    "            \"sensors-config\": {\n",
    "                \"_99db0dc7-ccda-4ed5-a772-a7db362e9818\": {\n",
    "                    \"nominal-value\": 100,\n",
    "                    \"perunit-confidence-band\": 0.02,\n",
    "                    \"aggregation-interval\": 5,\n",
    "                    \"perunit-drop-rate\": 0.01\n",
    "                },\n",
    "                \"_ee65ee31-a900-4f98-bf57-e752be924c4d\": {},\n",
    "                \"_f2673c22-654b-452a-8297-45dae11b1e14\": {}\n",
    "            },\n",
    "            \"random-seed\": 0,\n",
    "            \"default-aggregation-interval\": 30,\n",
    "            \"passthrough-if-not-specified\": False,\n",
    "            \"default-perunit-confidence-band\": 0.01,\n",
    "            \"default-perunit-drop-rate\": 0.05\n",
    "        }\n",
    "    }]\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The simulation start message can be built from within the application or copied from the GridAPPS-D Viz by starting a new simulation and copying the start message from the log file. Pausing the simulation immediately after it starts is extremely helpful in locating the start message and copying it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![sim-start-message](images/3.5/sim_start_message.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Starting the Simulation \n",
    "\n",
    "To start the simulation, import the `gridappsd.simulation` library, which provides multiple shortcut functions for running and controlling a simulation. \n",
    "\n",
    "The simulation start message python dictionary created in the previous section is then passed as an argument to Simulation object. \n",
    "\n",
    "The `.start_simulation()` method is then used to pass the Simulation API call to the GOSS Message Bus and the GridAPPS-D Platform.\n",
    "\n",
    "The simulation id can be obtained by invoking the `.simulation_id` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from gridappsd.simulation import Simulation # Import Simulation Library\n",
    "\n",
    "simulation_obj = Simulation(gapps_sim, run_config_123) # Create Simulation object\n",
    "simulation_obj.start_simulation() # Start Simulation\n",
    "\n",
    "simulation_id = simulation_obj.simulation_id # Obtain Simulation ID\n",
    "print(\"Successfully started simulation with simulation_id: \", simulation_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The simulation start message can also be saved as a JSON file that is loaded using the `json.load(\"filename.json\")` method. The simulation is then started by passing the start message to the `.start_simulation()` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "from gridappsd.simulation import Simulation # Import Simulation Library\n",
    "\n",
    "run123_config = json.load(open(\"Run123NodeFileSimAPI.json\")) # Pull simulation start message from saved file\n",
    "    \n",
    "simulation_obj = Simulation(gapps, run123_config) # Create Simulation object\n",
    "simulation_obj.start_simulation() # Start Simulation\n",
    "\n",
    "simulation_id = simulation_obj.simulation_id # Obtain Simulation ID\n",
    "print(\"Successfully started simulation with simulation_id: \", simulation_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pausing, Resuming, or Stopping a Simulation\n",
    "\n",
    "Whether running a simulation through the GridAPPS-D Viz or a parallel digital twin simulation started from within your application, it is frequently useful to be able to pause, resume, or stop the simulation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using the gridappsd.simulation Python Library\n",
    "\n",
    "For simulations that are started using the `.start_simulation()` method of the `gridappsd.simulation` python library, it is possible to use the associated methods to pause, resume, and stop simulations. \n",
    "\n",
    "The library provides the following methods:\n",
    "\n",
    "* `.pause()` -- Pause the simulation\n",
    "\n",
    "\n",
    "* `.resume()` -- Resume the simulation\n",
    "\n",
    "\n",
    "* `.resume_pause_at(pause_time)` -- Resume the simulation, and then pause it in so many seconds\n",
    "\n",
    "\n",
    "* `.stop()` -- Stop the simulation\n",
    "\n",
    "\n",
    "* `.run_loop()` -- Loop the entire simulation until interrupted"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using a Topic + Message API Call\n",
    "\n",
    "If the simulation was started using the GridAPPS-D Viz, then pause, resume, and stop commands need to be issued using an API call to the Simulation API specifying the topic and message. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Specifying the Topic\n",
    "\n",
    "This is a dynamic `/topic/` communication channel that is best implemented by importing the GriAPPSD-Python library function for generating the correct topic. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "viz_simulation_id = \"paste sim_id here\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from gridappsd.topics import simulation_input_topic\n",
    "topic = simulation_input_topic(viz_simulation_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Pause Message\n",
    "\n",
    "The simulation can be paused using a very simple pause message. Since no response is expected from the Simulation API, we used the `.send(topic, message)` method for the GridAPPS-D connection object.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "message = {\"command\": \"pause\"}\n",
    "gapps.send(topic, message)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Resume Message\n",
    "\n",
    "The simulation can be resumed using a very simple message. Since no response is expected from the Simulation API, we used the `.send(topic, message)` method for the GridAPPS-D connection object.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "message = {\"command\": \"resume\"}\n",
    "gapps.send(topic, message)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Resume then Pause Message\n",
    "\n",
    "The simulation can be resumed and then paused using a very simple message. Since no response is expected from the Simulation API, we used the `.send(topic, message)` method for the GridAPPS-D connection object.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "message = {\n",
    "    \"command\": \"resumePauseAt\",\n",
    "    \"input\": {\"pauseIn\": 10}\n",
    "}\n",
    "gapps.send(topic, message)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![GridAPPS-D_narrow.png](../images/GridAPPS-D_narrow.png)"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Tags",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}