module-content/docs/source/api_usage/3.2-API-Message-Structure.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# API Message Structure"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This section introduces the format used for passing messages to the GridAPPS-D API and how to wrap those messages using the GridAPPSD-Python Library."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Python Dictionaries VS JSON Strings\n",
    "\n",
    "One of the confusing aspects of passing messages to and from the GridAPPS-D Platform and APIs is the difference between Python Dictionaries and JSON scripts, which look identical.\n",
    "\n",
    "__JSON__ is a _serialization format_. That is, JSON is a way of representing structured data in the form of a textual string.\n",
    "\n",
    "A __Python Dictionary__ is a _data structure_. That is, it is a way of storing data in memory that provides certain abilities to the code: in the case of dictionaries, those abilities include rapid lookup and enumeration.\n",
    "\n",
    "It is possible to convert between the two by importing the JSON library: `import json`. Full documentation of JSON-Python interoperability and usage is available in [Python Docs](https://docs.python.org/3/library/json.html).\n",
    "\n",
    "Use the `json.dumps()` method to serialize a dictionary as a JSON string. Use the `json.loads()` to import a JSON file and convert it into a dictionary. But the two are not the same: dictionaries are for working with data in your program, and JSON is for storing it or sending it around between programs.\n",
    "\n",
    "With the GridAPPSD-Python Library, it is possible to pass query arguments as either a python dictionary or as a string. Both approaches will provide the same results."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__1) Format API call message as a dictionary__\n",
    "\n",
    "This is the most direct approach, and will be used most often throughout this set of notebook tutorials. The format and structure of the python dictionary is explained in the next section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_mrid = \"_49AD8E07-3BF9-A4E2-CB8F-C3722F837B62\" # IEEE 13 Node used for all example queries\n",
    "\n",
    "# Format message as python dictionary\n",
    "message = {\n",
    "    \"requestType\": \"QUERY_OBJECT_IDS\",\n",
    "    \"resultFormat\": \"JSON\",\n",
    "    \"modelId\": model_mrid,\n",
    "    \"objectType\": \"LoadBreakSwitch\"\n",
    "} "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Specify correct topic\n",
    "topic = \"goss.gridappsd.process.request.data.powergridmodel\"\n",
    "\n",
    "# Pass API Call to GridAPPS-D Platform\n",
    "gapps.get_response(topic, message)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__2) Format API call message as a string__\n",
    "\n",
    "This approach uses quotations (either `'  '` or `\"   \"`) to wrap the API call (identical to the python dictionary) as JSON-formatted text, concatenated into a string. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_mrid = \"_49AD8E07-3BF9-A4E2-CB8F-C3722F837B62\" # IEEE 13 Node used for all example queries\n",
    "\n",
    "# Format message as JSON text wrapped as a string\n",
    "message = \"\"\"\n",
    "{\n",
    "    \"requestType\": \"QUERY_OBJECT_IDS\",\n",
    "    \"resultFormat\": \"JSON\",\n",
    "    \"modelId\": \"%s\",\n",
    "    \"objectType\": \"LoadBreakSwitch\"\n",
    "} \n",
    "\"\"\" % model_mrid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Specify correct topic\n",
    "topic = \"goss.gridappsd.process.request.data.powergridmodel\"\n",
    "\n",
    "# Pass API Call to GridAPPS-D Platform\n",
    "gapps.get_response(topic, message)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Structure of a GridAPPS-D Message\n",
    "\n",
    "The structure of messages in GridAPPS-D follows that of a Python Dictionary using a data structure that is more generally known as an associative array. An excellent tutorial on advanced usage of the python dictionary structure is available on [Real Python](https://realpython.com/python-dicts/).\n",
    "\n",
    "A dictionary consists of a collection of __key-value pairs__. Each key-value pair maps the __key__ to its associated __value__.\n",
    "\n",
    "* A dictionary is defined by enclosing a comma-separated list of key-value pairs in curly braces ( __{ }__ ). \n",
    "\n",
    "* A colon ( __:__ ) separates each key from its associated value. \n",
    "\n",
    "* Square brackets ( __[ ]__ ) are used for a list of values associated to a particular key. \n",
    "\n",
    "* Additional curly braces ( __{ }__ ) can be used for cases where multiple key-value pairs (e.g. equipment setpoints) are associated with a particular key (e.g. an equipment class).\n",
    "\n",
    "\n",
    "The general dictionary format used for GridAPPS-D messages is\n",
    "\n",
    "```\n",
    "message = {\n",
    "    \"key1\": \"value1\",\n",
    "    \"key2\": [\"value21\", \"value22\"],\n",
    "    \"key3\": {\n",
    "        \"key31\": \"value31\",\n",
    "        \"key32\": \"value32\"\n",
    "        },\n",
    "      .\n",
    "      .\n",
    "      .\n",
    "    \"key\": \"value\"\n",
    "}\n",
    "\n",
    "```\n",
    "\n",
    "__Important__: Be sure to pay attention to placement of commas ( __,__ ) at the end of each line. Commas are placed at the end of each line _except_ the last line. Incorrect comma placement will result in a syntax exception. \n",
    "\n",
    "The particular set of key-value pairs for each GridAPPS-D API is covered in detail in Lessons 2.1 through 2.7."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Parsing Returned Data\n",
    "\n",
    "After passing an API call, the GridAPPS-D Platform returns a JSON string that is subsequently converted into a python dictionary by the GridAPPSD-Python Library. This section will outline how to parse the data returned. \n",
    "\n",
    "For this example, we are going to use a simple query from the PowerGrid Model API (covered in Lesson 2.2.) to obtain the details of a piece of equipment using its unique mRID (introduced in the next lesson)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_mrid = \"_49AD8E07-3BF9-A4E2-CB8F-C3722F837B62\" # IEEE 13 Node used for all example queries\n",
    "\n",
    "# Specify correct topic\n",
    "topic = \"goss.gridappsd.process.request.data.powergridmodel\"\n",
    "\n",
    "message = {\n",
    "        \"modelId\": model_mrid,\n",
    "        \"requestType\": \"QUERY_OBJECT_DICT\",\n",
    "        \"resultFormat\": \"JSON\",\n",
    "        \"objectType\": \"LinearShuntCompensator\",\n",
    "}\n",
    "\n",
    "# Pass API Call to GridAPPS-D Platform\n",
    "response = gapps.get_response(topic, message)\n",
    "\n",
    "import json\n",
    "with open(\"foo.txt\", 'w') as out:\n",
    "    out.write(json.dumps(response, indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The structure of the python dictionary returned by the API is three key-value pairs for the keys of \n",
    "\n",
    "* `'data'` -- this is the data you requested\n",
    "\n",
    "* `'responseComplete'` -- true or false\n",
    "\n",
    "* `'id'` -- unique id associated with the API response dictionary\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A typical API response will the structure below:\n",
    "\n",
    "```\n",
    "response = {\n",
    "    'data': [{'key1': 'value1',\n",
    "        'key2': ['value21', 'value22']},\n",
    "        {'key1': 'value1',\n",
    "         'key2': ['value21', 'value22']}],\n",
    "     'responseComplete': True,\n",
    "     'id': '12345678'\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first step is to filter the dictionary to just the data requested: `response['data']`. The result will be a list object.\n",
    "\n",
    "Note: _some_ API calls will also need to additional filters of `[results][bindings]`. The STOMP Client presented in the next section is very helpful for previewing the structure of the dictionary returned by GridAPPS-D."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = gapps.get_response(topic, message)\n",
    "response_obj = response['data']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As `response_obj` is of the python type `list` rather than `dict`, it is necessary to use numerical indices instead of keys to access the values. A simple `for` loop is very helpful here. \n",
    "\n",
    "In this example, we want to filter the results to create a list that contains just the name and mRID of the capacitor banks in the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "capacitors = []\n",
    "for index in response_obj:\n",
    "    cap_name = index['IdentifiedObject.name']\n",
    "    cap_mrid = index['id']\n",
    "    message = dict(name = cap_name,\n",
    "                   mrid = cap_mrid)\n",
    "    capacitors.append(message)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[[Return to Top](#Table-of-Contents)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using the STOMP Client\n",
    "\n",
    "The GridAPPS-D Visualization App includes a feature to pass API call messages through the GUI using the Simple Text Oriented Messaging Protocol (STOMP).\n",
    "\n",
    "Open the Viz App, which is hosted on [localhost:8080](http://localhost:8080/) (note: cloud-hosted installations will use the IP address of the server). "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Select __Stomp Client__ from the main drop-down menu:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![menus](images/3.2/menus.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![select-stomp](images/3.2/menu_select_stomp.png)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This opens the STOMP Client, which can be used to pass a message to any of the GridAPPS-D APIs to preview results or debug the API call message."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![stomp-client](images/3.2/STOMP_client.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Specifying the Topic\n",
    "\n",
    "The appropriate GridAPPS-D topic  needs to be copied and pasted into the __Destination Topic__ box at the top of the window. The topic specifies on which channel the STOMP Client will communicate with the GridAPPS-D Platform and to which API the message needs to be delivered. \n",
    "\n",
    "A complete list of GridAPPS-D topics was provided in [API Communication Channels](../api_usage/3.1--Lesson-3.1--API-Communication-Channels.ipynb) and will also be provided in context for each of the API calls detailed in subsequent lessons. \n",
    "\n",
    "__IMPORTANT:__ Remember to remove the python wrapping quotations at the beginning and end of the topic. For example, if the python-wrapped topic was\n",
    "\n",
    "`topic = \"goss.gridappsd.process.request.data.powergridmodel\" # Specify the topic`\n",
    "\n",
    "then the topic that is entered in the Stomp Client __Destination Topic__ box is simply\n",
    "\n",
    "`goss.gridappsd.process.request.data.powergridmodel`\n",
    "\n",
    "__IMPORTANT:__ The GridAPPSD-Python shortcut functions will not work in the STOMP Client. The full text string versions must be used. \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Entering the Request Message\n",
    "\n",
    "The __Request__ box accepts an API call message identical to those provided in these notebook lessons.\n",
    "\n",
    "__IMPORTANT:__ Remember to remove the python wrapping at the beginning and end of the message. For example, if the python-wrapped message was\n",
    "\n",
    "`message = \"{\"requestType\": \"QUERY_MODEL_NAMES\", \"resultFormat\": \"JSON\"}\" # Sample PowerGrid Model API Call`\n",
    "\n",
    "then the message that is entered in the Stomp Client __Request__ box is simply\n",
    "\n",
    "`{\"requestType\": \"QUERY_MODEL_NAMES\", \"resultFormat\": \"JSON\"}`\n",
    "\n",
    "The STOMP client will automatically flag any errors in the JSON message."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Submitting a Request\n",
    "\n",
    "After entering the topic and message, click __Send request__ to send the API call to the GridAPPS-D Platform. The response will be displayed in the box below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![stomp-output](images/3.2/STOMP_output.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It can be seen that the response from the STOMP Client is identical to that obtained by passing the same topic and message using the GridAPPSD-Python Library:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from gridappsd import GridAPPSD # Import Libraries\n",
    "gapps = GridAPPSD(\"('localhost', 61613)\", username='system', password='manager') # Connect to Platform\n",
    "topic = \"goss.gridappsd.process.request.data.powergridmodel\" # Specify correct Topic\n",
    "message = {\n",
    "    \"requestType\": \"QUERY_MODEL_NAMES\",\n",
    "    \"resultFormat\": \"JSON\"\n",
    "} # Sample PowerGrid Model API message\n",
    "gapps.get_response(topic, message) # Pass API call to Platform"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![GridAPPS-D_narrow.png](../images/GridAPPS-D_narrow.png)"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Tags",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}