notebooks/NDEx Python Client 3.1 Tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# THIS TUTORIAL HAS BEEN DEPRECATED\n",
    "\n",
    "Please use the updated NDEx python client tutorial in the Jupyter notebook named:\n",
    "\n",
    "**NDEx2 Client v2.0 Tutorial.ipynb**\n",
    "\n",
    "which is located in the same directory as this notebook"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# NDEx Python Client v3.1 Tutorial -- DEPRECATED"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "In this tutorial you will learn to use the NDEx Python Client v3.1. The NDEx client is a module that simplifies access to the NDEx Server API and provides convenience methods for common operations on networks. \n",
    "\n",
    "This tutorial requires Python 2.7.9 and the ndex module. \n",
    "\n",
    "See [the NDEx Python Client](https://github.com/ndexbio/ndex-python) for installation instructions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Modules Required for this Tutorial"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import ndex.client as nc\n",
    "import io\n",
    "import json\n",
    "from IPython.display import HTML\n",
    "from time import sleep"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setting up NDEx Clients"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this section you will configure two client objects to access the public NDEx server. \n",
    "\n",
    "The first will enable you to make anonymous requests. \n",
    "\n",
    "The second will enable you to perform operations requiring authentication, such as saving networks to your account."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Anonymous Clients"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following code creates an NDEx client object to access the NDEx public server anonymously, then tests the client by getting the current server status."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "anon client: 16524 networks, 874 users, 90 groups\n"
     ]
    }
   ],
   "source": [
    "anon_ndex=nc.Ndex(\"http://public.ndexbio.org\")\n",
    "anon_ndex.update_status()\n",
    "networks = anon_ndex.status.get(\"networkCount\")\n",
    "users = anon_ndex.status.get(\"userCount\")\n",
    "groups = anon_ndex.status.get(\"groupCount\")\n",
    "print(\"anon client: %s networks, %s users, %s groups\" % (networks, users, groups))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Personal Clients"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A personal client enables you to perform operations requiring authentication, such as saving networks to your account.\n",
    "\n",
    "You must first create an account on the NDEx Public Server to create a personal client object."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following code creates an NDEx client object to access your account on the NDEx Public Server, then tests the client by getting the current server status.\n",
    "\n",
    "For convenience and clarity in this example, we use the variables ‘my_account’ and ‘my_password’ to hold the strings for your account name and password. Edit the values for these variables before running the following cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "my_ndex client: 16524 networks, 874 users, 90 groups\n"
     ]
    }
   ],
   "source": [
    "my_account=\"my_account\"\n",
    "my_password=\"my_password\"\n",
    "try:\n",
    "    my_ndex=nc.Ndex(\"http://public.ndexbio.org\", my_account, my_password)\n",
    "    my_ndex.update_status()\n",
    "    networks = my_ndex.status.get(\"networkCount\")\n",
    "    users = my_ndex.status.get(\"userCount\")\n",
    "    groups = my_ndex.status.get(\"groupCount\")\n",
    "    print(\"my_ndex client: %s networks, %s users, %s groups\" % (networks, users, groups))\n",
    "except Exception as inst:\n",
    "    print(\"Could not access account %s with password %s\" % (my_account, my_password))\n",
    "    print(inst.args)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Working with the NDEx Network Using the Anonymous Client"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get Network Information by Accession"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can access a network by its accession id, which is a universally unique identifier (UUID) assigned to the network by the NDEx server. All networks have a UUID and they are unique across all servers. No two networks will share an UUID.\n",
    "\n",
    "In this step, you will get basic information about the network and retrieve a Network Summary structure. The ‘Metabolism of RNA’ network is in the NDEx Tutorials account on the public NDEx server; its UUID is ‘9ed0cd55-9ac0-11e4-9499-000c29202374’\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "anon_ndex will access the network using the **get_network_summary(network_id)** which returns a NetworkSummary as  \n",
    "a dictionary:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><tr><td>properties</td><td><table><tr><td>ndex:sourceFormat</td><td>SIF</td></tr><tr><td>ORGANISM</td><td>http://identifiers.org/taxonomy/9606</td></tr><tr><td>URI</td><td>http://purl.org/pc2/4/Pathway_098d5b97ab0d39bfcf905771f06d18a5</td></tr><tr><td>Source</td><td>http://purl.org/pc2/4/reactome46_human</td></tr><tr><td>reference</td><td></td></tr></table></td></tr><tr><td>edgeCount</td><td>4344</td></tr><tr><td>name</td><td>Metabolism of RNA</td></tr><tr><td>warnings</td><td>ElementCount missing in Metadata of aspect edgeAttributes<br>Metadata element of aspect edgeAttributes is removed by NDEx because the element count is 0.<br>ElementCount missing in Metadata of aspect edgeCitations<br>ElementCount in Metadata of aspect edgeCitations is set to 591 by NDEx server.<br>ElementCount missing in Metadata of aspect networkAttributes<br>ElementCount in Metadata of aspect networkAttributes is set to 6 by NDEx server.<br>ElementCount missing in Metadata of aspect nodeAttributes<br>ElementCount in Metadata of aspect nodeAttributes is set to 688 by NDEx server.<br>ElementCount missing in Metadata of aspect nodeCitations<br>Metadata element of aspect nodeCitations is removed by NDEx because the element count is 0.<br>ElementCount missing in Metadata of aspect provenanceHistory<br></td></tr><tr><td>isReadOnly</td><td>False</td></tr><tr><td>isValid</td><td>True</td></tr><tr><td>errorMessage</td><td>None</td></tr><tr><td>creationTime</td><td>1421111281685</td></tr><tr><td>uri</td><td>None</td></tr><tr><td>visibility</td><td>PUBLIC</td></tr><tr><td>isShowcase</td><td>True</td></tr><tr><td>owner</td><td>ndextutorials</td></tr><tr><td>subnetworkIds</td><td>[]</td></tr><tr><td>externalId</td><td>9ed0cd55-9ac0-11e4-9499-000c29202374</td></tr><tr><td>version</td><td>Official</td></tr><tr><td>nodeCount</td><td>361</td></tr><tr><td>modificationTime</td><td>1489082848348</td></tr><tr><td>ownerUUID</td><td>5e63f1cb-9aaf-11e4-8424-000c29202374</td></tr><tr><td>isDeleted</td><td>False</td></tr><tr><td>description</td><td>*** This network is used for tutorial purpose only and will not be updated. ***</td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ns=anon_ndex.get_network_summary('9ed0cd55-9ac0-11e4-9499-000c29202374')\n",
    "def summary2table(object):\n",
    "    table = \"<table>\"\n",
    "    for key, value in object.items():\n",
    "        if key == \"warnings\":\n",
    "            warning_list = \"\"\n",
    "            for warning in value:\n",
    "                warning_list += \"%s<br>\" % warning\n",
    "            value = warning_list\n",
    "        if key == \"properties\":\n",
    "            property_table = \"<table>\"\n",
    "            for property in value:\n",
    "                property_table += \"<tr>\" \n",
    "                property_table += \"<td>%s</td><td>%s</td>\" % (property.get(\"predicateString\"), property.get(\"value\"))\n",
    "                property_table += \"</tr>\"\n",
    "            property_table += \"</table>\"\n",
    "            value = property_table\n",
    "                \n",
    "        table += \"<tr>\" \n",
    "        table += \"<td>%s</td><td>%s</td>\" % (key, value)\n",
    "        table += \"</tr>\"\n",
    "    table += \"</table>\"\n",
    "    return table\n",
    "\n",
    "HTML(summary2table(ns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Find Networks by Search – Simple Search"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "You can search for networks by the text in their name and description as well as the names and controlled vocabulary terms associated with their nodes. The input is a search string that conforms to Lucene search string syntax, but in its simplest form is one or more search terms separated by spaces.\n",
    "\n",
    "anon_ndex will perform the network search using the **search_networks(search_string=\"\", account_name=None, start=0, size=100, include_groups=False)** method that returns a list of NetworkSummary dictionaries:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3 networks found.\n"
     ]
    }
   ],
   "source": [
    "metabolic_networks=anon_ndex.search_networks('metabo*')\n",
    "print(\"%s networks found.\" % (len(metabolic_networks)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The search can also be limited to a specific account and to a number of search results:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2 networks found\n",
      "\n",
      "Networks:\n",
      "\n",
      "  Metabolism\n",
      "  Metabolism of proteins\n"
     ]
    }
   ],
   "source": [
    "metabolic_networks=anon_ndex.search_networks(search_string='metabo* owner:ndextutorials', size=2)\n",
    "print(\"%s networks found\" % (len(metabolic_networks.get('networks'))))\n",
    "print(\"\\nNetworks:\\n\")\n",
    "for ns in metabolic_networks.get('networks'): print(\"  %s\" % ns.get('name'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get a Network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "You can obtain an entire network as a CX stream, which is an NDEx format that is optimized for streaming networks. \n",
    "This is performed as a monolithic operation, so care should be taken when requesting very large networks. Applications\n",
    "can use the **get_network_summary** method to check the node and edge counts for a network before attempting to use \n",
    "**get_network_as_cx_stream**. The stream is contained in a Response object from the \n",
    "[Python requests library](http://docs.python-requests.org/en/master/).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "anon_ndex will use the **get_network_as_cx_stream(network_id)** method to get a Response object (from the [Python requests library](http://docs.python-requests.org/en/master/))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Received 665516 characters of CX\n"
     ]
    }
   ],
   "source": [
    "response=anon_ndex.get_network_as_cx_stream('9ed0cd55-9ac0-11e4-9499-000c29202374')\n",
    "print(\"Received %s characters of CX\" % len(response.content))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Query a Network – Neighborhood Query\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "You can retrieve a ‘neighborhood’ subnetwork of a network as a CX stream. The query finds the subnetwork by first \n",
    "identifying nodes that are associated with identifiers in the search_string, then traversing a specified number of \n",
    "edges starting from those nodes.  The **search_depth** parameter controls the search, defaults to 1 edge and can be no more than 3 edges. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The query to anon_ndex will use the **get_neighborhood(network_id, search_string, search_depth=1, edge_limit=2500)** method to get a CX object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Query result network contains 20 nodes and 26 edges.\n"
     ]
    }
   ],
   "source": [
    "query_result_cx=anon_ndex.get_neighborhood('9ed0cd55-9ac0-11e4-9499-000c29202374', 'XRN1')\n",
    "\n",
    "def getNumberOfNodesAndEdgesFromCX(cx):\n",
    "    numberOfEdges = numberOfNodes = 0;\n",
    "    for aspect in cx:\n",
    "        if 'metaData' in aspect:\n",
    "            metaData = aspect['metaData']\n",
    "            for element in metaData:\n",
    "                if (('name' in element) and (element['name'] == 'nodes')):\n",
    "                    numberOfNodes = element['elementCount']\n",
    "                if (('name' in element) and (element['name'] == 'edges')):\n",
    "                    numberOfEdges = element['elementCount']\n",
    "            break\n",
    "    return numberOfNodes, numberOfEdges\n",
    "\n",
    "\n",
    "nodes, edges = getNumberOfNodesAndEdgesFromCX(query_result_cx)\n",
    "\n",
    "print(\"Query result network contains %s nodes and %s edges.\" % (nodes, edges))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Working with the NDEx Network Using Your Personal Client"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a Network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "You can create a new network on an NDEx server if you have a CX stream. The network is created in the user account associated with the client object. All methods that create or modify content on the NDEx server require authentication, so you will use the my_ndex client object that you set up at the start of the tutorial and will create a network in your account."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the previous section, your neighborhood query retrieved a small network which was bound to the variable **query_result_cx**.\n",
    "\n",
    "We will now save this network to your account using **save_new_network(network)** and receive the URI for the new network. The URI includes the network UUID."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "URI of the newly created network 032caab6-ba9d-11e7-94d3-0ac135e8bacf is http://public.ndexbio.org/v2/network/032caab6-ba9d-11e7-94d3-0ac135e8bacf\n",
      "New network has been validated by the server.\n"
     ]
    }
   ],
   "source": [
    "uri = my_ndex.save_new_network(query_result_cx)\n",
    "uuid = uri.rpartition('/')[-1]\n",
    "print(\"URI of the newly created network %s is %s\" % (uuid, uri))\n",
    "for i in range(0, 4):\n",
    "    sleep(1)\n",
    "    new_summary = my_ndex.get_network_summary(uuid)\n",
    "    if new_summary.get(\"isValid\"):\n",
    "        print(\"New network has been validated by the server.\")\n",
    "        break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Update Network Profile"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the network UUID, you can update the name, description and version of the new network using the method **update_network_profile(network_id, network_profile)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "new name = Renamed Network\n",
      "new description = New Description\n",
      "new version = 2.0\n"
     ]
    }
   ],
   "source": [
    "#we use ID of the network we created at previous step\n",
    "network_profile={\"name\":\"Renamed Network\", \"description\":\"New Description\", \"version\":\"2.0\"}\n",
    "my_ndex.update_network_profile(uuid, network_profile)\n",
    "new_summary = my_ndex.get_network_summary(uuid)\n",
    "print(\"new name = %s\" % new_summary.get('name'))\n",
    "print(\"new description = %s\" % new_summary.get('description'))\n",
    "print(\"new version = %s\" % new_summary.get('version'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Set Read-Only"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The new network can be set to **read-only** using the **set_read_only(network_id, boolean)** method, preventing unintended modification. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The read only status is True\n"
     ]
    }
   ],
   "source": [
    "#make network read-only\n",
    "my_ndex.set_read_only(uuid, True)\n",
    "new_summary = my_ndex.get_network_summary(uuid)\n",
    "print(\"The read only status is %s\" % new_summary.get('isReadOnly'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The network is then reverted to read-write, enabling modification."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The read only status has been reverted to False\n"
     ]
    }
   ],
   "source": [
    "#revert network to original state (make it read-write again)\n",
    "my_ndex.set_read_only(uuid, False)\n",
    "new_summary = my_ndex.get_network_summary(uuid)\n",
    "print(\"The read only status has been reverted to %s\" % new_summary.get('isReadOnly'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Delete a Network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, the query result network is deleted using **delete_network(networkId)**.\n",
    "\n",
    "Note that you can only delete networks that you own.\n",
    "\n",
    "Be careful: there is no method to undo a deletion."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tutorial Complete\n"
     ]
    }
   ],
   "source": [
    "my_ndex.delete_network(uuid)\n",
    "print(\"Tutorial Complete\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}