doc/source/notebooks/08_nodesets.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Node sets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preamble\n",
    "The code in this section is identical to the code in sections \"Introduction\" and \"Loading\" from the previous tutorial. It assumes that you have already downloaded the circuit. If not, take a look to the notebook **01_circuits** (Downloading a circuit)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import bluepysnap\n",
    "import pandas as pd\n",
    "\n",
    "circuit_path = \"sonata/circuit_sonata.json\"\n",
    "circuit = bluepysnap.Circuit(circuit_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "As briefly mentioned in [node properties notebook](./03_node_properties.ipynb), node set is a predetermined collection of queries for nodes. They are saved in a JSON file which is usually added into the circuit and/or simulation config. For a more in-depth explanation, please see: [SONATA Node Sets - Circuit Documentation](https://sonata-extension.readthedocs.io/en/latest/sonata_nodeset.html).\n",
    "\n",
    "We can directly access node sets in snap if it's added to the circuit config:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Mosaic': ['All'],\n",
       " 'All': ['thalamus_neurons'],\n",
       " 'thalamus_neurons': {'population': 'thalamus_neurons'},\n",
       " 'Excitatory': {'synapse_class': 'EXC'},\n",
       " 'Inhibitory': {'synapse_class': 'INH'},\n",
       " 'Rt_RC': {'mtype': 'Rt_RC'},\n",
       " 'VPL_IN': {'mtype': 'VPL_IN'},\n",
       " 'VPL_TC': {'mtype': 'VPL_TC'},\n",
       " 'bAC_IN': {'etype': 'bAC_IN'},\n",
       " 'cAD_noscltb': {'etype': 'cAD_noscltb'},\n",
       " 'cNAD_noscltb': {'etype': 'cNAD_noscltb'},\n",
       " 'dAD_ltb': {'etype': 'dAD_ltb'},\n",
       " 'dNAD_ltb': {'etype': 'dNAD_ltb'},\n",
       " 'mc0;Rt': {'region': 'mc0;Rt'},\n",
       " 'mc0;VPL': {'region': 'mc0;VPL'},\n",
       " 'mc1;Rt': {'region': 'mc1;Rt'},\n",
       " 'mc1;VPL': {'region': 'mc1;VPL'},\n",
       " 'mc2;Rt': {'region': 'mc2;Rt'},\n",
       " 'mc2;VPL': {'region': 'mc2;VPL'},\n",
       " 'mc3;Rt': {'region': 'mc3;Rt'},\n",
       " 'mc3;VPL': {'region': 'mc3;VPL'},\n",
       " 'mc4;Rt': {'region': 'mc4;Rt'},\n",
       " 'mc4;VPL': {'region': 'mc4;VPL'},\n",
       " 'mc5;Rt': {'region': 'mc5;Rt'},\n",
       " 'mc5;VPL': {'region': 'mc5;VPL'},\n",
       " 'mc6;Rt': {'region': 'mc6;Rt'},\n",
       " 'mc6;VPL': {'region': 'mc6;VPL'},\n",
       " 'IN': {'mtype': {'$regex': '.*IN'}, 'region': {'$regex': '.*'}},\n",
       " 'TC': {'mtype': {'$regex': '.*TC'}, 'region': {'$regex': '.*'}},\n",
       " 'mc0_Column': {'region': {'$regex': 'mc0.*'}},\n",
       " 'mc1_Column': {'region': {'$regex': 'mc1.*'}},\n",
       " 'mc2_Column': {'region': {'$regex': 'mc2.*'}},\n",
       " 'mc3_Column': {'region': {'$regex': 'mc3.*'}},\n",
       " 'mc4_Column': {'region': {'$regex': 'mc4.*'}},\n",
       " 'mc5_Column': {'region': {'$regex': 'mc5.*'}},\n",
       " 'mc6_Column': {'region': {'$regex': 'mc6.*'}},\n",
       " 'gapjunctions': ['thalamus_neurons'],\n",
       " 'gap-junctions': ['gapjunctions'],\n",
       " 'ML_afferents': {'population': 'MedialLemniscus_projections'},\n",
       " 'CT_afferents': {'population': 'CorticoThalamic_projections'}}"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "circuit.node_sets.content"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To prove a point, let's query some ids using a node set, and compare it with querying with a similar query:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ids found: 64856\n",
      "Queries result in the same outcome : True\n"
     ]
    }
   ],
   "source": [
    "node_set_result = circuit.nodes.ids('VPL_TC')\n",
    "print(f'Ids found: {len(node_set_result)}')\n",
    "\n",
    "query_result = circuit.nodes.ids({'mtype': 'VPL_TC'})\n",
    "print(f'Queries result in the same outcome : {node_set_result == query_result}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll go more into querying later at this document. For now, let's go over other aspects of node sets.\n",
    "\n",
    "## Features / usecases\n",
    "Sometimes, we may want to work with node sets that aren't found in a circuit or simulation config. This can be due to\n",
    "* experimenting\n",
    "* can't / don't want to modify the config file\n",
    "* combining node sets\n",
    "* etc.\n",
    "\n",
    "First of all, let's see how we can open / create node sets.\n",
    "\n",
    "### Opening a node set file\n",
    "For demonstration purposes, let's open the circuit's node sets from a file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Contents match: True\n"
     ]
    }
   ],
   "source": [
    "node_sets_circuit = bluepysnap.node_sets.NodeSets.from_file('./sonata/networks/nodes/node_sets.json')\n",
    "print(f\"Contents match: {node_sets_circuit.content == circuit.node_sets.content}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating node sets on the fly\n",
    "\n",
    "If we want to, for example, test node sets without having to write them to a file and load that over and over again. We can create node sets directly from a dict:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'nodes_0-9': {'node_id': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]}}"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "node_sets_0_9 = bluepysnap.node_sets.NodeSets.from_dict({'nodes_0-9': {'node_id': [*range(10)]}})\n",
    "node_sets_0_9.content"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This may be handy if you can't modify the existing nodesets file, but want to augment it with nodesets.\n",
    "\n",
    "### Combining node sets\n",
    "So now that we have two nodesets, `node_sets_circuit` read from a file and `node_sets_0_9` created from a dict, let's combine them. Naturally, we could also open two node sets from files and combine them.\n",
    "\n",
    "For this purpose, node sets objects have an `NodeSets.update()` method. `update` takes another node sets object as an argument, and adds all its node sets to itself. \n",
    "\n",
    "Let's update the `node_sets_circuit` with `node_sets_0_9`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Mosaic': ['All'],\n",
       " 'All': ['thalamus_neurons'],\n",
       " 'thalamus_neurons': {'population': 'thalamus_neurons'},\n",
       " 'Excitatory': {'synapse_class': 'EXC'},\n",
       " 'Inhibitory': {'synapse_class': 'INH'},\n",
       " 'Rt_RC': {'mtype': 'Rt_RC'},\n",
       " 'VPL_IN': {'mtype': 'VPL_IN'},\n",
       " 'VPL_TC': {'mtype': 'VPL_TC'},\n",
       " 'bAC_IN': {'etype': 'bAC_IN'},\n",
       " 'cAD_noscltb': {'etype': 'cAD_noscltb'},\n",
       " 'cNAD_noscltb': {'etype': 'cNAD_noscltb'},\n",
       " 'dAD_ltb': {'etype': 'dAD_ltb'},\n",
       " 'dNAD_ltb': {'etype': 'dNAD_ltb'},\n",
       " 'mc0;Rt': {'region': 'mc0;Rt'},\n",
       " 'mc0;VPL': {'region': 'mc0;VPL'},\n",
       " 'mc1;Rt': {'region': 'mc1;Rt'},\n",
       " 'mc1;VPL': {'region': 'mc1;VPL'},\n",
       " 'mc2;Rt': {'region': 'mc2;Rt'},\n",
       " 'mc2;VPL': {'region': 'mc2;VPL'},\n",
       " 'mc3;Rt': {'region': 'mc3;Rt'},\n",
       " 'mc3;VPL': {'region': 'mc3;VPL'},\n",
       " 'mc4;Rt': {'region': 'mc4;Rt'},\n",
       " 'mc4;VPL': {'region': 'mc4;VPL'},\n",
       " 'mc5;Rt': {'region': 'mc5;Rt'},\n",
       " 'mc5;VPL': {'region': 'mc5;VPL'},\n",
       " 'mc6;Rt': {'region': 'mc6;Rt'},\n",
       " 'mc6;VPL': {'region': 'mc6;VPL'},\n",
       " 'IN': {'mtype': {'$regex': '.*IN'}, 'region': {'$regex': '.*'}},\n",
       " 'TC': {'mtype': {'$regex': '.*TC'}, 'region': {'$regex': '.*'}},\n",
       " 'mc0_Column': {'region': {'$regex': 'mc0.*'}},\n",
       " 'mc1_Column': {'region': {'$regex': 'mc1.*'}},\n",
       " 'mc2_Column': {'region': {'$regex': 'mc2.*'}},\n",
       " 'mc3_Column': {'region': {'$regex': 'mc3.*'}},\n",
       " 'mc4_Column': {'region': {'$regex': 'mc4.*'}},\n",
       " 'mc5_Column': {'region': {'$regex': 'mc5.*'}},\n",
       " 'mc6_Column': {'region': {'$regex': 'mc6.*'}},\n",
       " 'gapjunctions': ['thalamus_neurons'],\n",
       " 'gap-junctions': ['gapjunctions'],\n",
       " 'ML_afferents': {'population': 'MedialLemniscus_projections'},\n",
       " 'CT_afferents': {'population': 'CorticoThalamic_projections'},\n",
       " 'nodes_0-9': {'node_id': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]}}"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "node_sets_circuit.update(node_sets_0_9)\n",
    "node_sets_circuit.content"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "as we can see, the node sets object contains the newly created node set `nodes_0-9`.\n",
    "\n",
    "**WARNING:** if the node sets object already contains node sets with same names as in the update, those node sets will be overwritten. The names of the overwritten node sets are returned in the `update` function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwritten node sets: {'nodes_0-9'}\n",
      "content[\"nodes_0-9\"]: {'node_id': [1]}\n"
     ]
    }
   ],
   "source": [
    "# Let's overwrite 'nodes_0-9'\n",
    "fake_0_9_node_set = bluepysnap.node_sets.NodeSets.from_dict({'nodes_0-9': {'node_id': [1]}})\n",
    "overwritten = node_sets_circuit.update(fake_0_9_node_set)\n",
    "print(f'Overwritten node sets: {overwritten}')\n",
    "print(f'content[\"nodes_0-9\"]: {node_sets_circuit.content[\"nodes_0-9\"]}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Compound node sets\n",
    "Compound node sets are literally node sets that are composed of other node sets. Let's create node sets with one compound node set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'nodes_0-4': {'node_id': [0, 1, 2, 3, 4]},\n",
       " 'nodes_5-9': {'node_id': [5, 6, 7, 8, 9]},\n",
       " 'nodes_0-9': ['nodes_0-4', 'nodes_5-9']}"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "node_sets_compound = bluepysnap.node_sets.NodeSets.from_dict({\n",
    "    'nodes_0-4': {'node_id': [*range(5)]},\n",
    "    'nodes_5-9': {'node_id': [*range(5,10)]},\n",
    "    'nodes_0-9': ['nodes_0-4', 'nodes_5-9'], # Compound node set with node set names in a list results in OR case\n",
    "})\n",
    "node_sets_compound.content"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NOTE:** Compound node sets always represent \"OR\" instead of \"AND\". I.e., the queries return results belonging to any of the node sets listed in a compound node set.\n",
    "\n",
    "### Referring to a node set in a `NodeSets` object\n",
    "`NodeSets` object works kind of like a `dict` in the sense that if you wish to refer to a specific node set, the syntax is the same as in `dict`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<bluepysnap.node_sets.NodeSet at 0x7fffa4493580>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "node_sets_circuit['VPL_TC']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Above, we got a `NodeSet` (not `NodeSets`!) object, i.e., one instance of a node set. For our purposes, we don't really have to know what it is, as long as we know how to access it. This will become handy in querying."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "In this notebook we took a deeper look into node sets. In the next notebook of the series, we learn how to query nodes in SNAP with and without node sets."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}