module-2/week-2/day-4-nosql-mongodb/nosql-enkeboll.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1 style='text-align:center'>NoSQL - Not Only SQL</h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### What's wrong with SQL? "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- SQL offers a ton of structure for storing data \n",
    "    - That structure requires data to come in, in a certain way (aka your data must have structure) \n",
    "    - Structure comes at the cost of speed \n",
    "    \n",
    "    \n",
    "- SQL structure is very rigid - if you want to change the schema it requires you to change all of your existing data to match the new schema \n",
    "\n",
    "\n",
    "- Large data requires distributed computing (many computers working together to accomplish the same task) - Executing distributed joins is a very complex problem in relational databases. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### What does NoSQL offer? "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Schemaless − Number of fields, content and size of the data object can differ from one data object to another.\n",
    "- You can store virtually any kind of data. \n",
    "- Structure of a single object is clear.\n",
    "- No complex joins.\n",
    "- To scale up and handle more queries, just add more machines\n",
    "- You can change the schema of your database on the fly"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Types of NoSQL Databases"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img style='width: 400px' src='images/nosql-types.png/'>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Document databases</b> pair each key with a complex data structure known as a document. Documents can contain many different key-value pairs, or key-array pairs, or even nested documents."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img  style='align: center; width:150px' src='images/mongodb.png' />\n",
    "<img style='align: center;' src='images/couchdb.png' />\n",
    "<img style='align: center; width: 200px' src='images/documentdb.png' />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Graph stores</b> are used to store information about networks of data, such as social connections. Graph stores include Neo4J and Giraph."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img  style='align: center; width:150px' src='images/ApacheGiraph.svg' />\n",
    "<img style='align: center;' src='images/neo4j.png' />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Key-value</b> stores are the simplest NoSQL databases. Every single item in the database is stored as an attribute name (or 'key'), together with its value. Examples of key-value stores are Riak and Berkeley DB. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Wide-column stores</b> such as Cassandra and HBase are optimized for queries over large datasets, and store columns of data together, instead of rows."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What is MongoDB"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "MongoDB stores data in flexible, JSON-like documents, meaning fields can vary from document to document and data structure can be changed over time"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Data Structure</b>\n",
    "\n",
    "Single Entry = Document"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```json\n",
    "{ \n",
    "  _id: ObjectId(8af37bd7891c), \n",
    "  title: 'MongoDB Lab',\n",
    "  description: 'Introductory lab on how to use MongoDB',\n",
    "  by: 'Flatiron School',\n",
    "  topics: ['mongodb', 'database', 'NoSQL', 'JSON']  \n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can embed documents inside documents! "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src ='images/househouse.gif' />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```json\n",
    "{ \n",
    "  _id: ObjectId(8af37bd78ssc), \n",
    "  title: 'Other Lab',\n",
    "  description: 'Introductory lab on how to use something',\n",
    "  by: 'Flatiron School',\n",
    "  topics: ['blah', 'blah', 'blah', 'blah'],\n",
    "  author: {\n",
    "            _id: ObjectId(83928shkjw183),\n",
    "            name: 'Andy Enkeboll',\n",
    "            building: 'Metropolitan Square'\n",
    "          }\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Why would we want to nest objects? "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Multiple Documents = Collection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```json\n",
    "{ \n",
    "  _id: ObjectId(8af37bd7891c), \n",
    "  title: 'MongoDB Lab',\n",
    "  description: 'Introductory lab on how to use MongoDB',\n",
    "  by: 'Flatiron School',\n",
    "  topics: ['mongodb', 'database', 'NoSQL', 'JSON']  \n",
    "}, \n",
    "{ \n",
    "  _id: ObjectId(8af37bd78ssc), \n",
    "  title: 'Other Lab',\n",
    "  description: 'Introductory lab on how to use something',\n",
    "  by: 'Flatiron School',\n",
    "  topics: ['blah', 'blah', 'blah', 'blah']  \n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Working with MongoDB"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Assuming you have installed/setup mongo and pip installed pymongo..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install pymongo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pymongo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "myclient = pymongo.MongoClient(\"mongodb://127.0.0.1:27017/\")\n",
    "\n",
    "# grab a database from your server \n",
    "db = myclient['friday']\n",
    "\n",
    "#t his can be a new one or an existing one\n",
    "# (if it doesn't exist, it will get create when you write data into it)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['admin', 'amazon', 'config', 'example_data', 'friday', 'local', 'thursday']"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "myclient.list_database_names()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# initialize an empty collection - this where your 'documents' will go\n",
    "coll = db['new_collection']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<pymongo.results.InsertOneResult at 0x10bb45ec8>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "example_data = {'name': 'John Doe', 'address': '123 elm street',\n",
    "                'age': 28, 'children': ['Jane', 'Joe']}\n",
    "coll.insert_one(example_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ObjectId('5d83a966ceb690210acf74c0')"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "_.inserted_id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "#get all the documents in a collection\n",
    "query = coll.find({})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'_id': ObjectId('5d83a8cfceb690210acf74be'), 'name': 'John Doe', 'address': '123 elm street', 'age': 28, 'children': ['Jane', 'Joe']}\n",
      "{'_id': ObjectId('5d83a966ceb690210acf74c0'), 'name': 'John Doe', 'address': '123 elm street', 'age': 28, 'children': ['Jane', 'Joe']}\n"
     ]
    }
   ],
   "source": [
    "for document in query:\n",
    "    print(document)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<pymongo.results.InsertManyResult at 0x10bf32988>"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "example_data_2 = [{'name': 'Andy', 'address': 'DC'},\n",
    "                  {'name': 'Murat', 'address': 'MD'},\n",
    "                  {'name': 'Josh'}\n",
    "                  ]\n",
    "coll.insert_many(example_data_2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[ObjectId('5d83a9d0ceb690210acf74c1'),\n",
       " ObjectId('5d83a9d0ceb690210acf74c2'),\n",
       " ObjectId('5d83a9d0ceb690210acf74c3')]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "_.inserted_ids"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "query_1 = coll.find({})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'_id': ObjectId('5d83a8cfceb690210acf74be'), 'name': 'John Doe', 'address': '123 elm street', 'age': 28, 'children': ['Jane', 'Joe']}\n",
      "{'_id': ObjectId('5d83a966ceb690210acf74c0'), 'name': 'John Doe', 'address': '123 elm street', 'age': 28, 'children': ['Jane', 'Joe']}\n",
      "{'_id': ObjectId('5d83a9d0ceb690210acf74c1'), 'name': 'Andy', 'address': 'DC'}\n",
      "{'_id': ObjectId('5d83a9d0ceb690210acf74c2'), 'name': 'Murat', 'address': 'MD'}\n",
      "{'_id': ObjectId('5d83a9d0ceb690210acf74c3'), 'name': 'Josh'}\n"
     ]
    }
   ],
   "source": [
    "for document in query_1:\n",
    "    print(document)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "query_2 = coll.find({'name': 'Murat'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'_id': ObjectId('5d83a9d0ceb690210acf74c2'), 'name': 'Murat', 'address': 'MD'}\n"
     ]
    }
   ],
   "source": [
    "for document in query_2:\n",
    "    print(document)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<pymongo.results.UpdateResult at 0x10bf371c8>"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#updating records is super easy! \n",
    "record_to_update = {'name' : 'John Doe'}\n",
    "update_1 = {'$set': {'age': 29, 'birthday': '2/8/1990'}}\n",
    "\n",
    "coll.update_many(record_to_update, update_1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#searching in a list in a document\n",
    "query_4 = mycollection.find({'children': 'Jane'})\n",
    "for item in query_4:\n",
    "    print(item)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#removing a key:value from a document\n",
    "update_2 = {'$unset': {'brithday': ''}}\n",
    "\n",
    "coll.update_many(record_to_update, update_2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "query_5 = coll.find({'name': 'John Doe'})\n",
    "for item in query_5:\n",
    "    print(item)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<pymongo.results.DeleteResult at 0x10bf37e08>"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#delete record\n",
    "coll.delete_one({'name' : 'John Doe'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<pymongo.results.DeleteResult at 0x10bf29c88>"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#delete all records\n",
    "coll.delete_many({})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Working with Images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "resp = requests.get('https://www.dictionary.com/e/wp-content/uploads/2018/04/mongo.jpg')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "img = resp.content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "bytes"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(img)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<pymongo.results.InsertOneResult at 0x10c270c88>"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "newdict = {'a': 4, 'image': img}\n",
    "coll.insert_one(newdict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ObjectId('5d83acd9ceb690210acf74c4')"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "_.inserted_id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import Image\n",
    "\n",
    "results = coll.find_one({'_id': _})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results.get('b', [])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [],
   "source": [
    "newimg = results.get('image')\n",
    "\n",
    "# len(newimg)\n",
    "# Image(newimg)\n",
    "with open('/Users/enkeboll/Downloads/mongo.jpg', 'wb') as f:\n",
    "    f.write(newimg)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}