Projects/step_1/C1/Guided Project_Profitable App Profiles for the App Store and Google Play Markets.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Profitable App Profiles for the App Store and Google Play Markets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our aim in this project is to find mobile app profiles that are profitable for the App Store and Google Play markets. We're working as data analysts for a company that builds Android and iOS mobile apps, and our job is to enable our team of developers to make data-driven decisions with respect to the kind of apps they build.\n",
    "\n",
    "At our company, we only build apps that are free to download and install, and our main source of revenue consists of in-app ads. This means that our revenue for any given app is mostly influenced by the number of users that use our app. Our goal for this project is to analyze data to help our developers understand what kinds of apps are likely to attract more users."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Opening and Exploring the Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " To avoid spending resources on collecting new data ourselves, we should first try to see if we can find any relevant existing data at no cost. Luckily, these are two data sets that seem suitable for our goals:\n",
    "\n",
    "A data set containing data about approximately 10,000 Android apps from Google Play; the data was collected in August 2018. You can download the data set directly from [this link](https://www.kaggle.com/lava18/google-play-store-apps).\n",
    "A data set containing data about approximately 7,000 iOS apps from the App Store; the data was collected in July 2017. You can download the data set directly from [this link](https://www.kaggle.com/ramamet4/app-store-apple-data-set-10k-apps)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Read files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from csv import reader\n",
    "\n",
    "# Read and saved as list of list data\n",
    "apple_data=list(reader(open('AppleStore.csv')))\n",
    "\n",
    "#Remove header row from apple dataset\n",
    "apple_header=apple_data[:1]\n",
    "\n",
    "# Saved rest of data removing header row from apple dataset\n",
    "apple=apple_data[1:]\n",
    "\n",
    "#Read and saved as list of list data\n",
    "google_data=list(reader(open('googleplaystore.csv')))\n",
    "\n",
    "# Remove header row from google dataset\n",
    "google_header=google_data[:1]\n",
    "\n",
    "#Saved rest of data removing header row from google dataset\n",
    "google=google_data[1:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### create explore_dataset() function to explore both datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def explore_dataset(data,start_index,end_index,rows_and_columns=False):\n",
    "    dataset_slice=data[start_index:end_index]\n",
    "    print(data[:6])\n",
    "    if  rows_and_columns:\n",
    "        print('no of rows %s'%(len(data)))\n",
    "        print('no of columns %s'%(len(data[0])))        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[['284882215', 'Facebook', '389879808', 'USD', '0.0', '2974676', '212', '3.5', '3.5', '95.0', '4+', 'Social Networking', '37', '1', '29', '1'], ['389801252', 'Instagram', '113954816', 'USD', '0.0', '2161558', '1289', '4.5', '4.0', '10.23', '12+', 'Photo & Video', '37', '0', '29', '1'], ['529479190', 'Clash of Clans', '116476928', 'USD', '0.0', '2130805', '579', '4.5', '4.5', '9.24.12', '9+', 'Games', '38', '5', '18', '1'], ['420009108', 'Temple Run', '65921024', 'USD', '0.0', '1724546', '3842', '4.5', '4.0', '1.6.2', '9+', 'Games', '40', '5', '1', '1'], ['284035177', 'Pandora - Music & Radio', '130242560', 'USD', '0.0', '1126879', '3594', '4.0', '4.5', '8.4.1', '12+', 'Music', '37', '4', '1', '1'], ['429047995', 'Pinterest', '74778624', 'USD', '0.0', '1061624', '1814', '4.5', '4.0', '6.26', '12+', 'Social Networking', '37', '5', '27', '1']]\n",
      "no of rows 7197\n",
      "no of columns 16\n"
     ]
    }
   ],
   "source": [
    "# explore apple dataset\n",
    "\n",
    "explore_dataset(apple,1,6,True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[['Photo Editor & Candy Camera & Grid & ScrapBook', 'ART_AND_DESIGN', '4.1', '159', '19M', '10,000+', 'Free', '0', 'Everyone', 'Art & Design', 'January 7, 2018', '1.0.0', '4.0.3 and up'], ['Coloring book moana', 'ART_AND_DESIGN', '3.9', '967', '14M', '500,000+', 'Free', '0', 'Everyone', 'Art & Design;Pretend Play', 'January 15, 2018', '2.0.0', '4.0.3 and up'], ['U Launcher Lite – FREE Live Cool Themes, Hide Apps', 'ART_AND_DESIGN', '4.7', '87510', '8.7M', '5,000,000+', 'Free', '0', 'Everyone', 'Art & Design', 'August 1, 2018', '1.2.4', '4.0.3 and up'], ['Sketch - Draw & Paint', 'ART_AND_DESIGN', '4.5', '215644', '25M', '50,000,000+', 'Free', '0', 'Teen', 'Art & Design', 'June 8, 2018', 'Varies with device', '4.2 and up'], ['Pixel Draw - Number Art Coloring Book', 'ART_AND_DESIGN', '4.3', '967', '2.8M', '100,000+', 'Free', '0', 'Everyone', 'Art & Design;Creativity', 'June 20, 2018', '1.1', '4.4 and up'], ['Paper flowers instructions', 'ART_AND_DESIGN', '4.4', '167', '5.6M', '50,000+', 'Free', '0', 'Everyone', 'Art & Design', 'March 26, 2017', '1.0', '2.3 and up']]\n",
      "no of rows 10841\n",
      "no of columns 13\n"
     ]
    }
   ],
   "source": [
    "#explore google app dataset\n",
    "\n",
    "explore_dataset(google,1,6,True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "####  From above data exploration tried to identify columns which can be helpful in analysis\n",
    "* In apple dataset 'price','user_rating','prime_gener','size_bytes' columns are very important \n",
    "* In google dataset 'type','price','Geners','Reviews','Installs','Size' columns are important"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deleting Wrong Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Detect inaccurate data, and correct or remove it.\n",
    "* Detect duplicate data, and remove the duplicates."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In  my Company we only build apps that are free to download and install, and that are directed toward an English-speaking audience.\n",
    "SO,\n",
    "* Remove apps which aren't free\n",
    "* Remove non-Englisg language apps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Google Play data set has a [dedicated discussion](https://www.kaggle.com/ramamet4/app-store-apple-data-set-10k-apps/discussion) section, and we can see that one of the discussions outlines an error for row 10472. Let's print this row and compare it against the header and another row that is correct."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Life Made WI-Fi Touchscreen Photo Frame', '1.9', '19', '3.0M', '1,000+', 'Free', '0', 'Everyone', '', 'February 11, 2018', '1.0.19', '4.0 and up']\n",
      "\n",
      "\n",
      "[['App', 'Category', 'Rating', 'Reviews', 'Size', 'Installs', 'Type', 'Price', 'Content Rating', 'Genres', 'Last Updated', 'Current Ver', 'Android Ver']]\n",
      "['FI CFL', 'FINANCE', '3.7', '112', '3.9M', '10,000+', 'Free', '0', 'Everyone', 'Finance', 'July 5, 2018', '1.1.1', '5.0 and up']\n"
     ]
    }
   ],
   "source": [
    "print(google[10472])  #incorrect row\n",
    "print('\\n')\n",
    "print(google_header) # google dataset header row\n",
    "print(google[10467])       # correct row"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In 10472 row rating is 19 which is out of range, so delete this row"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10841\n",
      "10840\n"
     ]
    }
   ],
   "source": [
    "print(len(google)) # before deleting row\n",
    "del google[10472]\n",
    "print(len(google)) #after deleting row"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Removing Duplicate Entries:Part one"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Count total duplicate and unique entries of apps in google data set.\n",
    "Apple dataset have no duplicate entries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "No of duplicate entries in google dataset 1181\n",
      "No of unique entries in google daataset 9659\n",
      "No of duplicate entries in apple dataset 0\n",
      "No of unique entries in apple daataset 7197\n"
     ]
    }
   ],
   "source": [
    "# Google dataset\n",
    "\n",
    "duplicate_apps=[]\n",
    "unique_apps=[]\n",
    "for app in google:\n",
    "    app_name=app[0]\n",
    "    if app_name in unique_apps:\n",
    "        duplicate_apps.append(app_name)\n",
    "    else:\n",
    "        unique_apps.append(app_name)\n",
    "print('No of duplicate entries in google dataset',len(duplicate_apps))\n",
    "print('No of unique entries in google daataset',len(unique_apps))\n",
    "\n",
    "# Apple dataset\n",
    "\n",
    "duplicate_apps_apple=[]\n",
    "unique_apps_apple=[]\n",
    "for app in apple:\n",
    "    app_name=app[0]\n",
    "    if app_name in unique_apps:\n",
    "        duplicate_apps_apple.append(app_name)\n",
    "    else:\n",
    "        unique_apps_apple.append(app_name)\n",
    "print('No of duplicate entries in apple dataset',len(duplicate_apps_apple))\n",
    "print('No of unique entries in apple daataset',len(unique_apps_apple))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Quick PDF Scanner + OCR FREE', 'Box', 'Google My Business', 'ZOOM Cloud Meetings', 'join.me - Simple Meetings']\n"
     ]
    }
   ],
   "source": [
    "# print few duplicate apps \n",
    "print(duplicate_apps[:5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Quick PDF Scanner + OCR FREE', 'BUSINESS', '4.2', '80805', 'Varies with device', '5,000,000+', 'Free', '0', 'Everyone', 'Business', 'February 26, 2018', 'Varies with device', '4.0.3 and up']\n",
      "['Quick PDF Scanner + OCR FREE', 'BUSINESS', '4.2', '80805', 'Varies with device', '5,000,000+', 'Free', '0', 'Everyone', 'Business', 'February 26, 2018', 'Varies with device', '4.0.3 and up']\n",
      "['Quick PDF Scanner + OCR FREE', 'BUSINESS', '4.2', '80804', 'Varies with device', '5,000,000+', 'Free', '0', 'Everyone', 'Business', 'February 26, 2018', 'Varies with device', '4.0.3 and up']\n"
     ]
    }
   ],
   "source": [
    "# Check all duplicated rows of any duplicate app randomly chosen\n",
    "\n",
    "for app in google:\n",
    "    if app[0]==duplicate_apps[0]:\n",
    "        print(app)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Box', 'BUSINESS', '4.2', '159872', 'Varies with device', '10,000,000+', 'Free', '0', 'Everyone', 'Business', 'July 31, 2018', 'Varies with device', 'Varies with device']\n",
      "['Box', 'BUSINESS', '4.2', '159872', 'Varies with device', '10,000,000+', 'Free', '0', 'Everyone', 'Business', 'July 31, 2018', 'Varies with device', 'Varies with device']\n",
      "['Box', 'BUSINESS', '4.2', '159872', 'Varies with device', '10,000,000+', 'Free', '0', 'Everyone', 'Business', 'July 31, 2018', 'Varies with device', 'Varies with device']\n"
     ]
    }
   ],
   "source": [
    "for app in google:\n",
    "    if app[0]==duplicate_apps[5]:\n",
    "        print(app)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Garena Free Fire', 'GAME', '4.5', '5465624', '53M', '100,000,000+', 'Free', '0', 'Teen', 'Action', 'August 3, 2018', '1.21.0', '4.0.3 and up']\n",
      "['Garena Free Fire', 'GAME', '4.5', '5476569', '53M', '100,000,000+', 'Free', '0', 'Teen', 'Action', 'August 3, 2018', '1.21.0', '4.0.3 and up']\n",
      "['Garena Free Fire', 'GAME', '4.5', '5476569', '53M', '100,000,000+', 'Free', '0', 'Teen', 'Action', 'August 3, 2018', '1.21.0', '4.0.3 and up']\n",
      "['Garena Free Fire', 'GAME', '4.5', '5534114', '53M', '100,000,000+', 'Free', '0', 'Teen', 'Action', 'August 3, 2018', '1.21.0', '4.0.3 and up']\n"
     ]
    }
   ],
   "source": [
    "for app in google:\n",
    "    if app[0]==\"Garena Free Fire\":\n",
    "        print(app)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Observation: In duplicate entries of above three apps i found few apps have all data enties in rows same but few apps differ in 'Reviews' entries and rest data is same.\n",
    "\n",
    "* I can keep row with highest no of reviews in dataset because higest no of reviews would be latest and removing rest of duplicate entries"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Removing Duplicate Entries:Part Two"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[['App', 'Category', 'Rating', 'Reviews', 'Size', 'Installs', 'Type', 'Price', 'Content Rating', 'Genres', 'Last Updated', 'Current Ver', 'Android Ver']]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check the index of 'Review' column\n",
    "\n",
    "print(google_header)\n",
    "google_header[0].index('Reviews')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Creating dictionary where each key is a unique app name and corresponding\n",
    "value is the highest no of reviews of that app"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9659\n"
     ]
    }
   ],
   "source": [
    "# iniliasise empty dictionary to store max review of apps\n",
    "reviews_max={}\n",
    "\n",
    "for row in google:\n",
    "    app_name=row[0]\n",
    "    reviews=float(row[3])\n",
    "    if app_name in reviews_max and reviews_max[app_name]<reviews:\n",
    "            reviews_max[app_name]=float(row[3])\n",
    "    # avoided else to prevent wrong updastes in reviews   \n",
    "    if app_name not in reviews_max:            \n",
    "        reviews_max[app_name]=reviews\n",
    "print(len(reviews_max))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* I will use reviews_max dictionary to delete duplicate entries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# create empty list to store cleaned data\n",
    "google_clean=[]\n",
    "\n",
    "#create empty list to store app names\n",
    "already_added=[]\n",
    "\n",
    "#loop through the google data set\n",
    "for app_row in google:\n",
    "    app_name=app_row[0]\n",
    "    reviews=float(app_row[3])\n",
    "    if (reviews==reviews_max[app_name]) and (app_name not in already_added):\n",
    "        google_clean.append(app_row)\n",
    "        already_added.append(app_name) #keep this inside if block ,i made misrake here now corrected"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* explore cleaned google data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[['Photo Editor & Candy Camera & Grid & ScrapBook', 'ART_AND_DESIGN', '4.1', '159', '19M', '10,000+', 'Free', '0', 'Everyone', 'Art & Design', 'January 7, 2018', '1.0.0', '4.0.3 and up'], ['U Launcher Lite – FREE Live Cool Themes, Hide Apps', 'ART_AND_DESIGN', '4.7', '87510', '8.7M', '5,000,000+', 'Free', '0', 'Everyone', 'Art & Design', 'August 1, 2018', '1.2.4', '4.0.3 and up'], ['Sketch - Draw & Paint', 'ART_AND_DESIGN', '4.5', '215644', '25M', '50,000,000+', 'Free', '0', 'Teen', 'Art & Design', 'June 8, 2018', 'Varies with device', '4.2 and up'], ['Pixel Draw - Number Art Coloring Book', 'ART_AND_DESIGN', '4.3', '967', '2.8M', '100,000+', 'Free', '0', 'Everyone', 'Art & Design;Creativity', 'June 20, 2018', '1.1', '4.4 and up'], ['Paper flowers instructions', 'ART_AND_DESIGN', '4.4', '167', '5.6M', '50,000+', 'Free', '0', 'Everyone', 'Art & Design', 'March 26, 2017', '1.0', '2.3 and up']]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "9659"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(google_clean[:5])\n",
    "len(google_clean)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Removing Non-English Apps:Part One"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "* we use English for the apps we develop at our company, and we'd like to analyze only the apps that are directed toward an English-speaking audience"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* The numbers corresponding to the characters we commonly use in an English text are all in the range 0 to 127. We will use this concept to remove non english apps."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### * Write function that takes string and returns false if there's any character in string that doesn't falls in 0 to 127 range"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Approach 1\n",
    "\n",
    "def english_apps(input_string):\n",
    "    for s in input_string:\n",
    "        if ord(s) in range(0,128):\n",
    "            return True\n",
    "        else:\n",
    "            return False\n",
    "\n",
    "# Approach 2\n",
    "\n",
    "def englishh_apps(input_string):\n",
    "    for s in input_string:\n",
    "        if ord(s)>127:\n",
    "            return False\n",
    "        else:\n",
    "            return True\n",
    "        \n",
    "# I used two approaches to define function and both are correct"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "False\n",
      "False\n",
      "True\n",
      "True\n"
     ]
    }
   ],
   "source": [
    "# Check function whether it checks english apps corectly\n",
    "# Checking both functions but in code i will use one of it \n",
    "\n",
    "print(english_apps('爱奇艺PPS -《欢乐颂2》电视剧热播'))\n",
    "print(englishh_apps('爱奇艺PPS -《欢乐颂2》电视剧热播'))\n",
    "\n",
    "print(english_apps('Instagram'))\n",
    "print(englishh_apps('Instagram'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "englishh_apps('Instachat 😜')\n",
    "# function is considering emoji withing range which is not valid \n",
    "#and counting app name as english "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "english_apps('😜')\n",
    "# when passed single emoji ,it returned falls means function is correct\n",
    "# but why it returns True above ???????"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Removing Non-English Apps:Part Two"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Go™ Free Office Suite' and 'Instachat 😜'.Function will not count in english apps.This is because emojis and characters like ™ fall outside the ASCII range and have corresponding numbers over 127."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* If we're going to use the function we've created, we'll lose useful data since many English apps will be incorrectly labeled as non-English. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "128540"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ord('😜')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " * To minimize the impact of data loss, we'll only remove an app if its name has more than three characters with corresponding numbers falling outside the ASCII range."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def english_appss(input_string):\n",
    "    count=0\n",
    "    for s in input_string:\n",
    "        if ord(s)>127:\n",
    "            count+=1\n",
    "    if count<=3:\n",
    "        return True \n",
    "    else:\n",
    "         return False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "english_appss('Instachat 😜😜😜😜')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "english_appss('爱奇艺PPS -《欢乐颂2》电视剧热播')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "english_apps('Instachat 😜')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Above function is now fairly better for removing non english apps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# for google dataset\n",
    "\n",
    "# initialise empty list to store rows having english language apps only\n",
    "english_nonduplicate=[]\n",
    "\n",
    "# remove rows having non english apps entries\n",
    "\n",
    "for app_row in google_clean:\n",
    "    app_name=app_row[0]\n",
    "    if english_appss(app_name):\n",
    "        english_nonduplicate.append(app_row)\n",
    "        \n",
    "# For Apple dataset\n",
    "\n",
    "english_apple=[]\n",
    "\n",
    "# remove rows having non english apps entries\n",
    "\n",
    "for app_row in apple:\n",
    "    app_name=app_row[0]\n",
    "    if english_appss(app_name):\n",
    "        english_apple.append(app_row)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9614\n",
      "[['Photo Editor & Candy Camera & Grid & ScrapBook', 'ART_AND_DESIGN', '4.1', '159', '19M', '10,000+', 'Free', '0', 'Everyone', 'Art & Design', 'January 7, 2018', '1.0.0', '4.0.3 and up'], ['U Launcher Lite – FREE Live Cool Themes, Hide Apps', 'ART_AND_DESIGN', '4.7', '87510', '8.7M', '5,000,000+', 'Free', '0', 'Everyone', 'Art & Design', 'August 1, 2018', '1.2.4', '4.0.3 and up'], ['Sketch - Draw & Paint', 'ART_AND_DESIGN', '4.5', '215644', '25M', '50,000,000+', 'Free', '0', 'Teen', 'Art & Design', 'June 8, 2018', 'Varies with device', '4.2 and up'], ['Pixel Draw - Number Art Coloring Book', 'ART_AND_DESIGN', '4.3', '967', '2.8M', '100,000+', 'Free', '0', 'Everyone', 'Art & Design;Creativity', 'June 20, 2018', '1.1', '4.4 and up'], ['Paper flowers instructions', 'ART_AND_DESIGN', '4.4', '167', '5.6M', '50,000+', 'Free', '0', 'Everyone', 'Art & Design', 'March 26, 2017', '1.0', '2.3 and up']]\n"
     ]
    }
   ],
   "source": [
    "#explore data in new list having only english apps and non duplicated row \n",
    "#entries in google data set\n",
    "print(len(english_nonduplicate))\n",
    "print(english_nonduplicate[:5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7197\n",
      "[['284882215', 'Facebook', '389879808', 'USD', '0.0', '2974676', '212', '3.5', '3.5', '95.0', '4+', 'Social Networking', '37', '1', '29', '1'], ['389801252', 'Instagram', '113954816', 'USD', '0.0', '2161558', '1289', '4.5', '4.0', '10.23', '12+', 'Photo & Video', '37', '0', '29', '1'], ['529479190', 'Clash of Clans', '116476928', 'USD', '0.0', '2130805', '579', '4.5', '4.5', '9.24.12', '9+', 'Games', '38', '5', '18', '1'], ['420009108', 'Temple Run', '65921024', 'USD', '0.0', '1724546', '3842', '4.5', '4.0', '1.6.2', '9+', 'Games', '40', '5', '1', '1'], ['284035177', 'Pandora - Music & Radio', '130242560', 'USD', '0.0', '1126879', '3594', '4.0', '4.5', '8.4.1', '12+', 'Music', '37', '4', '1', '1']]\n"
     ]
    }
   ],
   "source": [
    "#explore data in new list having only english apps and non duplicated row \n",
    "#entries in apple data set\n",
    "print(len(english_apple))\n",
    "print(english_apple[:5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Isolate the Free Apps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "str"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Google dataset\n",
    "\n",
    "# check index of price item\n",
    "price_index=google_header[0].index('Price')\n",
    "print(price_index)\n",
    "\n",
    "# check type of price value whether it is string or float\n",
    "type(google[0][7])\n",
    "\n",
    "# it is string, so convert in float"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[['id',\n",
       "  'track_name',\n",
       "  'size_bytes',\n",
       "  'currency',\n",
       "  'price',\n",
       "  'rating_count_tot',\n",
       "  'rating_count_ver',\n",
       "  'user_rating',\n",
       "  'user_rating_ver',\n",
       "  'ver',\n",
       "  'cont_rating',\n",
       "  'prime_genre',\n",
       "  'sup_devices.num',\n",
       "  'ipadSc_urls.num',\n",
       "  'lang.num',\n",
       "  'vpp_lic']]"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "apple_header"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4\n",
      "[['284882215', 'Facebook', '389879808', 'USD', '0.0', '2974676', '212', '3.5', '3.5', '95.0', '4+', 'Social Networking', '37', '1', '29', '1'], ['389801252', 'Instagram', '113954816', 'USD', '0.0', '2161558', '1289', '4.5', '4.0', '10.23', '12+', 'Photo & Video', '37', '0', '29', '1'], ['529479190', 'Clash of Clans', '116476928', 'USD', '0.0', '2130805', '579', '4.5', '4.5', '9.24.12', '9+', 'Games', '38', '5', '18', '1']]\n"
     ]
    }
   ],
   "source": [
    "# Apple dataset\n",
    "\n",
    "# check index of price item\n",
    "price_index=apple_header[0].index('price')\n",
    "print(price_index)\n",
    "\n",
    "# check type of price value whether it is string or float\n",
    "type(apple[0][4])\n",
    "\n",
    "print(apple[:3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# For google dataset\n",
    "#initialise list to store apps data which are free\n",
    "free_english_nonduplicate=[]\n",
    "\n",
    "# Loop over english_nonduplicate list and isolate free apps in separate list\n",
    "\n",
    "for app_row in english_nonduplicate:\n",
    "    if app_row[7]=='0':\n",
    "        free_english_nonduplicate.append(app_row)\n",
    "        \n",
    "\n",
    "# For apple data set\n",
    "\n",
    "#initialise list to store apps data which are free\n",
    "free_english_apple=[]\n",
    "\n",
    "# Loop over english_nonduplicate list and isolate free apps in separate list\n",
    "\n",
    "for app_row in english_apple:\n",
    "    if app_row[4]=='0.0':\n",
    "        free_english_apple.append(app_row)\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "8864"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check no of rows left in google data set after above analysis\n",
    "len(free_english_nonduplicate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4056"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check no of rows left in apple data set after above analysis\n",
    "len(free_english_apple)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Most Common Apps by Genre: Part One"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " So far, I spent a good amount of time on cleaning data, and:\n",
    "* Removed inaccurate data\n",
    "* Removed duplicate app entries\n",
    "* Removed non-English apps\n",
    "* Isolated the free apps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we mentioned in the introduction, our aim is to determine the kinds of apps that are likely to attract more users because our revenue is highly influenced by the number of people using our apps."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### To minimize risks and overhead, our validation strategy for an app idea is comprised of three steps:\n",
    "\n",
    "* Build a minimal Android version of the app, and add it to Google Play.\n",
    "* If the app has a good response from users, we develop it further.\n",
    "* If the app is profitable after six months, we build an iOS version of the app and add it to the App Store."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Observation:\n",
    "I can use 'Category' , 'Geners' , 'size' , 'curr_version' , 'Reviews' columns for analysis.\n",
    "* \"Geners\"  and 'Category' is most suitable for creating frequency regarding geners in google dataset.\n",
    "* 'prime_genre' in apple data set"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Most Common Apps by Genre: Part Two"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Create freq_table() function to find most common geners"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def freq_table(dataset,index):\n",
    "    table={}\n",
    "    total=0\n",
    "    for app_row in dataset:\n",
    "        total+=1\n",
    "        item=app_row[index]\n",
    "        if item in table:\n",
    "            table[item]+=1\n",
    "        else:\n",
    "            table[item]=1\n",
    "    freq_percentage={}\n",
    "    for key in table:\n",
    "        freq_percentage[key]=(table[key]/total)*100\n",
    "    return freq_percentage"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create display_table() function to display frequency table of columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def display_table(dataset,index):\n",
    "    table=freq_table(dataset,index)\n",
    "    table_display=[]\n",
    "    for key in table:\n",
    "        key_val_as_tuple=(table[key],key)\n",
    "        table_display.append(key_val_as_tuple)\n",
    "    table_sorted=sorted(table_display,reverse=True)\n",
    "    for entry in table_sorted:\n",
    "        print(entry[1],':',entry[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9\n",
      "1\n"
     ]
    }
   ],
   "source": [
    "# find index values of 'Geners' and 'Category'\n",
    "print(google_header[0].index('Genres'))\n",
    "print(google_header[0].index('Category'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tools : 8.449909747292418\n",
      "Entertainment : 6.069494584837545\n",
      "Education : 5.347472924187725\n",
      "Business : 4.591606498194946\n",
      "Productivity : 3.892148014440433\n",
      "Lifestyle : 3.892148014440433\n",
      "Finance : 3.7003610108303246\n",
      "Medical : 3.531137184115524\n",
      "Sports : 3.463447653429603\n",
      "Personalization : 3.3167870036101084\n",
      "Communication : 3.2378158844765346\n",
      "Action : 3.1024368231046933\n",
      "Health & Fitness : 3.0798736462093865\n",
      "Photography : 2.944494584837545\n",
      "News & Magazines : 2.7978339350180503\n",
      "Social : 2.6624548736462095\n",
      "Travel & Local : 2.3240072202166067\n",
      "Shopping : 2.2450361010830324\n",
      "Books & Reference : 2.1435018050541514\n",
      "Simulation : 2.0419675090252705\n",
      "Dating : 1.861462093862816\n",
      "Arcade : 1.8501805054151623\n",
      "Video Players & Editors : 1.7712093862815883\n",
      "Casual : 1.7599277978339352\n",
      "Maps & Navigation : 1.3989169675090252\n",
      "Food & Drink : 1.2409747292418771\n",
      "Puzzle : 1.128158844765343\n",
      "Racing : 0.9927797833935018\n",
      "Role Playing : 0.9363718411552346\n",
      "Libraries & Demo : 0.9363718411552346\n",
      "Auto & Vehicles : 0.9250902527075812\n",
      "Strategy : 0.9138086642599278\n",
      "House & Home : 0.8235559566787004\n",
      "Weather : 0.8009927797833934\n",
      "Events : 0.7107400722021661\n",
      "Adventure : 0.6768953068592057\n",
      "Comics : 0.6092057761732852\n",
      "Beauty : 0.5979241877256317\n",
      "Art & Design : 0.5979241877256317\n",
      "Parenting : 0.4963898916967509\n",
      "Card : 0.45126353790613716\n",
      "Casino : 0.42870036101083037\n",
      "Trivia : 0.41741877256317694\n",
      "Educational;Education : 0.39485559566787\n",
      "Board : 0.3835740072202166\n",
      "Educational : 0.3722924187725632\n",
      "Education;Education : 0.33844765342960287\n",
      "Word : 0.2594765342960289\n",
      "Casual;Pretend Play : 0.236913357400722\n",
      "Music : 0.2030685920577617\n",
      "Racing;Action & Adventure : 0.16922382671480143\n",
      "Puzzle;Brain Games : 0.16922382671480143\n",
      "Entertainment;Music & Video : 0.16922382671480143\n",
      "Casual;Brain Games : 0.13537906137184114\n",
      "Casual;Action & Adventure : 0.13537906137184114\n",
      "Arcade;Action & Adventure : 0.12409747292418773\n",
      "Action;Action & Adventure : 0.10153429602888085\n",
      "Educational;Pretend Play : 0.09025270758122744\n",
      "Simulation;Action & Adventure : 0.078971119133574\n",
      "Parenting;Education : 0.078971119133574\n",
      "Entertainment;Brain Games : 0.078971119133574\n",
      "Board;Brain Games : 0.078971119133574\n",
      "Parenting;Music & Video : 0.06768953068592057\n",
      "Educational;Brain Games : 0.06768953068592057\n",
      "Casual;Creativity : 0.06768953068592057\n",
      "Art & Design;Creativity : 0.06768953068592057\n",
      "Education;Pretend Play : 0.056407942238267145\n",
      "Role Playing;Pretend Play : 0.04512635379061372\n",
      "Education;Creativity : 0.04512635379061372\n",
      "Role Playing;Action & Adventure : 0.033844765342960284\n",
      "Puzzle;Action & Adventure : 0.033844765342960284\n",
      "Entertainment;Creativity : 0.033844765342960284\n",
      "Entertainment;Action & Adventure : 0.033844765342960284\n",
      "Educational;Creativity : 0.033844765342960284\n",
      "Educational;Action & Adventure : 0.033844765342960284\n",
      "Education;Music & Video : 0.033844765342960284\n",
      "Education;Brain Games : 0.033844765342960284\n",
      "Education;Action & Adventure : 0.033844765342960284\n",
      "Adventure;Action & Adventure : 0.033844765342960284\n",
      "Video Players & Editors;Music & Video : 0.02256317689530686\n",
      "Sports;Action & Adventure : 0.02256317689530686\n",
      "Simulation;Pretend Play : 0.02256317689530686\n",
      "Puzzle;Creativity : 0.02256317689530686\n",
      "Music;Music & Video : 0.02256317689530686\n",
      "Entertainment;Pretend Play : 0.02256317689530686\n",
      "Casual;Education : 0.02256317689530686\n",
      "Board;Action & Adventure : 0.02256317689530686\n",
      "Video Players & Editors;Creativity : 0.01128158844765343\n",
      "Trivia;Education : 0.01128158844765343\n",
      "Travel & Local;Action & Adventure : 0.01128158844765343\n",
      "Tools;Education : 0.01128158844765343\n",
      "Strategy;Education : 0.01128158844765343\n",
      "Strategy;Creativity : 0.01128158844765343\n",
      "Strategy;Action & Adventure : 0.01128158844765343\n",
      "Simulation;Education : 0.01128158844765343\n",
      "Role Playing;Brain Games : 0.01128158844765343\n",
      "Racing;Pretend Play : 0.01128158844765343\n",
      "Puzzle;Education : 0.01128158844765343\n",
      "Parenting;Brain Games : 0.01128158844765343\n",
      "Music & Audio;Music & Video : 0.01128158844765343\n",
      "Lifestyle;Pretend Play : 0.01128158844765343\n",
      "Lifestyle;Education : 0.01128158844765343\n",
      "Health & Fitness;Education : 0.01128158844765343\n",
      "Health & Fitness;Action & Adventure : 0.01128158844765343\n",
      "Entertainment;Education : 0.01128158844765343\n",
      "Communication;Creativity : 0.01128158844765343\n",
      "Comics;Creativity : 0.01128158844765343\n",
      "Casual;Music & Video : 0.01128158844765343\n",
      "Card;Action & Adventure : 0.01128158844765343\n",
      "Books & Reference;Education : 0.01128158844765343\n",
      "Art & Design;Pretend Play : 0.01128158844765343\n",
      "Art & Design;Action & Adventure : 0.01128158844765343\n",
      "Arcade;Pretend Play : 0.01128158844765343\n",
      "Adventure;Education : 0.01128158844765343\n"
     ]
    }
   ],
   "source": [
    "# display freq table for 'Genres' column\n",
    "display_table(free_english_nonduplicate,9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Observation:\n",
    "'Tools' , 'Entertainment' , 'Education' , 'Business' , genres are most common.It means these genres have  most of users.Making app based on thede genres will be profitable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "FAMILY : 18.907942238267147\n",
      "GAME : 9.724729241877256\n",
      "TOOLS : 8.461191335740072\n",
      "BUSINESS : 4.591606498194946\n",
      "LIFESTYLE : 3.9034296028880866\n",
      "PRODUCTIVITY : 3.892148014440433\n",
      "FINANCE : 3.7003610108303246\n",
      "MEDICAL : 3.531137184115524\n",
      "SPORTS : 3.395758122743682\n",
      "PERSONALIZATION : 3.3167870036101084\n",
      "COMMUNICATION : 3.2378158844765346\n",
      "HEALTH_AND_FITNESS : 3.0798736462093865\n",
      "PHOTOGRAPHY : 2.944494584837545\n",
      "NEWS_AND_MAGAZINES : 2.7978339350180503\n",
      "SOCIAL : 2.6624548736462095\n",
      "TRAVEL_AND_LOCAL : 2.33528880866426\n",
      "SHOPPING : 2.2450361010830324\n",
      "BOOKS_AND_REFERENCE : 2.1435018050541514\n",
      "DATING : 1.861462093862816\n",
      "VIDEO_PLAYERS : 1.7937725631768955\n",
      "MAPS_AND_NAVIGATION : 1.3989169675090252\n",
      "FOOD_AND_DRINK : 1.2409747292418771\n",
      "EDUCATION : 1.1620036101083033\n",
      "ENTERTAINMENT : 0.9589350180505415\n",
      "LIBRARIES_AND_DEMO : 0.9363718411552346\n",
      "AUTO_AND_VEHICLES : 0.9250902527075812\n",
      "HOUSE_AND_HOME : 0.8235559566787004\n",
      "WEATHER : 0.8009927797833934\n",
      "EVENTS : 0.7107400722021661\n",
      "PARENTING : 0.6543321299638989\n",
      "ART_AND_DESIGN : 0.6430505415162455\n",
      "COMICS : 0.6204873646209386\n",
      "BEAUTY : 0.5979241877256317\n"
     ]
    }
   ],
   "source": [
    "# display freq table for 'Category' column\n",
    "display_table(free_english_nonduplicate,1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Observation:\n",
    "'Family' , 'Game' , 'Tools' , 'Business' are most popular categories"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "11"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check index of 'prime_genre' column\n",
    "apple_header[0].index('prime_genre')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Games : 55.64595660749507\n",
      "Entertainment : 8.234714003944774\n",
      "Photo & Video : 4.117357001972387\n",
      "Social Networking : 3.5256410256410255\n",
      "Education : 3.2544378698224854\n",
      "Shopping : 2.983234714003945\n",
      "Utilities : 2.687376725838264\n",
      "Lifestyle : 2.3175542406311638\n",
      "Finance : 2.0710059171597637\n",
      "Sports : 1.947731755424063\n",
      "Health & Fitness : 1.8737672583826428\n",
      "Music : 1.6518737672583828\n",
      "Book : 1.6272189349112427\n",
      "Productivity : 1.5285996055226825\n",
      "News : 1.4299802761341223\n",
      "Travel : 1.3806706114398422\n",
      "Food & Drink : 1.0601577909270217\n",
      "Weather : 0.7642998027613412\n",
      "Reference : 0.4930966469428008\n",
      "Navigation : 0.4930966469428008\n",
      "Business : 0.4930966469428008\n",
      "Catalogs : 0.22189349112426035\n",
      "Medical : 0.19723865877712032\n"
     ]
    }
   ],
   "source": [
    "# display freq table for 'prime_genre' column in apple dataset\n",
    "display_table(free_english_apple,11)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Observation:\n",
    "'Games' , 'Entertainment', 'Photo and Video', 'Social Networking' \n",
    "are most ppoppular genres on apple store."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Most Common Apps by Genre: Part Three"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Analyzing the frequency table generated for the prime_genre column of the App Store data set.\n",
    "\n",
    "* What is the most common genre? What is the runner-up?\n",
    "* What other patterns ?\n",
    "* What is the general impression — are most of the apps designed for practical purposes (education, shopping, utilities, productivity, lifestyle) or more for entertainment (games, photo and video, social networking, sports, music)?\n",
    "* Can i recommend an app profile for the App Store market based on this frequency table alone? \n",
    "* If there's a large number of apps for a particular genre, does that also imply that apps of that genre generally have a large number of users?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Observation:\n",
    "* 'Games' is the most common genre. 'Medical' is the least common genre.\n",
    "* Most of apps are designed for entertainment purpose. Lesser apps are designed for practical purpose.\n",
    "* If large no of apps are of particulare genre in store it does not necessarily mean it has large no users.It may be developers are interested or biased for games genre app.But it may also indicate large no of users as developer try to create apps which people more like and intended to use."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Analyze the frequency table you generated for the Category and Genres column of the Google Play data set.\n",
    "\n",
    "* What are the most common genres?\n",
    "* What other patterns ?\n",
    "* Compare the patterns you see for the Google Play market with those you saw for the App Store market.\n",
    "* Can you recommend an app profile based on what you found so far? Do the frequency tables you generated reveal the most frequent app genres or what genres have the most users?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Observaton:\n",
    "* 'Tools' in genres and 'Family' in category are most common genres.\n",
    "* Google play store has balanced apps designed for practical purpose and entertainment.\n",
    "* Comparing in both stores apps entertainment and games are most frequent genres "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Most Popular Apps by Genre on the App Store"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "what genres are the most popular (have the most users) ?\n",
    "We can use 'Installs' column but this information is missing in App store dataset , so we will take total number of user ratings as proxy into account "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Book': 1.6272189349112427,\n",
       " 'Business': 0.4930966469428008,\n",
       " 'Catalogs': 0.22189349112426035,\n",
       " 'Education': 3.2544378698224854,\n",
       " 'Entertainment': 8.234714003944774,\n",
       " 'Finance': 2.0710059171597637,\n",
       " 'Food & Drink': 1.0601577909270217,\n",
       " 'Games': 55.64595660749507,\n",
       " 'Health & Fitness': 1.8737672583826428,\n",
       " 'Lifestyle': 2.3175542406311638,\n",
       " 'Medical': 0.19723865877712032,\n",
       " 'Music': 1.6518737672583828,\n",
       " 'Navigation': 0.4930966469428008,\n",
       " 'News': 1.4299802761341223,\n",
       " 'Photo & Video': 4.117357001972387,\n",
       " 'Productivity': 1.5285996055226825,\n",
       " 'Reference': 0.4930966469428008,\n",
       " 'Shopping': 2.983234714003945,\n",
       " 'Social Networking': 3.5256410256410255,\n",
       " 'Sports': 1.947731755424063,\n",
       " 'Travel': 1.3806706114398422,\n",
       " 'Utilities': 2.687376725838264,\n",
       " 'Weather': 0.7642998027613412}"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create frequncy table for prime_genre column\n",
    "apple_genre_freq=freq_table(free_english_apple,11)\n",
    "apple_genre_freq"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Music : 56482.02985074627\n",
      "Health & Fitness : 19952.315789473683\n",
      "Food & Drink : 20179.093023255813\n",
      "Shopping : 18746.677685950413\n",
      "Navigation : 25972.05\n",
      "Photo & Video : 27249.892215568863\n",
      "Sports : 20128.974683544304\n",
      "Business : 6367.8\n",
      "Social Networking : 53078.195804195806\n",
      "Entertainment : 10822.961077844311\n",
      "Lifestyle : 8978.308510638299\n",
      "Utilities : 14010.100917431193\n",
      "Travel : 20216.01785714286\n",
      "News : 15892.724137931034\n",
      "Games : 18924.68896765618\n",
      "Medical : 459.75\n",
      "Book : 8498.333333333334\n",
      "Catalogs : 1779.5555555555557\n",
      "Education : 6266.333333333333\n",
      "Productivity : 19053.887096774193\n",
      "Finance : 13522.261904761905\n",
      "Weather : 47220.93548387097\n",
      "Reference : 67447.9\n"
     ]
    }
   ],
   "source": [
    "# Loop over apple_genre_freq \n",
    "for genre in apple_genre_freq:\n",
    "    total=0\n",
    "    len_genre=0\n",
    "    for app_row in free_english_apple:\n",
    "        genre_app=app_row[11]\n",
    "        if genre_app==genre:\n",
    "            n_ratings=float(app_row[5])\n",
    "            total+=n_ratings\n",
    "            len_genre+=1\n",
    "    avg_n_ratings = total / len_genre\n",
    "    print(genre, ':', avg_n_ratings)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Most Popular Apps by Genre on Google Play"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1,000,000+ : 15.726534296028879\n",
      "100,000+ : 11.552346570397113\n",
      "10,000,000+ : 10.548285198555957\n",
      "10,000+ : 10.198555956678701\n",
      "1,000+ : 8.393501805054152\n",
      "100+ : 6.915613718411552\n",
      "5,000,000+ : 6.825361010830325\n",
      "500,000+ : 5.561823104693141\n",
      "50,000+ : 4.7721119133574\n",
      "5,000+ : 4.512635379061372\n",
      "10+ : 3.5424187725631766\n",
      "500+ : 3.2490974729241873\n",
      "50,000,000+ : 2.3014440433213\n",
      "100,000,000+ : 2.1322202166064983\n",
      "50+ : 1.917870036101083\n",
      "5+ : 0.78971119133574\n",
      "1+ : 0.5076714801444043\n",
      "500,000,000+ : 0.2707581227436823\n",
      "1,000,000,000+ : 0.22563176895306858\n",
      "0+ : 0.04512635379061372\n",
      "0 : 0.01128158844765343\n"
     ]
    }
   ],
   "source": [
    "# Explore 'Installs' column in google dataset\n",
    "display_table(free_english_nonduplicate,5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* We don't know whether an app with 100,000+ installs has 100,000 installs, 200,000, or 350,000. However, we don't need very precise data for our purposes — we only want to find out which app genres attract the most users, and we don't need perfect precision with respect to the number of users.\n",
    "\n",
    "* We're going to leave the numbers as they are, which means that we'll consider that an app with 100,000+ installs has 100,000 installs, and an app with 1,000,000+ installs has 1,000,000 installs, and so on. \n",
    "* To perform computations, however, we'll need to convert each install number from string to float. \n",
    "* This means we need to remove the commas and the plus characters, otherwise the conversion will fail and raise an error."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'COMMUNICATION': 3.2378158844765346, 'MAPS_AND_NAVIGATION': 1.3989169675090252, 'SOCIAL': 2.6624548736462095, 'LIBRARIES_AND_DEMO': 0.9363718411552346, 'AUTO_AND_VEHICLES': 0.9250902527075812, 'PARENTING': 0.6543321299638989, 'GAME': 9.724729241877256, 'ART_AND_DESIGN': 0.6430505415162455, 'PRODUCTIVITY': 3.892148014440433, 'FINANCE': 3.7003610108303246, 'NEWS_AND_MAGAZINES': 2.7978339350180503, 'WEATHER': 0.8009927797833934, 'SPORTS': 3.395758122743682, 'COMICS': 0.6204873646209386, 'DATING': 1.861462093862816, 'FAMILY': 18.907942238267147, 'BOOKS_AND_REFERENCE': 2.1435018050541514, 'HEALTH_AND_FITNESS': 3.0798736462093865, 'ENTERTAINMENT': 0.9589350180505415, 'BUSINESS': 4.591606498194946, 'TRAVEL_AND_LOCAL': 2.33528880866426, 'PHOTOGRAPHY': 2.944494584837545, 'EVENTS': 0.7107400722021661, 'MEDICAL': 3.531137184115524, 'SHOPPING': 2.2450361010830324, 'LIFESTYLE': 3.9034296028880866, 'BEAUTY': 0.5979241877256317, 'TOOLS': 8.461191335740072, 'PERSONALIZATION': 3.3167870036101084, 'VIDEO_PLAYERS': 1.7937725631768955, 'EDUCATION': 1.1620036101083033, 'FOOD_AND_DRINK': 1.2409747292418771, 'HOUSE_AND_HOME': 0.8235559566787004}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'ART_AND_DESIGN': 0.6430505415162455,\n",
       " 'AUTO_AND_VEHICLES': 0.9250902527075812,\n",
       " 'BEAUTY': 0.5979241877256317,\n",
       " 'BOOKS_AND_REFERENCE': 2.1435018050541514,\n",
       " 'BUSINESS': 4.591606498194946,\n",
       " 'COMICS': 0.6204873646209386,\n",
       " 'COMMUNICATION': 3.2378158844765346,\n",
       " 'DATING': 1.861462093862816,\n",
       " 'EDUCATION': 1.1620036101083033,\n",
       " 'ENTERTAINMENT': 0.9589350180505415,\n",
       " 'EVENTS': 0.7107400722021661,\n",
       " 'FAMILY': 18.907942238267147,\n",
       " 'FINANCE': 3.7003610108303246,\n",
       " 'FOOD_AND_DRINK': 1.2409747292418771,\n",
       " 'GAME': 9.724729241877256,\n",
       " 'HEALTH_AND_FITNESS': 3.0798736462093865,\n",
       " 'HOUSE_AND_HOME': 0.8235559566787004,\n",
       " 'LIBRARIES_AND_DEMO': 0.9363718411552346,\n",
       " 'LIFESTYLE': 3.9034296028880866,\n",
       " 'MAPS_AND_NAVIGATION': 1.3989169675090252,\n",
       " 'MEDICAL': 3.531137184115524,\n",
       " 'NEWS_AND_MAGAZINES': 2.7978339350180503,\n",
       " 'PARENTING': 0.6543321299638989,\n",
       " 'PERSONALIZATION': 3.3167870036101084,\n",
       " 'PHOTOGRAPHY': 2.944494584837545,\n",
       " 'PRODUCTIVITY': 3.892148014440433,\n",
       " 'SHOPPING': 2.2450361010830324,\n",
       " 'SOCIAL': 2.6624548736462095,\n",
       " 'SPORTS': 3.395758122743682,\n",
       " 'TOOLS': 8.461191335740072,\n",
       " 'TRAVEL_AND_LOCAL': 2.33528880866426,\n",
       " 'VIDEO_PLAYERS': 1.7937725631768955,\n",
       " 'WEATHER': 0.8009927797833934}"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create frequency table to get unique app genres in category column\n",
    "category_freq=freq_table(free_english_nonduplicate,1)\n",
    "print(category_freq)\n",
    "category_freq        \n",
    "# Why different ordering of keys in dict with print and without print\n",
    "# statement???????????????????????????????????????"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "COMMUNICATION : 38456119.167247385\n",
      "MAPS_AND_NAVIGATION : 4056941.7741935486\n",
      "SOCIAL : 23253652.127118643\n",
      "LIBRARIES_AND_DEMO : 638503.734939759\n",
      "AUTO_AND_VEHICLES : 647317.8170731707\n",
      "PARENTING : 542603.6206896552\n",
      "GAME : 15588015.603248259\n",
      "ART_AND_DESIGN : 1986335.0877192982\n",
      "PRODUCTIVITY : 16787331.344927534\n",
      "FINANCE : 1387692.475609756\n",
      "NEWS_AND_MAGAZINES : 9549178.467741935\n",
      "WEATHER : 5074486.197183099\n",
      "SPORTS : 3638640.1428571427\n",
      "COMICS : 817657.2727272727\n",
      "DATING : 854028.8303030303\n",
      "FAMILY : 3695641.8198090694\n",
      "BOOKS_AND_REFERENCE : 8767811.894736841\n",
      "HEALTH_AND_FITNESS : 4188821.9853479853\n",
      "ENTERTAINMENT : 11640705.88235294\n",
      "BUSINESS : 1712290.1474201474\n",
      "TRAVEL_AND_LOCAL : 13984077.710144928\n",
      "PHOTOGRAPHY : 17840110.40229885\n",
      "EVENTS : 253542.22222222222\n",
      "MEDICAL : 120550.61980830671\n",
      "SHOPPING : 7036877.311557789\n",
      "LIFESTYLE : 1437816.2687861272\n",
      "BEAUTY : 513151.88679245283\n",
      "TOOLS : 10801391.298666667\n",
      "PERSONALIZATION : 5201482.6122448975\n",
      "VIDEO_PLAYERS : 24727872.452830188\n",
      "EDUCATION : 1833495.145631068\n",
      "FOOD_AND_DRINK : 1924897.7363636363\n",
      "HOUSE_AND_HOME : 1331540.5616438356\n"
     ]
    }
   ],
   "source": [
    "# Loop over category freq \n",
    "for category in category_freq:\n",
    "    total=0\n",
    "    len_category=0\n",
    "    for app_row in free_english_nonduplicate:\n",
    "        category_app=app_row[1]\n",
    "        if category_app==category:\n",
    "            n_installs=app_row[5].replace(',','').replace('+','')\n",
    "            total+=float(n_installs)\n",
    "            len_category+=1\n",
    "    avg_installs=total/len_category\n",
    "    print(category,':',avg_installs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## In this project, we went through a complete data science workflow:\n",
    "* We started by clarifying the goal of our project.\n",
    "* We collected relevant data.\n",
    "* We cleaned the data to prepare it for analysis.\n",
    "* We analyzed the cleaned data."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}