Weekly/Weed_10/content_analytics_impact-skeleton.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Content Analytics & Impact"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Scenario  \n",
    "\n",
    "Imagine you want to advise your managing editor on strategies for your news organization to help them maximize their audience size and reach. (Pick a twitter account as \"your\" news organization, which you'll analyze). \n",
    "\n",
    "**Question 1**  \n",
    "What time of day should the social media editor share links to get maximal retweets and / or favorites on Twitter? Is there a difference in what the optimal time is on weekends versus weekdays?\n",
    "\n",
    "**Question 2**  \n",
    "Should the social media editor embed an image in tweets in order to get more retweets?\n",
    "\n",
    "**Question 3**  \n",
    "Should the social media editor use more or less hashtags in tweets in order to get more retweets?\n",
    "\n",
    "Answer the previous questions to provide a report. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Import libraries\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Makes it so that you can scroll horizontally to see all columns of an output DataFrame\n",
    "pd.set_option('display.max_columns', None)\n",
    "# Make it so urls and tweets won't get truncated when we print them out\n",
    "pd.set_option('display.max_colwidth', -1)\n",
    "\n",
    "# This magic function allows you to see the charts directly within the notebook. \n",
    "%matplotlib inline\n",
    "\n",
    "# This command will make the plots more attractive by adopting the commone style of ggplot\n",
    "matplotlib.style.use(\"ggplot\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import tweepy\n",
    "\n",
    "#Setup and authenticate Tweepy - INSERT your keys here. \n",
    "CONSUMER_KEY = \"\"\n",
    "CONSUMER_SECRET = \"\"\n",
    "ACCESS_TOKEN = \"\"\n",
    "ACCESS_TOKEN_SECRET = \"\"\n",
    "\n",
    "auth = tweepy.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)\n",
    "auth.set_access_token(ACCESS_TOKEN, ACCESS_TOKEN_SECRET)\n",
    "api = tweepy.API(auth)\n",
    "\n",
    "me = api.me()\n",
    "print me.screen_name"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Collect Data\n",
    "\n",
    "The next cell contains code to get historical tweets from an account. Set the `account` parameter to that of your news organization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting page  2\n",
      "Collecting page  3\n",
      "Collecting page  4\n",
      "Collecting page  5\n",
      "Collecting page  6\n",
      "Collecting page  7\n",
      "Collecting page  8\n",
      "Collecting page  9\n",
      "Collecting page  10\n",
      "Collecting page  11\n",
      "Collecting page  12\n",
      "Collecting page  13\n",
      "Collecting page  14\n",
      "Collecting page  15\n",
      "Collecting page  16\n",
      "Total number of tweets collected:  2048\n"
     ]
    }
   ],
   "source": [
    "import sys\n",
    "\n",
    "# According to Twitter docs: https://dev.twitter.com/rest/reference/get/statuses/user_timeline\n",
    "# Can only get 3,200 tweets into the history of an account\n",
    "# And you can get 200 at a time, so that's 16 sets of 200 that we can get. \n",
    "\n",
    "# Set the account you want to collect data for using screen name\n",
    "account = \"baltimoresun\"\n",
    "page_size = 200\n",
    "all_tweets = [] # A list that will contain upto 3,200 tweets (we exclude RTs)\n",
    "\n",
    "# Get the first page of tweets\n",
    "max_id = sys.maxint\n",
    "tweets = api.user_timeline(account, count=page_size, include_rts=False)\n",
    "for tweet in tweets:\n",
    "    if tweet.id < max_id:\n",
    "        max_id = tweet.id\n",
    "    all_tweets.append(tweet)\n",
    "\n",
    "# Get subsequent pages of tweets\n",
    "num_pages_to_collect = 16\n",
    "for page in range(1, num_pages_to_collect):\n",
    "    print \"Collecting page \", (page+1)\n",
    "    tweets = api.user_timeline(account, count=page_size, include_rts=False, max_id=max_id)\n",
    "    for tweet in tweets:\n",
    "        if tweet.id < max_id:\n",
    "            max_id = tweet.id\n",
    "        all_tweets.append(tweet)\n",
    "        \n",
    "print \"Total number of tweets collected: \", len(all_tweets)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{u'contributors': None,\n",
      " u'coordinates': None,\n",
      " u'created_at': u'Fri Mar 25 18:10:17 +0000 2016',\n",
      " u'entities': {u'hashtags': [],\n",
      "               u'media': [{u'display_url': u'pic.twitter.com/liDRDt4u3F',\n",
      "                           u'expanded_url': u'http://twitter.com/BaltSunTV/status/713378676851671042/photo/1',\n",
      "                           u'id': 713378676729974785,\n",
      "                           u'id_str': u'713378676729974785',\n",
      "                           u'indices': [103, 126],\n",
      "                           u'media_url': u'http://pbs.twimg.com/media/CeZuIHjUEAE1hnM.jpg',\n",
      "                           u'media_url_https': u'https://pbs.twimg.com/media/CeZuIHjUEAE1hnM.jpg',\n",
      "                           u'sizes': {u'large': {u'h': 684, u'resize': u'fit', u'w': 1024},\n",
      "                                      u'medium': {u'h': 401, u'resize': u'fit', u'w': 600},\n",
      "                                      u'small': {u'h': 227, u'resize': u'fit', u'w': 340},\n",
      "                                      u'thumb': {u'h': 150, u'resize': u'crop', u'w': 150}},\n",
      "                           u'source_status_id': 713378676851671042,\n",
      "                           u'source_status_id_str': u'713378676851671042',\n",
      "                           u'source_user_id': 14921442,\n",
      "                           u'source_user_id_str': u'14921442',\n",
      "                           u'type': u'photo',\n",
      "                           u'url': u'https://t.co/liDRDt4u3F'}],\n",
      "               u'symbols': [],\n",
      "               u'urls': [{u'display_url': u'bsun.md/1ZAdf8b',\n",
      "                          u'expanded_url': u'http://bsun.md/1ZAdf8b',\n",
      "                          u'indices': [79, 102],\n",
      "                          u'url': u'https://t.co/PsaoLw9sOh'}],\n",
      "               u'user_mentions': [{u'id': 14921442,\n",
      "                                   u'id_str': u'14921442',\n",
      "                                   u'indices': [3, 13],\n",
      "                                   u'name': u'BaltSunTV',\n",
      "                                   u'screen_name': u'BaltSunTV'}]},\n",
      " u'extended_entities': {u'media': [{u'display_url': u'pic.twitter.com/liDRDt4u3F',\n",
      "                                    u'expanded_url': u'http://twitter.com/BaltSunTV/status/713378676851671042/photo/1',\n",
      "                                    u'id': 713378676729974785,\n",
      "                                    u'id_str': u'713378676729974785',\n",
      "                                    u'indices': [103, 126],\n",
      "                                    u'media_url': u'http://pbs.twimg.com/media/CeZuIHjUEAE1hnM.jpg',\n",
      "                                    u'media_url_https': u'https://pbs.twimg.com/media/CeZuIHjUEAE1hnM.jpg',\n",
      "                                    u'sizes': {u'large': {u'h': 684, u'resize': u'fit', u'w': 1024},\n",
      "                                               u'medium': {u'h': 401, u'resize': u'fit', u'w': 600},\n",
      "                                               u'small': {u'h': 227, u'resize': u'fit', u'w': 340},\n",
      "                                               u'thumb': {u'h': 150, u'resize': u'crop', u'w': 150}},\n",
      "                                    u'source_status_id': 713378676851671042,\n",
      "                                    u'source_status_id_str': u'713378676851671042',\n",
      "                                    u'source_user_id': 14921442,\n",
      "                                    u'source_user_id_str': u'14921442',\n",
      "                                    u'type': u'photo',\n",
      "                                    u'url': u'https://t.co/liDRDt4u3F'}]},\n",
      " u'favorite_count': 0,\n",
      " u'favorited': False,\n",
      " u'geo': None,\n",
      " u'id': 713427801836232706,\n",
      " u'id_str': u'713427801836232706',\n",
      " u'in_reply_to_screen_name': None,\n",
      " u'in_reply_to_status_id': None,\n",
      " u'in_reply_to_status_id_str': None,\n",
      " u'in_reply_to_user_id': None,\n",
      " u'in_reply_to_user_id_str': None,\n",
      " u'is_quote_status': False,\n",
      " u'lang': u'en',\n",
      " u'place': None,\n",
      " u'possibly_sensitive': False,\n",
      " u'retweet_count': 1,\n",
      " u'retweeted': False,\n",
      " u'retweeted_status': {u'contributors': None,\n",
      "                       u'coordinates': None,\n",
      "                       u'created_at': u'Fri Mar 25 14:55:05 +0000 2016',\n",
      "                       u'entities': {u'hashtags': [],\n",
      "                                     u'media': [{u'display_url': u'pic.twitter.com/liDRDt4u3F',\n",
      "                                                 u'expanded_url': u'http://twitter.com/BaltSunTV/status/713378676851671042/photo/1',\n",
      "                                                 u'id': 713378676729974785,\n",
      "                                                 u'id_str': u'713378676729974785',\n",
      "                                                 u'indices': [88, 111],\n",
      "                                                 u'media_url': u'http://pbs.twimg.com/media/CeZuIHjUEAE1hnM.jpg',\n",
      "                                                 u'media_url_https': u'https://pbs.twimg.com/media/CeZuIHjUEAE1hnM.jpg',\n",
      "                                                 u'sizes': {u'large': {u'h': 684, u'resize': u'fit', u'w': 1024},\n",
      "                                                            u'medium': {u'h': 401, u'resize': u'fit', u'w': 600},\n",
      "                                                            u'small': {u'h': 227, u'resize': u'fit', u'w': 340},\n",
      "                                                            u'thumb': {u'h': 150, u'resize': u'crop', u'w': 150}},\n",
      "                                                 u'type': u'photo',\n",
      "                                                 u'url': u'https://t.co/liDRDt4u3F'}],\n",
      "                                     u'symbols': [],\n",
      "                                     u'urls': [{u'display_url': u'bsun.md/1ZAdf8b',\n",
      "                                                u'expanded_url': u'http://bsun.md/1ZAdf8b',\n",
      "                                                u'indices': [64, 87],\n",
      "                                                u'url': u'https://t.co/PsaoLw9sOh'}],\n",
      "                                     u'user_mentions': []},\n",
      "                       u'extended_entities': {u'media': [{u'display_url': u'pic.twitter.com/liDRDt4u3F',\n",
      "                                                          u'expanded_url': u'http://twitter.com/BaltSunTV/status/713378676851671042/photo/1',\n",
      "                                                          u'id': 713378676729974785,\n",
      "                                                          u'id_str': u'713378676729974785',\n",
      "                                                          u'indices': [88, 111],\n",
      "                                                          u'media_url': u'http://pbs.twimg.com/media/CeZuIHjUEAE1hnM.jpg',\n",
      "                                                          u'media_url_https': u'https://pbs.twimg.com/media/CeZuIHjUEAE1hnM.jpg',\n",
      "                                                          u'sizes': {u'large': {u'h': 684,\n",
      "                                                                                u'resize': u'fit',\n",
      "                                                                                u'w': 1024},\n",
      "                                                                     u'medium': {u'h': 401,\n",
      "                                                                                 u'resize': u'fit',\n",
      "                                                                                 u'w': 600},\n",
      "                                                                     u'small': {u'h': 227,\n",
      "                                                                                u'resize': u'fit',\n",
      "                                                                                u'w': 340},\n",
      "                                                                     u'thumb': {u'h': 150,\n",
      "                                                                                u'resize': u'crop',\n",
      "                                                                                u'w': 150}},\n",
      "                                                          u'type': u'photo',\n",
      "                                                          u'url': u'https://t.co/liDRDt4u3F'}]},\n",
      "                       u'favorite_count': 0,\n",
      "                       u'favorited': False,\n",
      "                       u'geo': None,\n",
      "                       u'id': 713378676851671042,\n",
      "                       u'id_str': u'713378676851671042',\n",
      "                       u'in_reply_to_screen_name': None,\n",
      "                       u'in_reply_to_status_id': None,\n",
      "                       u'in_reply_to_status_id_str': None,\n",
      "                       u'in_reply_to_user_id': None,\n",
      "                       u'in_reply_to_user_id_str': None,\n",
      "                       u'is_quote_status': False,\n",
      "                       u'lang': u'en',\n",
      "                       u'place': None,\n",
      "                       u'possibly_sensitive': False,\n",
      "                       u'retweet_count': 1,\n",
      "                       u'retweeted': False,\n",
      "                       u'source': u'<a href=\"http://dlvr.it\" rel=\"nofollow\">dlvr.it</a>',\n",
      "                       u'text': u\"Hollywood Anonymous: Portrayals of recovery on 'Love', 'Flaked' https://t.co/PsaoLw9sOh https://t.co/liDRDt4u3F\",\n",
      "                       u'truncated': False,\n",
      "                       u'user': {u'contributors_enabled': False,\n",
      "                                 u'created_at': u'Tue May 27 14:44:00 +0000 2008',\n",
      "                                 u'default_profile': False,\n",
      "                                 u'default_profile_image': False,\n",
      "                                 u'description': u'TV recaps and coverage from @baltsunarts contributors and @DavidZurawik.',\n",
      "                                 u'entities': {u'description': {u'urls': []},\n",
      "                                               u'url': {u'urls': [{u'display_url': u'baltimoresun.com/entertainment/\\u2026',\n",
      "                                                                   u'expanded_url': u'http://www.baltimoresun.com/entertainment/tv/tv-lust/',\n",
      "                                                                   u'indices': [0, 22],\n",
      "                                                                   u'url': u'http://t.co/zl6SZiSA6S'}]}},\n",
      "                                 u'favourites_count': 22,\n",
      "                                 u'follow_request_sent': False,\n",
      "                                 u'followers_count': 4229,\n",
      "                                 u'following': False,\n",
      "                                 u'friends_count': 2435,\n",
      "                                 u'geo_enabled': True,\n",
      "                                 u'has_extended_profile': False,\n",
      "                                 u'id': 14921442,\n",
      "                                 u'id_str': u'14921442',\n",
      "                                 u'is_translation_enabled': False,\n",
      "                                 u'is_translator': False,\n",
      "                                 u'lang': u'en',\n",
      "                                 u'listed_count': 322,\n",
      "                                 u'location': u'Baltimore, MD',\n",
      "                                 u'name': u'BaltSunTV',\n",
      "                                 u'notifications': False,\n",
      "                                 u'profile_background_color': u'F58A2E',\n",
      "                                 u'profile_background_image_url': u'http://pbs.twimg.com/profile_background_images/4822219/twitter_collage.jpg',\n",
      "                                 u'profile_background_image_url_https': u'https://pbs.twimg.com/profile_background_images/4822219/twitter_collage.jpg',\n",
      "                                 u'profile_background_tile': True,\n",
      "                                 u'profile_banner_url': u'https://pbs.twimg.com/profile_banners/14921442/1441816207',\n",
      "                                 u'profile_image_url': u'http://pbs.twimg.com/profile_images/641649721296027648/N8VoKANo_normal.jpg',\n",
      "                                 u'profile_image_url_https': u'https://pbs.twimg.com/profile_images/641649721296027648/N8VoKANo_normal.jpg',\n",
      "                                 u'profile_link_color': u'0E69C4',\n",
      "                                 u'profile_sidebar_border_color': u'9DEF2A',\n",
      "                                 u'profile_sidebar_fill_color': u'FD9426',\n",
      "                                 u'profile_text_color': u'000000',\n",
      "                                 u'profile_use_background_image': True,\n",
      "                                 u'protected': False,\n",
      "                                 u'screen_name': u'BaltSunTV',\n",
      "                                 u'statuses_count': 15705,\n",
      "                                 u'time_zone': u'Eastern Time (US & Canada)',\n",
      "                                 u'url': u'http://t.co/zl6SZiSA6S',\n",
      "                                 u'utc_offset': -14400,\n",
      "                                 u'verified': False}},\n",
      " u'source': u'<a href=\"https://about.twitter.com/products/tweetdeck\" rel=\"nofollow\">TweetDeck</a>',\n",
      " u'text': u\"RT @BaltSunTV: Hollywood Anonymous: Portrayals of recovery on 'Love', 'Flaked' https://t.co/PsaoLw9sOh https://t.co/liDRDt4u3F\",\n",
      " u'truncated': False,\n",
      " u'user': {u'contributors_enabled': False,\n",
      "           u'created_at': u'Thu Sep 13 20:21:39 +0000 2007',\n",
      "           u'default_profile': False,\n",
      "           u'default_profile_image': False,\n",
      "           u'description': u'Light for all. For customer service, customersatisfaction@baltsun.com. For our journalists: http://t.co/NrQDTYSV3V',\n",
      "           u'entities': {u'description': {u'urls': [{u'display_url': u'bsun.md/staff',\n",
      "                                                     u'expanded_url': u'http://bsun.md/staff',\n",
      "                                                     u'indices': [92, 114],\n",
      "                                                     u'url': u'http://t.co/NrQDTYSV3V'}]},\n",
      "                         u'url': {u'urls': [{u'display_url': u'baltimoresun.com',\n",
      "                                             u'expanded_url': u'http://baltimoresun.com',\n",
      "                                             u'indices': [0, 22],\n",
      "                                             u'url': u'http://t.co/9qupGo3R00'}]}},\n",
      "           u'favourites_count': 1631,\n",
      "           u'follow_request_sent': False,\n",
      "           u'followers_count': 159587,\n",
      "           u'following': False,\n",
      "           u'friends_count': 12426,\n",
      "           u'geo_enabled': True,\n",
      "           u'has_extended_profile': False,\n",
      "           u'id': 8861752,\n",
      "           u'id_str': u'8861752',\n",
      "           u'is_translation_enabled': False,\n",
      "           u'is_translator': False,\n",
      "           u'lang': u'en',\n",
      "           u'listed_count': 3409,\n",
      "           u'location': u'Baltimore, Maryland',\n",
      "           u'name': u'The Baltimore Sun',\n",
      "           u'notifications': False,\n",
      "           u'profile_background_color': u'75CBF0',\n",
      "           u'profile_background_image_url': u'http://pbs.twimg.com/profile_background_images/245679812/bsunlow.jpg',\n",
      "           u'profile_background_image_url_https': u'https://pbs.twimg.com/profile_background_images/245679812/bsunlow.jpg',\n",
      "           u'profile_background_tile': False,\n",
      "           u'profile_banner_url': u'https://pbs.twimg.com/profile_banners/8861752/1414186163',\n",
      "           u'profile_image_url': u'http://pbs.twimg.com/profile_images/520989247491166209/blL63-Iz_normal.jpeg',\n",
      "           u'profile_image_url_https': u'https://pbs.twimg.com/profile_images/520989247491166209/blL63-Iz_normal.jpeg',\n",
      "           u'profile_link_color': u'15225E',\n",
      "           u'profile_sidebar_border_color': u'C6E2EE',\n",
      "           u'profile_sidebar_fill_color': u'FFE600',\n",
      "           u'profile_text_color': u'663B12',\n",
      "           u'profile_use_background_image': True,\n",
      "           u'protected': False,\n",
      "           u'screen_name': u'baltimoresun',\n",
      "           u'statuses_count': 109320,\n",
      "           u'time_zone': u'Eastern Time (US & Canada)',\n",
      "           u'url': u'http://t.co/9qupGo3R00',\n",
      "           u'utc_offset': -14400,\n",
      "           u'verified': True}}\n"
     ]
    }
   ],
   "source": [
    "# What does a single tweet look like?\n",
    "# Documentation for what gets delivered in a tweet object: https://dev.twitter.com/overview/api/tweets\n",
    "import pprint \n",
    "\n",
    "for i, tweet in enumerate(all_tweets):\n",
    "    if i < 1:\n",
    "        pprint.pprint(tweet._json)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# For example, to get at a few relevant fields\n",
    "for i, tweet in enumerate(all_tweets):\n",
    "    # Comment out the next line if you want to loop through all the tweets in the all_tweets list\n",
    "    if i < 1:\n",
    "        print tweet.favorite_count\n",
    "        print tweet.retweet_count\n",
    "        print tweet.text\n",
    "        print tweet.created_at # The time will be reported in UTC (Universal Coordinated Time)\n",
    "        print tweet.entities.has_key(\"media\") # Does the tweet have \"media\" e.g. an image associated?\n",
    "        print tweet.entities[\"hashtags\"] # List of hashtags that have been parsed out of the tweet\n",
    "        print \"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question 1**  \n",
    "What time of day should the social media editor share links to get maximal retweets and / or favorites on Twitter? Is there a difference in what the optimal time is on weekends versus weekdays?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question 2**  \n",
    "Should the social media editor embed an image in tweets in order to get more retweets?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question 3**  \n",
    "Should the social media editor use more or less hashtags in tweets in order to get more retweets?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Shareback + Discuss\n",
    "\n",
    "What were your team's findings? What was the best time of day to tweet to get RTs and favorites? What are you really measuring (and not measuring) with these metrics?\n",
    "\n",
    "What else could you possibly measure here? Are there other metrics that might be more telling?\n",
    "\n",
    "What are some possible limitations of this approach to content optimization? \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}