Tech Blog Links

This is a scraper that will get links from a set of web pages and will post them on social media

Setup

• Install mongodb
• Install python3.5 and just python3 setup.py install

If you want to contribute be sure that you:

• install the package and requirements

pip3 install -r requirements-dev.txt
python3 setup.py develop

• run flake8 . in the root of the project and fix the errors, before pushing your changes

Run

export FB_PAGE_ID=<your_fb_page_id> 
export FB_ACCESS_TOKEN=<your_fb_page_token>
export TW_CONSUMER_KEY=<your_twitter_consumer_key>
export TW_CONSUMER_SECRET=<your_twitter_consumer_secret>
export TW_ACCESS_TOKEN=<your_twitter_token>
export TW_ACCESS_TOKEN_SECRET=<your_twitter_token_secret>

python3 tbl/manage.py --file <config.json>                    # to fetch and store the links
python3 tbl/manage.py --post                                  # to post a random link on facebook and twitter
python3 tbl/manage.py --file <config.json> --test             # to test the configuration
python3 tbl/manage.py --file <config.json> --remove Instagram # remove all links of one blog

Config example

[
  {
    "name": "Facebook",
    "url": "https://code.facebook.com/posts/rss"
  },
  {
    "name": "Quora",
    "url": "https://engineering.quora.com/rss"
  }
]

OAuth Tokens

Facebook access token

Debug tool: https://developers.facebook.com/tools/debug/accesstoken

Steps to get the extended token:

1. Go to Graph Explorer and generate a user token with scope manage_pages,publish_pages

2. Generate an extended user token

curl -i -X GET "https://graph.facebook.com/oauth/access_token?grant_type=fb_exchange_token&client_id={APP_ID}&client_secret={APP_SECRET}&fb_exchange_token={USER_TOKEN}"

3. Take the access token from the response and go in graph explorer and request an page access token: {PAGE_ID}?fields=access_token

Check the token with the debug tool, should countain Expires: Never

Twitter access tokens

1. Go to Twitter Application Manager and create an app
2. Generate access token using this guide
3. Go to Keys & Access Tokens tab on your new created app