Skip to content

A simple service to connect Media Cloud and CLIFF, tagging stories in MediaCloud with geographic entities found by CLIFF

Notifications You must be signed in to change notification settings

mitmedialab/MediaCloud-GeoCoder

Repository files navigation

MediaCloud Story GeoCoder

This is a simple script to geocode stories in the MediaCloud database with geographic mentions and country/state of focus using the CLIFF-CLAVIN geocoder.

Installation

First install the MediaCloud API client library. Then install these python dependencies:

pip install -r requirements.pip

Now you need to set up a Redis queue somewhere:

  • On OSX you should use homebrew - brew install redis.
  • On Unbuntu, do apt-get install redis-server (the config file ends up in /etc/redis/redis.conf).

Now copy settings.config.sample to settings.config and be sure to fill in these properties:

  • queue.url
  • mediacloud.key
  • medialcoud.place_tag_set_name
  • cliff.url

Use

First you need to run the fetch-stories.py script load stories into the queue for geocoding (you can do this on a cron job if you want).

Then run the the Celery work server to geocode to stories and post results back to MediaCloud: celery -A mediameter worker -l info.

If you set up Celery as a service on Ubuntu then you can run sudo service celeryd start to start the service:

  • copy the daemon script to `/etc/init.d/celeryd' and make it executable
  • copy the example configuration to /etc/default/celeryd and change the CELERYD_NODES name to something you will recognize, change CELERY_BIN and CELERY_CHDIR to point at your virtualenv, set CELERYD_LOG_LEVEL= "DEBUG" if you want more logging
  • create a new unpriveleged celery user: sudo groupadd celery; sudo useradd -g celery celery

If you need to empty out all your queues, just redis-cli flushall.

About

A simple service to connect Media Cloud and CLIFF, tagging stories in MediaCloud with geographic entities found by CLIFF

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages