Skip to content
Code for making an API to contribute data to DC2's data lake
Python
Branch: master
Clone or download
This branch is 13 commits behind datacommunitydc:master.

Latest commit

Fetching latest commit…
Cannot retrieve the latest commit at this time.

Files

Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
LICENSE
README.md
Twitter_Meetup_Utility.py
api.py
datacommunitydc_apy.wsgi
default
default-ssl
example_config.py

README.md

Code for making an API to contribute data to DC2's data lake

Running the Flask API Locally:

REQUIRED SERVICES

To run locally, you will need to have a redis server and amqp server running and a celery worker with the adjunct to accept jobs.

On ubuntu 12.04 or later, simply run to install these services and begin running them on default ports.

sudo apt-get redis-server python-celery

DEPENDENCIES

Python packages required include:

flask boto redis amqp magic xlrd

Install with:

sudo easy_install magic sudo pip install Flask boto redis amqp xlrd

Note that several of these dependencies overlap with the adjunct processor.

Additionally, you will need the ThreeScalePY package which must be downloaded from github and installed manually. Clone this repository and follow instructions for setup: https://github.com/3scale/3scale_ws_api_for_python

CONFIGURATION

Copy sample_config.py to config.py and alter the appropriate variables if necessary.

RUNNING

To run the api, type:

python api.py

This terminal will now display status messages from the API.

CALLING

The api will be running on your localhost with port 5000 by default.

To call, use the following example curl calls with valid app_id and app_key pairs:

POST file data: curl -i -F "app_id=12345EXAMPLE" -F "app_key=abc6943jqq509x26EXAMPLE" -F "file=@/path/to/1984_Excerpt.txt;type=text/plain" "http://127.0.0.1:5000/v1/documents"

GET the status of a job: curl -i -X GET "http://127.0.0.1:5000/v1/documents/queue/f4d732ae-4264-40b5-9540-bf75f4799a76?app_id=12345EXAMPLE&app_key=abc6943jqq509x26EXAMPLE"

GET the location of the result files: curl -i -X GET "http://127.0.0.1:5000/v1/documents/f4d732ae-4264-40b5-9540-bf75f4799a76?app_id=12345EXAMPLE&app_key=abc6943jqq509x26EXAMPLE"

MONITORING

You can monitor the status of the API from either the terminal from which it was started OR by viewing the api.log file.

CHANGELOG

You can’t perform that action at this time.