Code for making an API to contribute data to DC2's data lake
Running the Flask API Locally:
To run locally, you will need to have a redis server and amqp server running and a celery worker with the adjunct to accept jobs.
On ubuntu 12.04 or later, simply run to install these services and begin running them on default ports.
sudo apt-get redis-server python-celery
Python packages required include:
flask boto redis amqp magic xlrd
sudo easy_install magic sudo pip install Flask boto redis amqp xlrd
Note that several of these dependencies overlap with the adjunct processor.
Additionally, you will need the ThreeScalePY package which must be downloaded from github and installed manually. Clone this repository and follow instructions for setup: https://github.com/3scale/3scale_ws_api_for_python
Copy sample_config.py to config.py and alter the appropriate variables if necessary.
To run the api, type:
This terminal will now display status messages from the API.
The api will be running on your localhost with port 5000 by default.
To call, use the following example curl calls with valid app_id and app_key pairs:
POST file data: curl -i -F "app_id=12345EXAMPLE" -F "app_key=abc6943jqq509x26EXAMPLE" -F "file=@/path/to/1984_Excerpt.txt;type=text/plain" "http://127.0.0.1:5000/v1/documents"
GET the status of a job: curl -i -X GET "http://127.0.0.1:5000/v1/documents/queue/f4d732ae-4264-40b5-9540-bf75f4799a76?app_id=12345EXAMPLE&app_key=abc6943jqq509x26EXAMPLE"
GET the location of the result files: curl -i -X GET "http://127.0.0.1:5000/v1/documents/f4d732ae-4264-40b5-9540-bf75f4799a76?app_id=12345EXAMPLE&app_key=abc6943jqq509x26EXAMPLE"
You can monitor the status of the API from either the terminal from which it was started OR by viewing the api.log file.