AutoMATES API (on carp)

Overview

The automates API is a web service that provides access to functionality from different parts of the AutoMATES code base. It is a python Flask web app that is located in the repository at automates/apps/automates (a very creative and descriptive path name :) ). Documentation for the api is located here.

Example calls:

I recommend postman for testing calls. A suite of test calls that can be imported to postman are located here named AutoMATES.postman_collection.json.

Note the header value "api key" in the examples. To use these examples for local development, simply remove this header. In order to use these examples with the API deployed on carp, you need to replace the <API_KEY> value with a valid api key.

curl --location --request POST 'http://127.0.0.1:5000/api/v1/translate' \
--header 'Content-Type: application/json' \
--header 'apikey: <API_KEY>' \
--data-raw '{
	"source_code_files": [
		{
			"file_name": "PID.c",
			"file_type": "c",
			"base64_encoding": "I2luY2x1ZGUgPHN0ZGlvLmg+CgpzdHJ1Y3QgX3BpZAp7CglmbG9hdCBTZXRTcGVlZDsKCWZsb2F0IEFjdHVhbFNwZWVkOwoJZmxvYXQgZXJyOwoJZmxvYXQgZXJyX2xhc3Q7CglmbG9hdCBLcCwgS2ksIEtkOwoJZmxvYXQgdm9sdGFnZTsKCWZsb2F0IGludGVncmFsOwp9IHBpZDsKCi8vIGludCB0ZXN0ID0gMTA7Cgp2b2lkIFBJRF9pbml0KCkKewoJcHJpbnRmKCJQSURfaW5pdCBiZWdpbiBcbiIpOwoJcGlkLlNldFNwZWVkID0gMC4wOwoJcGlkLkFjdHVhbFNwZWVkID0gMC4wOwoJcGlkLmVyciA9IDAuMDsKCXBpZC5lcnJfbGFzdCA9IDAuMDsKCXBpZC52b2x0YWdlID0gMC4wOwoJcGlkLmludGVncmFsID0gMC4wOwoJcGlkLktwID0gMC4yOwoJcGlkLktpID0gMC4wMTU7CglwaWQuS2QgPSAwLjI7CglwcmludGYoIlBJRF9pbml0IGVuZCBcbiIpOwp9CgpmbG9hdCBQSURfcmVhbGl6ZShmbG9hdCBzcGVlZCkKewoJcGlkLlNldFNwZWVkID0gc3BlZWQ7CglwaWQuZXJyID0gcGlkLlNldFNwZWVkIC0gcGlkLkFjdHVhbFNwZWVkOwoJcGlkLmludGVncmFsICs9IHBpZC5lcnI7CglwaWQudm9sdGFnZSA9IHBpZC5LcCAqIHBpZC5lcnIgKyBwaWQuS2kgKiBwaWQuaW50ZWdyYWwgKyBwaWQuS2QgKiAocGlkLmVyciAtIHBpZC5lcnJfbGFzdCk7CglwaWQuZXJyX2xhc3QgPSBwaWQuZXJyOwoJcGlkLkFjdHVhbFNwZWVkID0gcGlkLnZvbHRhZ2UgKiAxLjA7CglyZXR1cm4gcGlkLkFjdHVhbFNwZWVkOwp9CgppbnQgbWFpbigpCnsKCXByaW50ZigiU3lzdGVtIGJlZ2luIFxuIik7CglQSURfaW5pdCgpOwoJaW50IGNvdW50ID0gMDsKCXdoaWxlIChjb3VudCA8IDEwMCkKCXsKCQlmbG9hdCBzcGVlZCA9IFBJRF9yZWFsaXplKDIwLjApOwoJCXByaW50ZigiJWZcbiIsIHNwZWVkKTsKCQljb3VudCsrOwoJfQoJLy8gcHJpbnRmKCJjb3VudDogJWRcbiIsIGNvdW50KTsKCS8vIHByaW50ZigiU2V0U3BlZWQ6ICVmXG4iLCBwaWQuU2V0U3BlZWQpOwoJLy8gcHJpbnRmKCJBY3R1YWxTcGVlZDogJWZcbiIsIHBpZC5BY3R1YWxTcGVlZCk7CgkvLyBwcmludGYoImVycjogJWZcbiIsIHBpZC5lcnIpOwoJLy8gcHJpbnRmKCJlcnJfbGFzdDogJWZcbiIsIHBpZC5lcnJfbGFzdCk7CgkvLyBwcmludGYoInZvbHRhZ2U6ICVmXG4iLCBwaWQudm9sdGFnZSk7CgkvLyBwcmludGYoImludGVncmFsOiAlZlxuIiwgcGlkLmludGVncmFsKTsKCS8vIHByaW50ZigiS3A6ICVmXG4iLCBwaWQuS3ApOwoJLy8gcHJpbnRmKCJLaTogJWZcbiIsIHBpZC5LaSk7CgkvLyBwcmludGYoIktkOiAlZlxuIiwgcGlkLktkKTsKCXJldHVybiAwOwp9Cg=="
		}	
	],
	"documentation_files": [],
	"source_language": "c",
	"output_model": "GRFN"
}'

Developing locally

To develop locally, do the following:

1. Have your automates environment set up and activated.
2. Set the FLASK_APP env variable to automates.apps.automates.automates. On a linux system, run export FLASK_APP=automates.apps.automates.automates
3. Run python3 -m flask run

The web app should now be running locally!

AutoMATES API deployment on carp

To redeploy the AutoMATES API on carp:

1. Log into carp
2. Run cd /usr/local/automates-app/automates/ which will take you to a clone of the AutoMATES repo.
3. Pull the newest version.
4. Rebuild the docker container via docker build -t automates/local .
5. Change directory into the automates app via cd automates/apps/automates
6. Run docker-compose -f /usr/local/automates-app/automates/automates/apps/automates/docker-compose.yaml up --build -d and this will rebuild and deploy the app.

Proxy setup on hopper

The AutoMATES API is run on our machine carp. This does not have access to external networks outside of UofA. To get external traffic to the carp machine, we have a proxy set up on the machine hopper (I think that's the correct machine?) that essentially forwards external traffic to the API on carp inside of the UofA network. Below details how the traffic forwarding and token authentication works on hopper using nginx.

** Note this is already done, these are just notes in case it ever needs to be re done**

Installation/running

Install nginx

sudo apt-get update
sudo apt-get install nginx

Disable the Default Virtual Host:

sudo unlink /etc/nginx/sites-enabled/default

Create the Nginx Reverse Proxy

sudo vi etc/nginx/sites-available/reverse-proxy.conf

Basic conf file that passes every request to whatever IP is placed in . In the config file on hopper, the IP is configured as carps IP.

server {
    listen 80;
    location / {
        proxy_pass http://<IP HERE>; 
    }
}

Enable the virtual host

sudo ln -s /etc/nginx/sites-available/reverse-proxy.conf /etc/nginx/sites-enabled/reverse-proxy.conf
service nginx configtest
service nginx restart

Auth

from here

Auth map file in sites-available called auth-keys.config:

map $http_apikey $api_client_name {
		default "";
		"SOME API KEY" "Name of users of key (i.e. 'test', 'UA', 'GE', etc.";
}

update original reverse-proxy.conf to look like:

include /etc/nginx/sites-available/api-keys.conf;

server {
    listen 80;
    location / {
        auth_request /_validate_apikey; # Calls out to a validate api key endpoint
        proxy_pass http://<IP HERE>;
    }

    # Handle outgoing API key validation call. This makes it such that nginx is handling the authentication of tokens. Could remove this and set up a different service to handle.
    location = /_validate_apikey {
        internal;

        if ($http_apikey = "") {
            return 401; # Unauthorized
        }
        if ($api_client_name = "") {
            return 403; # Forbidden
        }

        return 204; # OK (no content)
    }
}

Restart the service for authorization to take effect.

service nginx restart