Integration Service for OpenReq classifier
This project is a simple IS for interfacing with OpenReq requirements classifier. Once launched locally or deployed remotely, it let you classify issues of GIT repositories.
In order to deploy the IS correctly you have to follow these steps:
Create GitHub App
Follow steps on GitHub to get a working GitHub App in your developer account. Point the following route to these endpoints:
- Homepage URL: SERVER_IP:PORT
- User authorization callback URL: SERVER_IP:PORT/auth
- Webhook URL: SERVER_IP:PORT/webhook
Note: When developing on your machine you can use
SERVER_IPfor all fields except webhook for which you need to use a connection forwarder like ngrok. More info on next section.
Create private key
A private key for the GitHub App is required in order to make calls server to server. In the app settings page go to section Private keys, click on Generate a private key and then copy it on your computer or server then set the proper field in configuration file to set the path accordingly.
Create config file
Once the app is created put a
config.py in the
app folder with these values compiled from the info you can get in the app settings page:
SECRET_KEY = "super secret key for the flask app" DEBUG = True TRAINING_TIMEOUT = 1800 CLASSIFICATION_TIMEOUT = 1800 CONFIDENCE_TRESHOLD = 50 LOG_PATH = "/var/log/is.log" LOG_FORMAT = "%(asctime)s - %(pathname)s:%(lineno)d - %(levelname)s - %(message)s" GITHUB_WEBHOOK_SECRET = "" GITHUB_CLIENT_ID = "" GITHUB_CLIENT_SECRET = "" GITHUB_APP_ID = "" GITHUB_PERSONAL_ACCESS_TOKEN = "" GITHUB_PRIV_KEY_PATH = "path to private key pem file" OPENREQ_BASEURL = "OPENREQHOST:port" SQLALCHEMY_DATABASE_URI = 'sqlite:////path/to/a/db/file.db' SQLALCHEMY_TRACK_MODIFICATIONS = False CELERY_BROKER_URL = 'REDISHOST:port'
Once requirements are met you can choose to deploy using two different setups based on your needs.
Remote setup - ansible
Remote setup is ideal for production environment when you can configure a dedicated host.
Prepare an host with ssh installed and an ansible user.
Create a file named "hosts" in the root of the ansible playbook and change
SERVER_IP with the right IP address
then type in the terminal:
$ ansible-playbook -i hosts main.yml
Note: Default configuration install app in the root of nginx
Local setup - docker
Docker setup is the best for development projects. To get a working docker image with the flask project build first the image with:
$ docker build -f docker/Dockerfile -t is:latest .
Then launch from your terminal using the docker-compose provided:
$ docker-compose up
And you will get a working version running on port 5000 with app folder mounted in /app and a complete environment with redis db and celery workers for async tasks.
Note: Remove debug flag in config.py for production (you will loose reloading feature on flask).
Setup ngrok port forwarding for webhook
Follow the good guide at GitHub to install and setup ngrok. This way you can forward remote calls to your localhost endpoints without the need to use dynamic DNS or port forwarding or proxy.
Note: Just remember to update SERVER_IP:PORT configuration for webhooks in your GitHub App settings page
IS app use asynchronous tasks in order to avoid timeout when calling training and classification endpoints. Celery framework is correctly configured in docker-compose but you can always start a new worker instance issuing the command directly from the app folder:
celery -A app.celery worker -l debug
Note: If you want to monitor Celery tasks from a web interface install Celery Flower