Create a Linux Virtual Machine in Azure and retrieve systemd journal logs with a RESTful web API.
The web API is used to provision a Virtual Machine (and accompanying resource group, network stack, and custom script extension). The API will record the created VM, and the custom script extension will stream systemd journal entries from the provisioned VM back to the web API. The end user will only have to initiate HTTP requests to the web service to retrieve these systemd boot events, optionally filtering by unit.
- Azure App Service Plan on Linux
- Azure App Service (Docker container)
- Azure Database for PostgreSQL
- Azure Linux VM (target provisioned machines)
The web API is developed in Python using Flask.
In order to maintain the state for provisioned VMs and their systemd journal events, a PostgreSQL database is used. The database primarily consists of two main tables: One for managing the VMs themselves, and the other for recording the systemd events. This software is designed to keep the raw events so that we have the original data. For data aggregations, the API can be consumed or developed further to provide that functionality. At the moment, the API only supports filtering the messages by systemd unit.
This architecture relies heavily on the custom script extension for a Linux VM. I chose a "push" method for the systemd journal events, as this allowed a natural flow from source to destination. The custom script extension does two things:
- On startup it'll pass all of the previous journal log entries from current boot to now to the Flask web API
- Wait every 5 seconds, and then retrieve and pass the last 5 seconds of journal events to the Flask web API
This allows a fairly constant stream of journal events from the VM to the API. 5 seconds appears to be a good amount of delay time so that the end user isn't waiting too long for log messages (close to real time), but also not too short where the VM is too chatty with the API.
From the start of development, a container was used for the Flask web API. This allowed for a consistent experience, from developing on my workstation to deploying to Azure.
[GET] /- Default route displaying the application name[POST] /vm/<name>/<size>- Create a virtual machine withnameandsize(options aresmall,medium, andlarge)[GET] /vm- List all provisioned virtual machines[GET] /vm/<name>- Get a virtual machine and specs[GET] /vm/<name>/boot- Get all systemd events for the VMname[GET] /vm/<name>/boot/<unit>- Get systemd events for VMnamefor a particularunit[POST] /vm/<name>/boot- Add a systemd event to the API (Note: this is reserved for provisioned VM usage)