This is a demo showing an FME Engine running in a docker container listening on a socket for jobs. The container mounts a directory containing a single workspace called AirQualityReport.fmw. This workspace has a published parameter Location which is the name of a city. It outputs an html page with the latest data on the air quality for that city as well as a google map containing data points on air quality throughout the city.
The FME Engine listens on a port awaiting jobs. A python script is used to send jobs to that engine which will then run the translation using the latest data and output an updated html page.
The html page is then served by an apache server that is also running in a docker container.
- Docker 1.12
- Python
This requires docker 1.12 as it uses the new service mode. This currently has release candidates but is not officially released. This will also run on the Docker for Mac or Docker for WIndows Beta as they use Docker 1.12.
Start the Docker containers. On Linux and mac, you can run the start_containers.sh script. On Windows you can run start_containers.ps1. This will launch two docker services. One is the FME Engine listening for jobs on port 7777, and one is the apache server serving html files on port 80.
Submit a job to the FME engine. You can submit a job by running submit_job.sh on Linux and Mac or submit_job.ps1 on Windows. This will run the python script FMEEngineClient.py to submit a job to the FME Engine listening on localhost on port 7777 for the city of Vancouver. After a few seconds you should see a message returned with "Translation Successful". The FME Engine will have read the latest data from a web service and generated an HTML page and placed it in the results folder which is also the directiry being served by apache.
You can then open a browser to your machine on port 80. You should see an index listing with a single html file called AirQualityReportVancouver.html that was generated by the engine. Click on it to view the report.
Now imagine we want to generate a report for several cities at the same time. We will need more than one engine to take care of this unless we want to wait a while.
Scale the FME Engines by running the command docker service scale fme_engine=10. This will launch more FME Engine containers waiting for jobs. They are all listening on the same port, but Docker will take care of routing requests to the local machine on that port to each of the containers using a round robin scheduling strategy.
Run the script submit_lots_of_jobs.sh (or submit_lots_of_jobs.ps1 on windows). This will submit around 30 jobs all at once to that port. The engine containers will run the jobs in parallel and will each generate a report.
Refresh the page being served on your localhost to see the additional reports that were generated. Click on them to view.
These containers are all running on your single host, but it isn't that much of a stretch to imagine these containers spread over multiple hosts, able to be scaled when required.
After you are done you can stop the services by running stop_containers.sh or stop_containers.ps1.