The VPLBDX project is an alternative execution sandbox to the VPL-Jail-System provided for the VPL Moodle plugin. This solution is based on docker swarm, docker containers and a python proxy allowing interactive execution both textual and graphical, and non-interactive execution for code evaluation purpose.
For more details about VPL, visit the VPL home page. The main benefit of this project is to be able to define a personalised sandbox environment for each VPL activity based on a minimal docker image called gblin/minivpl.
Since VPLBDX relies only on docker, it should only requires a system able to run docker.
In order to take into account different use case, we designed a solution based on docker swarm that deploys jails as services in order to be able to scale up to your needs.
Roughly, in a docker swarm, you may use as much (or as few) machine that you want in a transparent manner. One of them will be the swarm manager and the others will be workers. Services will be deployed on all the machine of the swarm and will be able to communicate through an overlay network.
The proxy will be in charge of receiving VPL requests and transfering - using load balancing - the request to one of the vpljail service running on the swarm. Including a specific execution file called vplbdx.conf in the VPL activity (details provided afterwards), the corresponding assigned vpljail service will pull - if needed - the requested docker image from docker hub on the swarm and will execute the VPL executions files into this last ; allowing fine tuning of each VPL activity.
On each machine you want to use in the swarm, you should install docker.
In the following, we consider that the manager machine is called manager and the workers are worker1 and worker2. On manager, get the repository content:
git clone https://github.com/GuillaumeBlin/vplbdx.git Edit the .env file at the root of the project
NB_REPLICAS=4 #Number of replicas of the jail service
#Optionnal TMPER_PORT=8088 #Port number of a tmper service for temporary storage service
#Optionnal SLPROXY_PORT=8087 #Port number of a sharelatex service
PROXY_PORT=8090 #Socket port number to listen for secure connections (https: and wss:) for the proxy
#Optionnal PROXY_MOODLE_PORT=443 #Port of a moodle proxy
#Optionnal PROXY_MOODLE_PATH=/mod/vpl/ws #Path of the moodle proxy
MAXTIME=600 #Maximum time for a request in second
MAXFILESIZE=64000000 #Maximum file size in bytes
MAXMEMORY=2000000 #Maximum memory size in bytes
MAXPROCESSES=500 #Maximum number of processes
MAXBODYSIZE=1024**2*5 #Maximum size of any html request between moodle platform and the jail serviceThe manager machine should have two files regarding your certificate with the following exact path
/vplbdx/ssl/secure.crt # Your ssl_certificate
/vplbdx/ssl/secure.key # Your ssl_certificate_keySelf-signed certificates can be created using openssl
openssl req -new -x509 -nodes -sha256 -days 365 -subj '/CN=MANAGER_IP' -key /vplbdx/ssl/secure.key -out /vplbdx/ssl/secure.certwhere MANAGER_IP is the IP of the manager
Then call the deploy script passing as arguments workers hostnames
./deploy.sh worker1 worker2Everything should be up and running with the current host as manager and worker1 and worker2 as workers. By default, 4 replicas of the service vplbdx are launched on the swarm. You may tune this number by replacing 4 in the line NB_REPLICAS=4 of the .env file by the desired number of replicas.
Just run the stopservices.sh script on the manager that will make all nodes leaving the swarm, remove the overlay network and stop all services.
./stopservices.shWe recommand you to set up a proxy on the moodle server that will be the only one communicating with the VPLBDX jail system (see afterwards).
Otherwise, you just need to set the URL of the execution server in the general module configuration or in the local execution server settings of your Moodle server to
https://MANAGER_IP:PROXY_PORT/where MANAGER_IP is a public ip to access the manager machine and PROXY_PORT is the value you set in the .env file.
Note that for security purpose, only secured connections are allowed one you use VPLBDX jail system.
In the VPL architecture, the client is connecting directly to the Jail system using websockets. For ease of lisibility, security and simplicity, as we pointed out, we recommand setting a proxy on the moodle server in order to have a uniq point of communication (the client will only refers to your moodle platform). You may then forbid any connection to your manager machine other than the one from your moodle platform.
Moodle platform is often deployed combined with an apache proxy. Configuring this one may be painful regarding websocket handling. This is why we suggest that you install nginx on your moodle server and put a web proxy up and running on a given port (4443 here for example) using the following nginx.conf file:
worker_processes auto;
events {
worker_connections 768;
}
http {
sendfile on;
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
server {
client_body_buffer_size 500M;
client_max_body_size 500M;
listen 4443 ssl;
ssl_certificate secure.crt;
ssl_certificate_key secure.key;
error_log /var/log/nginx.log error;
location /PROXY_MOODLE_PATH {
set $upstream MANAGER_IP:PROXY_MOODLE_PORT;
proxy_pass https://$upstream;
proxy_redirect off;
proxy_set_header Host $host;
proxy_http_version 1.1;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_ignore_client_abort on;
# VNC connection timeout
proxy_read_timeout 900s;
# Disable cache
proxy_buffering off;
}
}
}Once again, you will have to replace MANAGER_IP and PROXY_MOODLE_PORT such that MANAGER_IP is an ip for manager machine that is accessible only (if you want to straighten the security - accessible if you do not care) from the moodle server and PROXY_PORT is the value you set in the .env file. Moreover, your certifacte files (here secure.crt and secure.key) should be present in /etc/nginx/ folder.
The URL of the execution server in the configuration will then be
https://MOODLE_IP:4443/PROXY_MOODLE_PATHwhere MOODLE_IP is the public ip to access to your moodle platform (note that the port 4443may be changed by modifying listen 4443 ssl; line accordingly).
When designing your VPL activity on your moodle platform, just add an execution file called vplbdx.cfg which should contain a line in the following format
DOCKER=<image-name>where <image-name> is the name of a docker image available on the docker hub.
For example:
DOCKER=gblin/minivpljavaThe corresponding docker image will be pulled from the local registry if it exists. If not it will be first pull from the Docker Hub. If you need to force the pulling from the Docker Hub (e.g. for update purpose of the image), you can add the following line to your vplbdx.cfg file
FORCEPULLDo not let the FORCEPULL option while in production mode since it may induce network latencies
For sake of performance, a part of the jail service should be embedded into the corresponding image - namely, a websocket proxy able to pipe the moodle VPL term.js client to the shell of the docker and a websocket proxy handling noVNC communications.
We built a minimal docker image, based on bitami/minideb:stretch, called gblin/minivpl that embeds wss-shell (a standalone and secured version of websocket-shell-service project based on Node.js), websockify proxying a tightvncserver.
The simple manner is thus to build your own docker image inheriting from gblin/minivpl one.
For example, gblin/minivpljava docker image is defined only with the following Dockerfile :
FROM gblin/minivpl
RUN install_packages openjdk-8-jdk junit openjfx
RUN install_packages locales-all libva-glx1 If you want to do it "your way", you need to design a docker image that will have at least two bash scripts /vplbdx/start-vncserver.sh and /vplbdx/vpl_terminal_launcher.sh that will be in charge respectively of handling noVNC and term.js requests using websockets. A nice starting point would be to study the gblin/minivpl source files provided in misc/docker-images/minivpl directory for your information.
You can obtain logs of the different dockers using docker exec <container-id> cat <path_to_log_file> command.
For proxy, were <tab> is a tabulation key pressed to complete container name:
docker exec moodpy_proxy.1<tab> cat /var/log/nginx.logFor jails, were <num> is the number of the replica of the service (e.g. 1, 2, ...) and <tab> is a tabulation key pressed to complete container name:
docker exec moodpy_vplbdx.<num>.<tab> cat WS.log HTTP.log