The following information provides enough detail to install and deploy wasm-joey on an Amazon EC2 instance with:
- Additional storage device
- MySQL Database; including user, tables, sample data etc.
- All relevant dependencies; CORS, SSL etc.
If installing on alternative infrastructure where you are provided with root access, please create a new group and user like this
groupadd -g2021 ubuntu
useradd ubuntu -g ubuntu -s "/bin/bash" -d "/home/ubuntu" -m
Set new user's password
passwd ubuntu
Add sudo privileges for the new user
adduser ubuntu sudo
Change to new user
su ubuntu
There is a feature enhancement/issue which requests that the following commands are all written into a bash script. This will make it much quicker to install.
To begin, create a new EC2 Ubuntu Server instance which has an additional SSD i.e. m5d.large (which has 1 x 75 NVMe SSD).
sudo apt-get update
sudo apt-get -y upgrade
sudo apt-get install -y build-essentialView NVMe volume (which is not yet mounted/mapped/formatted)
lsblkThere is a ~300Gb drive with the name of nvme0n1
nvme0n1 259:2 0 279.4G 0 disk Create a file system
sudo mkfs -t ext4 /dev/nvme0n1 (on Azure it would be something like sudo mkfs -t ext4 /dev/sdb)
Part of the output from the above mkfs command will include the Filesystem UUID. Cut and paste this UUID because it will be used in an upcoming command.
Filesystem UUID: 6f6177fe-947a-46a2-b593-c36dfaaa8407Create an easily accesible mount point on the main drive (where the operating system runs) and then set the permissions of this mount point to the ubuntu user.
sudo mkdir /media/nvme
sudo chown -R $USER:$USER /media/nvme/Ensure that this drive is mounted each time the system is restarted. Add this line to the /etc/fstab file (remember the UUID from the previous step?).
UUID=37c7e8b0-a595-4046-9a43-5e7928a4a15a /media/nvme ext4 defaults 0 0Once the above commands have succeeded, reboot the instance.
sudo shutdown -r nowAfter the reboot, see the mounted ~300Gb NVMe SSD using the df command
df -h
/dev/nvme0n1 275G 65M 260G 1% /media/nvmeEnsure that the /media/nvme directory is owned by ubuntu by typing ls -la /media/nvme If it is not then type the following command
sudo chown -R $USER:$USER /media/nvme/Just a quick word about Rust; it is suggested that you use the SSD mount point because the .rustup folder can get quite large and max out disk space. To install Rust on the SSD just put these two lines in your ~/.profile file before you install Rust (using the standard command)
vi ~/.profile
Add these lines
export CARGO_HOME="/media/nvme"
export RUSTUP_HOME="/media/nvme"
Execute profile
source ~/.profile
Once you have performed the config above, please then follow these official Rust installation instructions to install Rust. Essentially running the following command should do it.
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
Create dir to house the application
mkdir /media/nvme/node_rpcAlter permissions
sudo chown -R $USER:$USER /media/nvme/node_rpc
Fetch the application code
cd /media/nvme/node_rpc
git clone https://github.com/second-state/wasm-joey.git
cd /media/nvme/node_rpc/wasm-joey/srcFetch
curl -sL https://deb.nodesource.com/setup_15.x | sudo -E bash -Install Node.js on the system
sudo apt-get install -y nodejsNOTE If you ever need to unistall and reinstall all of the node dependencies you can use this automated_node.sh script. Just sudo chmod a+x automated_node.sh then ./automated_node.sh.
The following are instruction on installing all of the node dependencies manually.
Content Type
npm install file-type
Axios
cd /media/nvme/node_rpc/wasm-joey/src
npm install axiosNode cache
cd /media/nvme/node_rpc/wasm-joey/src
npm install node-cacheuuid
cd /media/nvme/node_rpc/wasm-joey/src
npm install uuidUrllib
cd /media/nvme/node_rpc/wasm-joey/src
npm install urllibDotenv
cd /media/nvme/node_rpc/wasm-joey/src
npm install dotenvExpress
cd /media/nvme/node_rpc/wasm-joey/src
npm install expressBody parser
cd /media/nvme/node_rpc/wasm-joey/src
npm install body-parserMySQL driver
cd /media/nvme/node_rpc/wasm-joey/src
npm install mysqlCORS
cd /media/nvme/node_rpc/wasm-joey/src
npm install corsHTTPS
cd /media/nvme/node_rpc/wasm-joey/src
npm install httpsFormidable
cd /media/nvme/node_rpc/wasm-joey/src
npm install formidablecd /media/nvme/node_rpc/wasm-joey/src
npm install buffer-string-to-arraycd /media/nvme/node_rpc/wasm-joey/src
npm install randomstringcd /media/nvme/node_rpc/wasm-joey/src
npm install express-rate-limittimedatectl set-timezone Australia/BrisbaneYou can find your own timezone using the following command
timedatectl list-timezones | grep Los_Angeles
# returns America/Los_AngelesInstall MySQL on the system
sudo apt-get install -y mysql-serverCreate dir to house the database and update the default MySQL config
mkdir /media/nvme/joey_database
sudo chown -R mysql:mysql /media/nvme/joey_database/
sudo chmod 750 /media/nvme/joey_database/sudo apt-get -y remove --purge mysql*
sudo rm -rf /etc/mysql /var/lib/mysql
sudo apt-get -y autoremove
sudo apt-get -y autoclean
sudo apt install -y mysql-server
echo 'datadir = /media/nvme/joey_database' | sudo tee -a /etc/mysql/mysql.conf.d/mysqld.cnf
echo 'max_allowed_packet = 128M' | sudo tee -a /etc/mysql/mysql.conf.d/mysqld.cnf
echo 'wait_timeout = 28800' | sudo tee -a /etc/mysql/mysql.conf.d/mysqld.cnf
sudo mysqld --initialize --user=mysql
sudo /etc/init.d/mysql start
sudo mysql
Create new user and database for the application
CREATE DATABASE joeydb;
CREATE USER 'joey'@'localhost' IDENTIFIED BY 'your_password_here';
GRANT ALL PRIVILEGES ON joeydb . * TO 'joey'@'localhost';
FLUSH PRIVILEGES;Request to use this newly created database
use joeydb;Create blank tables for the application to use
CREATE TABLE wasm_executables(
wasm_id INT(6) NOT NULL AUTO_INCREMENT,
wasm_description CHAR(255) NOT NULL,
wasm_binary LONGBLOB NOT NULL,
wasm_state LONGTEXT NOT NULL,
storage_key LONGTEXT NOT NULL,
wasm_callback_object JSON NOT NULL,
usage_key binary(36),
admin_key binary(36),
PRIMARY KEY(wasm_id)
);Create a blank table which will store the logs for particular wasm_id
CREATE TABLE wasm_execution_log(
log_id INT(6) NOT NULL AUTO_INCREMENT,
wasm_executable_id INT(6) NOT NULL,
wasm_executable_state LONGTEXT NOT NULL,
execution_timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
execution_object LONGTEXT NOT NULL,
INDEX we_index (wasm_executable_id),
PRIMARY KEY(log_id),
FOREIGN KEY (wasm_executable_id) REFERENCES wasm_executables(wasm_id) ON DELETE CASCADE
);Alter root user
ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'your_password_here';
FLUSH PRIVILEGES;Then exit
quitConfigure Ubuntu to allow new MySQL directory
sudo vi /etc/apparmor.d/tunables/aliasAdd the following line
alias /var/lib/mysql/ -> /media/nvme/joey_database,Then restart AppArmor
sudo systemctl restart apparmorTighten MySQL security
sudo mysql_secure_installation utilitysudo apt-get update
sudo apt-get -y upgrade
npm install helmet
sudo apt-get install -y certbot
sudo certbot certonly --manualRun the following command to enable sufficient permissions for the files that certbot created on our behalf
sudo chown $USER:$USER -R /etc/letsencryptCreate the infrastructure for the permanent storage by following these rust_native_storage_library instructions specifically the Installing database (RocksDB) section at the bottom of the aforementioned link.
sudo apt-get install libboost-all-devsudo apt-get install -y llvmsudo apt-get install -y liblld-10-devsudo apt-get install -y libstdc++6sudo apt-get install -y g++cd /media/nvme/node_rpc/wasm-joey/srcexport CXX=g++-9You will need to alter the Git configuration on the machine where this installation procedure is being performed. Reason being, this machine will not have the SSH keys to communicate with git@github. If you add the following config, you will be able to successfully run the npm install --build-from-source ... below.
git config --global url."https://github.com/".insteadOf git@github.com:
git config --global url."https://".insteadOf git://You can check that this config worked by typing
git config -lYou can reverse/revert this Git config by typing git config --global --edit and then commenting out the appropriate sections.
cd /media/nvme/node_rpc/wasm-joey/srcPlease visit this link which details the requirements to continue and ensure that your system has these required dependencies before proceeding.
The following installation will require that npm is downgraded to 6.14.9. There is an npm cli issue which prevents us from using the latest npm for this particular build from source task.
Because of the complexity of dependency management, please install aptitude because it provides a way to automatically resolve depencency conflicts.
sudo apt install aptitude
sudo aptitude install npm
The following command is used to alter the npm version.
sudo npm install -g npm@6.14.9
Once you have temporarily downgraded npm, please go ahead and install the latest wasmedge-extensions like this
cd /media/nvme
git clone --recurse-submodules https://github.com/second-state/wasmedge-extensions.git
cd /media/nvme/wasmedge-extensions
npm install
cd /media/nvme/node_rpc/wasm-joey/src
npm install --build-from-source /media/nvme/wasmedge-extensionsIf getting the error does not have permission to access the dev dir "/root/.cache/node-gyp then try the following.
Make a new global cache dir and update profile i.e.
mkdir ~/.npm-global
npm config set prefix '~/.npm-global'Then update your ~/.profile file
export PATH=~/.npm-global/bin:$PATHThen run source ~/.profile command
Then run the commands again.
If you receive an error about permissions such as this Missing write access to /usr/local/lib/node_modules then please run the following command and then try the npm install --build-from-source command, outlined above, again.
sudo chown -R $USER /usr/local/lib/node_modules
Following the successful installation of the above, please go ahead and run the following command to return npm to the latest version
npm install npm@latest -g
Please note, if this is not a new installation (i.e. you are upgrading extensions) please delete all .so files from the /media/nvme/aot section and ensure that the manifest.txt file is blank/empty before restarting the FaaS.
Gas and invocation details are gathered for all wasm executables. Please create the following directory and adjust the permissions as follows.
mkdir /media/nvme/usage
sudo chown -R $user:$user /media/nvme/usage
Also be sure that this path is present in the /media/nvme/node_rpc/wasm-joey/src/.env configuration file as follows
usage_dir=/media/nvme/usage
If you want to turn usage statistics off and on, open the /media/nvme/node_rpc/wasm-joey/src/.env file and update the value for measure_gas_and_invocations i.e. use 0 for off and 1 for on.
To stop/pause gathering statistics, turn usage off like this.
measure_gas_and_invocations=0
To start/recommence gathering statistics, turn usage on like this.
measure_gas_and_invocations=1
WasmEdge generates AOT files which need to be stored on the solid state file system. Please create the following directory and adjust the permissions as follows.
mkdir /media/nvme/aot
sudo chown -R $user:$user /media/nvme/aot
Also be sure that this path is present in the /media/nvme/node_rpc/wasm-joey/src/.env configuration file as follows
aot_dir=/media/nvme/aot
Lastly, create a blank file which will be the future manifest of all wasm_ids and AOT file paths. Each time the system restarts or Joey is restarted we will need to import the wasm_id and AOT file paths.
touch /media/nvme/aot/manifest.txt
Obviously if you ever need to flush this manifest you would just remove and recreate a blank file
Fetch the code
mkdir /media/nvme/AIaaS
sudo chown -R $USER:$USER /media/nvme/AIaaS/
cd /media/nvme/AIaaS/
git clone https://github.com/second-state/AI-as-a-Service.git
Compile the necessary binaries to /usr/bin/ (/usr/bin is default setting as per the Cargo.toml's install settings)
rustup update nightly
rustup update stable
cd /media/nvme/AIaaS/AI-as-a-Service/native_model_zoo/http_proxy
cargo install --path .
Ensure that all of those binaries are available in the user (who is running Joey's path).
Set the system path so it can find the binaries via vi ~/.profile (then log out and back in again as shown above)
export PATH="/usr/bin:$PATH"
export PATH="/media/nvme/AIaaS/AI-as-a-Service/native_model_zoo/http_proxy/target/release:$PATH"
Follow these instructions to install tesseract OCR. If you would like to add extra languages (other than the English default) then download this repo, copy all of the .traineddata files to the /usr/share/tesseract-ocr/4.00/tessdata folder and then add this line to your ~/.profile file export TESSDATA_PREFIX="/usr/share/tesseract-ocr/4.00/tessdata".
Install translation software using the following command
sudo apt install translate-shell
Open the .env file and ensure that the base domain name is correct i.e.
vi /media/nvme/node_rpc/wasm-joey/src/.envserver_name=dev.rpc.ssvm.secondstate.ioOr
server_name=rpc.ssvm.secondstate.ioetc.
If you want to turn usage statistics off and on, open the /media/nvme/node_rpc/wasm-joey/src/.env file and update the value for measure_gas_and_invocations i.e. use 0 for off and 1 for on.
For example, to stop/pause gathering statistics, turn usage off like this.
measure_gas_and_invocations=0
To start/recommence gathering statistics, turn usage on like this.
measure_gas_and_invocations=1
cd /media/nvme/node_rpc/wasm-joey/src
nodejs server.jsThis section shows you how to build, compile, deploy and call a FaaS.
There are many pre-made functions that you can implement. Perhaps the simplest of them all is this hello world FaaS.
At present you can access the server logs via this URL
To set up the logging please install the following node package
npm i frontail -g
Then instead of just starting using forever start server.js use this more verbose command that creates a deliberate log file for frontail to access
forever start -o out.log -e err.log server.js
Then start frontail and allow it to serve the out.log file
nohup frontail out.log &
Node.js will store temporary files in /tmp as upload_*
Create a shell script with the following syntax in /home/ubuntu/purge_temp.sh
It will delete all tmp files which are older than 5 minutes
#!/bin/bash
find /tmp/upload_* -mmin +5 -type f -exec rm -fv {} \;
Then create the following cron which will execute once per minute
* * * * * /home/ubuntu/purge_temp.sh
If you would like to rate limit usage please perform the following steps
npm install --save express-rate-limit
Then add the following to the top of your server.js file
const rateLimit = require("express-rate-limit");
const limiter = rateLimit({
windowMs: 1 * 60 * 1000, // 1 minute
max: 2 // limit each IP to 2 requests per windowMs
});
// apply to all requests
app.use(limiter);
This is the base installer for the NVIDIA CUDA toolkit
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin
sudo mv cuda-ubuntu2004.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/11.1.0/local_installers/cuda-repo-ubuntu2004-11-1-local_11.1.0-455.23.05-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu2004-11-1-local_11.1.0-455.23.05-1_amd64.deb
sudo apt-key add /var/cuda-repo-ubuntu2004-11-1-local/7fa2af80.pub
sudo apt-get update
sudo apt-get -y install cuda
Follow these instructions on CUDA and FFMPEG
https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html
https://docs.nvidia.com/video-technologies/video-codec-sdk/ffmpeg-with-nvidia-gpu/index.html
Be sure to modify the configure file by updating any instances of compute_30 with compute_35
Be sure to use the following configure command for ffmpeg
./configure --enable-cuda-nvcc --enable-cuvid --enable-nvenc --enable-nonfree --enable-libnpp --extra-cflags=-I/usr/local/cuda-11.1/include --extra-ldflags=-L/usr/local/cuda-11.1/lib64
The following package will allow you to monitor GPU and Memory whilst executing Wasm executables
sudo apt-get install -y python3-pip
sudo pip3 install glances[gpu]
To run glances, use
sudo glances