SimFaaS: A Serverless Performance Simulator

This is a project done in PACS Lab aiming to develop a performance simulator for serverless computing platforms. Using this simulator, we can calculate Quality of Service (QoS) metrics like average response time, the average probability of cold start, average running servers (directly reflecting average cost), a histogram of different events, distribution of the number of servers throughout time, and many other characteristics.

The developed performance model can be used to debug/improve analytical performance models, try new and improved management schema, or dig up a whole lot of properties of a common modern scale-per-request serverless platform.

Artifacts

• PyPi Package
• Github Repo
• ReadTheDocs Documentation (PDF)
• Examples (MyBinder Jupyter Lab)
• Jupyter Notebook Docker Image

Requirements

• Python 3.6 or above
• PIP

Installation

Install using pip:

pip install simfaas

Upgrading using pip:

pip install simfaas --upgrade

For installation in development mode:

git clone https://github.com/pacslab/simfaas
cd simfaas
pip install -e .

And in case you want to be able to execute the examples:

pip install -r examples/requirements.txt

Running in Docker

To ease the process of installation and experimenttion with SimFaaS, we developed a docker image extending the Jupyter Notebook Data Science Stack. The resulting docker image is also available publicly on Docker Hub.

The only requirement for running the jupyter notebook stack is docker which can easily be installed:

sudo apt-get update && sudo apt-get -y install docker.io

docker ps
sudo docker ps

sudo usermod -aG docker $USER
sudo chown "$USER":"$USER" /home/"$USER"/.docker -R
sudo chmod g+rwx "/home/$USER/.docker" -R
sudo chown "$USER":"$USER" /var/run/docker.sock
sudo chmod g+rwx /var/run/docker.sock -R
sudo systemctl enable docker

To run the jupyter lab in the current directory, simply run the following command:

IMAGE_NAME=nimamahmoudi/jupyter-simfaas # or $(cat .dockername) if in root folder of the github repo
TARGET_PORT=8888 # The port on which the jupyter notebook will run

docker run -it --rm \
    -p $TARGET_PORT:8888 \
    -e JUPYTER_ENABLE_LAB=yes \
    --name jpsimfaas \
    -v "$(pwd)":/home/jovyan/work \
    $IMAGE_NAME

The container logs will contain the token you need to log into your jupyter lab session.

Usage

A simple usage of the serverless simulator is shown in the following:

from simfaas.ServerlessSimulator import ServerlessSimulator as Sim

sim = Sim(arrival_rate=0.9, warm_service_rate=1/1.991, cold_service_rate=1/2.244,
            expiration_threshold=600, max_time=1e6)
sim.generate_trace(debug_print=False, progress=True)
sim.print_trace_results()

Which prints an output similar to the following:

100%|██████████| 1000000/1000000 [00:42<00:00, 23410.45it/s]
Cold Starts / total requests:	 1213 / 898469
Cold Start Probability: 	     0.0014
Rejection / total requests:      0 / 898469
Rejection Probability: 		     0.0000
Average Instance Life Span:      6335.1337
Average Server Count:  		     7.6612
Average Running Count:  	     1.7879
Average Idle Count:  		     5.8733

Using this information, you can predict the behaviour of your system in production.

Development

In case you are interested in improving this work, you are always welcome to open up a pull request. In case you need more details or explanation, contact me.

To get up and running with the environment, run the following after installing Anaconda:

conda env create -f environment.yml
conda activate simenv
pip install -r requirements.txt
pip install -e .

After updating the README.md, use the following to update the README.rst accordingly:

bash .travis/readme_prep.sh

Examples

Some of the possible use cases of the serverless performance simulator are shown in the examples folder in our Github repository.

License

Unless otherwise specified:

MIT (c) 2020 Nima Mahmoudi & Hamzeh Khazaei

Citation

You can find the paper with details of the simultor in PACS lab website. You can use the following bibtex entry for citing our work:

@inproceedings{mahmoudi2021simfaas,
  author={Mahmoudi, Nima and Khazaei, Hamzeh},
  title={{SimFaaS: A Performance Simulator for Serverless Computing Platforms}},
  year={2021},
  publisher = {Springer},
  booktitle={{International Conference on Cloud Computing and Services Science}},
  pages={1--11},
  numpages = {11},
  keywords = {performance modelling, serverless computing, serverless, simulator, performance},
  series = {CLOSER '21},
  url_paper={},
  url_pdf={https://pacs.eecs.yorku.ca/pubs/pdf/SimFaaS_CLOSER2021_Website_Preprint.pdf}
}

@misc{mahmoudi2021simfaaspre,
  title={{SimFaaS: A Performance Simulator for Serverless Computing Platforms}},
  author={Nima Mahmoudi and Hamzeh Khazaei},
  year={2021},
  eprint={2102.08904},
  archivePrefix={arXiv},
  primaryClass={cs.DC},
  url_paper={https://arxiv.org/abs/2102.08904}
}