Multi-Tenant Workload Aggregation: Interactive Simulation & Visualization Tool

Overview

This simulation and visualization is inspired by Andrew Warfield's talk at FAST '23: "Building and Operating a Pretty Big Storage System (My Adventures in Amazon S3)".

This simulation was created to provide visualization of workload aggregation for a guest lecture in the Aalto University course "Networking at Scale and Advanced Applications" on March 18, 2025, covering Autoscaling challenges of multitenant SaaS platforms. You can find the slides here.

• Multi-tenancy significantly reduces the peak-to-average ratio of overall system load
  • In general, systems must provision for peak demand, not average load to meet performance quality requirements
  • Lower peak-to-average ratios directly translate to fewer overprovisioned resources
  • This results in improved resource utilization and cost efficiency

Please note that this script contains many assumptions about the simulated workloads and doesn't necessarily match reality. The goal is to visualize the effects of workload aggregation on system load. The presentation contains more details about cases where workloads are correlated and don't follow the model used in this simulation.

Features of the simulation

• Interactive Visualization: Dynamically adjust the number of aggregated workloads (k × k grid)
  • Keyboard Controls:
    • Use arrow keys (↑/↓/←/→) to adjust the number of workloads
    • Toggle between visualization modes with Tab key
    • Save screenshots with s key
    • Regenerate workloads by pressing Space or Enter key
    • Press q or Esc key to exit
• Two Visualization Modes:
  • Workload Mode: Shows individual workloads and their aggregation
  • Overprovisioning Mode: Visualizes how the required overprovisioning factor decreases as more workloads are aggregated
• Batch mode:
  • Generate image files and exit

Installation

1. Install uv: https://docs.astral.sh/uv/getting-started/installation/

2. Run the script directly (dependencies will be automatically installed when using uv):

   ./workload-aggregation-visualizer.py

Usage

Run the script to start in UI mode, the console will contain information about keybindings:

./workload-aggregation-visualizer.py

pass --help to see complete set of command line options

Visualization example

Images were generated with this command:

./workload-aggregation-visualizer.py --output-dir examples --format svg --plot-overprovisioning --batch

(Some images are omitted to simplify the example.)

Presentation that includes this visualization

• Guest lecture for Aalto University course “Networking at Scale and Advanced Applications” on March 18, 2025 about Autoscaling challenges of multitenant SaaS platforms by Lari Hotari

Resources related to multi-tenant SaaS and economics of scale in cloud services

These resources aren't necessarily directly related to the visualization in this project. They were the inspiration to create this visualization as well as some of the resources for the related presentation. Some resources are not directly related to multi-tenancy, but instead focus on reliability patterns for large systems.

• Andy Warfield: Building and Operating a Pretty Big Storage System (My Adventures in Amazon S3), part "Individual workloads are bursty", presentation at FAST'23
• Marc Brooker: Surprising Scalability of Multitenancy, blog post 2023-03-23
• Marc Brooker: Surprising Economics of Load-Balanced Systems, blog post 2020-08-06
• Marc Brooker: Try again: The tools and techniques behind resilient systems, presentation at AWS re:Invent 2024
• Jack Vanlightly: On the future of cloud services and BYOC, blog post 2023-09-25
• Elhemali, M., Gallagher, N., Gordon, N., Idziorek, J., Krog, R., Lazier, C., Mo, E., Mritunjai, A., Perianayagam, S., Rath, T., Sivasubramanian, S., Sorenson III, J.C., Sosothikul, S., Terry, D., & Vig, A. (2022). Amazon DynamoDB: A scalable, predictably performant, and fully managed NoSQL database service. Retrieved from https://www.amazon.science/publications/amazon-dynamodb-a-scalable-predictably-performant-and-fully-managed-nosql-database-service
  • Conference presentation on YouTube
• Ed Huang’s blog post series related to TiDB
  • The Road To Serverless: Intro & Why
  • The Road To Serverless: Storage Engine
  • The Road To Serverless: Multi-tenancy
• Jack Vanlightly’s blog post series
  • The Architecture of Serverless Data Systems
• Povzner, A., Mahajan, P., Gustafson, J., Rao, J., Juma, I., Min, F., Sridharan, S., Bhatia, N., Attaluri, G., Chandra, A., Kozlovski, S., Sivaram, R., Bradstreet, L., Barrett, B., Shah, D., Jacot, D., Arthur, D., Dagostino, R., McCabe, C., Obili, M. R., Prakasam, K., Sancio, J. G., Singh, V., Nikhil, A., & Gupta, K. (2023). Kora: A Cloud-Native Event Streaming Platform for Kafka. Proceedings of the VLDB Endowment, 16(12), 3822-3834. https://doi.org/10.14778/3611540.3611567
• AWS blog posts by David Yanacek
  • Fairness in multi-tenant systems
  • Using load shedding to avoid overload
• Nelson Elhage: Graceful behavior at capacity, blog post 2023-08-07
• Talat Uyarer & Rishabh Kedia: Crafting an Autoscaler for Apache Beam in a High-Volume Streaming Environment, blog post 2024-02-05
• Jamie Turner & James Cowling:Five Principles of Good System Design, podcast discussion 2024-11-06

License

This project is licensed under the MIT License with attribution requirements - see the LICENSE file for details.

Author

Created by Lari Hotari