Queue Simulator

goqueuesim

Table of Contents

• Introduction
• Usage
• Notable Feature Gaps
• Contributing

Introduction

Simulator built at Shopify originally purposed to prototype different Checkout Queue algorithms.

Open-sourced at similar low-fidelity to its prototypical origins but readily-actionable for improved usability.

🔗   Requirements

First, install project Go dependencies to your local environment:

go mod download

If you plan to run with lua-driven queues in Redis (ie. queue_type = "lua_driven_bins_queue" in config.go), you'll need to install Redis and start a server in the background.

The above step is not necessary for queue types other than lua_driven_bins_queue.

Dashboards

For dashboard statistics, this project hopes to eventually leverage dockerized Prometheus and Grafana.

However, for the moment, its statistics are built around Datadog so you'll need a Datadog API Key.

Once you have one ready, create a .env file with the following:

DD_API_KEY=XXXX
DD_DOGSTATSD_NON_LOCAL_TRAFFIC=true

and then use Docker to kickoff the Datadog agent:

docker-compose up
make

📖  Usage

For now, project usage is primarily driven by a Makefile with the following commands:

Make Task	Description
default	Compile the project binary executable & run it
run	Run the last compiled project binary executable
clean	Remove the last compiled project binary executable
test	Run tests

Running a simulation is thus as simple as executing make (build & execute) or make run (execute last build).

Experiment Configuration

Experiment configuration (e.g. algorithm used, client behaviour, etc.) can be modified under cmd/goqueuesim/config.go. Available parameters are documented in that file.

Additionally, client behaviour can be specified via JSON under config/simulation/client_distributions/.

A simple example config with 3 different client types might look like:

  {
    "representation_percent": 0.20,
    "client_type": "routinely_polling_client",
    "humanized_label": "fast_poller",
    "obeys_server_poll_after": false,
    "max_initial_delay_ms": 20000,
    "max_network_jitter_ms": 100,
    "custom_int_properties": {
      "dflt_poll_interval_seconds": 1
    }
  },
  {
    "representation_percent": 0.60,
    "client_type": "routinely_polling_client",
    "humanized_label": "strictly_obedient_poller",
    "obeys_server_poll_after": true,
    "max_initial_delay_ms": 20000,
    "max_network_jitter_ms": 450
  },
  {
    "representation_percent": 0.20,
    "client_type": "fully_disappearing_client",
    "humanized_label": "immediately_exiting_client",
    "obeys_server_poll_after": false,
    "max_initial_delay_ms": 20000,
    "max_network_jitter_ms": 1,
    "custom_int_properties": {
      "polls_before_disappearing": 1
    }
  }

See internal/client/impl/ for already-implemented example client_types.

Notable Feature Gaps

The main notable feature/usability gaps that have yet to be implemented are:

• Migrating our experiment statistics to leverage dockerized Prometheus and Grafana rather than Datadog
• Simplifying our build dependencies (incl. Redis) to a single containerized Docker image

Contributing

Please refer to the Contributing document if you are interested in contributing to goqueuesim!