Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SPIKE: Define and run benchmarking experiments #1400

Open
3 tasks
doodlesbykumbi opened this issue Apr 6, 2021 · 1 comment
Open
3 tasks

SPIKE: Define and run benchmarking experiments #1400

doodlesbykumbi opened this issue Apr 6, 2021 · 1 comment

Comments

@doodlesbykumbi
Copy link
Contributor

doodlesbykumbi commented Apr 6, 2021

Overview

The code changes to accommodate metrics collection will likely be shared among the different connectors, however there's nuance to each connector (at the very least when it comes to generating load). We'll likely want to carry out analysis on a variety of traffic profiles in the experiments to come. For example pg_bench is great for creating realistic traffic for Postgres.

The approach to running experiments is similar across connectors.

  1. Determine what is possible in terms of controlling the experiment
  2. Define the experiment
  3. Generate load
  4. Observe generated data

To simplify things this exercise will focus on Postgres to prove out the process of running experiments.

NOTE that at this stage the traffic profile of interest is the light operating profile, defined as:

  • 2000 open connections
  • 10 active (transmitting) connection
  • 50 waiting (non-transmitting) connections

Definition of done

  • A document exists capturing the means of generic realistic traffic for the Secretless TCP connectors
  • A set of experiments are defined for the Secretless TCP connectors
  • The set of experiments are run, and the resuting data is made available for reporting
@izgeri
Copy link
Contributor

izgeri commented Apr 6, 2021

The requirement from the feature spec is:

Secretless latency under ‘light’ load and maximum throughput (Mbps) per single Secretless container has been measured. ‘Light’ load or operating profile is defined as above, where Secretless is communicating with a database cluster with variable data size per request.

In #1398 we determine precisely how we can capture data within Secretless on throughput and latency. In this card, if I'm reading it correctly, we're asking:

  • How can we set up an experiment to determine maximum throughput?
  • How can we set up an experiment to measure latency so that:
    • Secretless is communicating with a database cluster with variable data size per request
    • Secretless is operating under a "light" operating profile (as defined in the description)
    • Secretless is operating at maximum throughput

@doodlesbykumbi doodlesbykumbi changed the title SPIKE: Generating realistic benchmarking traffic SPIKE: Define and run benchmarking experiments May 7, 2021
@doodlesbykumbi doodlesbykumbi self-assigned this May 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

2 participants