Skip to content

Commit

Permalink
[feat] make scenario examples more general
Browse files Browse the repository at this point in the history
- Simpler language
- Less specific annotation of the code, making it easier to change
  later, and easier for users to copy, paste, and modify without
  incoherent comments.
  • Loading branch information
MattDodsonEnglish committed May 23, 2023
1 parent bf8a62b commit 7943760
Show file tree
Hide file tree
Showing 6 changed files with 56 additions and 55 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,12 @@ number of total iterations, and the amount of iterations per VU is unimportant.
If the **time to complete** a number of test iterations is your concern, this executor should perform best.

An example use case is for quick performance tests in the developement build cycle.
As developers make changes, they might run the test against the local code to test for performance regressions. Thus the executor works well with a _shift-left_ policy, where emphasizes testing performance early in the development cycle when the cost of a fix is lowest.
As developers make changes, they might run the test against the local code to test for performance regressions.
Thus the executor works well with a _shift-left_ policy, where emphasizes testing performance early in the development cycle, when the cost of a fix is lowest.

## Example

In this example, we'll execute 200 total iterations shared by 10 VUs with a maximum duration of 30 seconds.
The following example schedules 200 total iterations shared by 10 VUs with a maximum test duration of 30 seconds.

<CodeGroup labels={[ "shared-iters.js" ]} lineNumbers={[true]}>

Expand All @@ -62,7 +63,7 @@ export const options = {

export default function () {
http.get('https://test.k6.io/contacts.php');
// We're injecting a processing pause for illustrative purposes only!
// Injecting sleep
// Sleep time is 500ms. Total iteration time is sleep + time to finish request.
sleep(0.5);
}
Expand All @@ -79,9 +80,8 @@ The following graph depicts the performance of the [example](#example) script:
Based upon our test scenario inputs and results:

* Test is limited to a fixed number of 200 iterations of the `default` function;
* the number of VUs is fixed to 10, and are initialized before the test begins;
* each _iteration_ of the `default` function is expected to be roughly 515ms, or ~2/s;
* maximum throughput (highest efficiency) is therefore expected to be ~20 iters/s, `2 iters/s * 10 VUs`;
* we then see that the maximum throughput is maintained for a larger portion of the test;
* the 8 second test duration will be the shortest of all executor methods;
* we know the distribution of iterations may be skewed; one VU may have performed 50 iterations, whereas another may have only performed 10.
* The number of VUs is fixed to 10, and are initialized before the test begins;
* Each _iteration_ of the `default` function is expected to be roughly 515ms, or ~2/s;
* Maximum throughput (highest efficiency) is therefore expected to be ~20 iters/s, `2 iters/s * 10 VUs`;
* The maximum throughput is maintained for a larger portion of the test;
* The distribution of iterations may be skewed: one VU may have performed 50 iterations, another only 10.
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@ partition between VUs.

## Example

In this example, we'll let 10 VUs execute 20 iterations _each_, for a total of 200 iterations, with
a maximum duration of 30 seconds.
The following example schedules 10 VUs to execute 20 iterations _each_.
The test runs 200 total iterations and has a maximum duration of 30 seconds.

<CodeGroup labels={[ "per-vu-iters.js" ]} lineNumbers={[true]}>

Expand All @@ -48,7 +48,7 @@ export const options = {

export default function () {
http.get('https://test.k6.io/contacts.php');
// We're injecting a processing pause for illustrative purposes only!
// Injecting sleep
// Sleep time is 500ms. Total iteration time is sleep + time to finish request.
sleep(0.5);
}
Expand All @@ -65,10 +65,10 @@ The following graph depicts the performance of the [example](#example) script:
Based upon our test scenario inputs and results:

* The number of VUs is fixed at 10, and are initialized before the test begins;
* total iterations are fixed at 20 iterations per VU, i.e. 200 iterations, `10 VUs * 20 iters each`;
* each _iteration_ of the `default` function is expected to be roughly 515ms, or ~2/s;
* maximum throughput (highest efficiency) is therefore expected to be ~20 iters/s, `2 iters/s * 10 VUs`;
* we then see that the maximum throughput is reached, but not maintained;
* because distribution of iterations is even amongst VUs, a _fast_ VU may finish early and be idle for the remainder of the test, thereby lowering _efficiency_;
* total duration of 9 seconds is slightly longer than [shared iterations](/using-k6/scenarios/executors/shared-iterations) due to lower efficiency;
* overall test duration lasts as long as the _slowest_ VU takes to complete 20 requests.
* Total iterations are fixed at 20 iterations per VU, i.e. 200 iterations, `10 VUs * 20 iters each`;
* Each _iteration_ of the `default` function is expected to be roughly 515ms, or ~2/s;
* Maximum throughput (highest efficiency) is therefore expected to be ~20 iters/s, `2 iters/s * 10 VUs`;
* The maximum throughput is reached, but not maintained;
* Because the distribution of iterations is even among VUs, a _fast_ VU may finish early and be idle for the remainder of the test, thereby lowering _efficiency_;
* Total duration of 9 seconds is slightly longer than [shared iterations](/using-k6/scenarios/executors/shared-iterations) due to lower efficiency;
* Overall test duration lasts as long as the _slowest_ VU takes to complete 20 requests.
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Use this executor if you need a specific number of VUs to run for a certain amou

## Example

In this example, we'll run 10 VUs constantly for a duration 30 seconds.
This examples schedules 10 VUs to run constantly for a duration 30 seconds.

<CodeGroup labels={[ "constant-vus.js" ]} lineNumbers={[true]}>

Expand All @@ -44,8 +44,8 @@ export const options = {

export default function () {
http.get('https://test.k6.io/contacts.php');
// We're injecting a processing pause for illustrative purposes only!
// Sleep time is 500ms. Total iteration time is sleep + time to finish request.
// Injecting sleep
// Total iteration time is sleep + time to finish request.
sleep(0.5);
}
```
Expand All @@ -61,8 +61,8 @@ The following graph depicts the performance of the [example](#example) script:
Based upon our test scenario inputs and results:

* The number of VUs is fixed at 10, and are initialized before the test begins;
* overall test duration is fixed at the configured 30 second duration;
* each _iteration_ of the `default` function is expected to be roughly 515ms, or ~2/s;
* maximum throughput (highest efficiency) is therefore expected to be ~20 iters/s, `2 iters/s * 10 VUs`;
* we see that the maximum throughput is reached and maintained for the majority of the test;
* approximately 600 iterations are therefore performed in total, `30 seconds * 20 iters/s`.
* Overall test duration is fixed at the configured 30 second duration;
* Each _iteration_ of the `default` function is expected to be roughly 515ms, or ~2/s;
* Maximum throughput (highest efficiency) is therefore expected to be ~20 iters/s, `2 iters/s * 10 VUs`;
* We see that the maximum throughput is reached and maintained for the majority of the test;
* Approximately 600 iterations are therefore performed in total, `30 seconds * 20 iters/s`.
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ of time.

## Example

In this example, we'll run a two-stage test, ramping up from 0 to 10 VUs over 20 seconds, then down
This example schedules a two-stage test, ramping up from 0 to 10 VUs over 20 seconds, then down
to 0 VUs over 10 seconds.

<CodeGroup labels={[ "ramping-vus.js" ]} lineNumbers={[true]}>
Expand All @@ -51,16 +51,20 @@ export const options = {

export default function () {
http.get('https://test.k6.io/contacts.php');
// We're injecting a processing pause for illustrative purposes only!
// Injecting sleep
// Sleep time is 500ms. Total iteration time is sleep + time to finish request.
sleep(0.5);
}
```

</CodeGroup>

> Note the setting of `gracefulRampDown` to 0 seconds, which could cause some iterations to be
interrupted during the ramp down stage.
<Blockquote mod="note" title="">

With [`gracefulRampDown`](/using-k6/scenarios/concepts/graceful-stop/#the-gracefulrampdown) set to 0 seconds, some iterations might be
interrupted during the rampdown stage.

</Blockquote>

## Observations

Expand All @@ -70,12 +74,12 @@ The following graph depicts the performance of the [example](#example) script:

Based upon our test scenario inputs and results:

* We've defined 2 stages for a total test duration of 30 seconds;
* stage 1 ramps _up_ VUs linearly from the `startVUs` of 0 to the target of 10 over a 20 second duration;
* from the 10 VUs at the end of stage 1, stage 2 then ramps _down_ VUs linearly to the target of 0 over a 10 second duration;
* each _iteration_ of the `default` function is expected to be roughly 515ms, or ~2/s;
* as the number of VUs changes, the iteration rate directly correlates; each addition of a VU increases the rate by \~2 iters/s, whereas each subtraction of a VU reduces by \~2 iters/s;
* our example performed ~300 iterations over the course of the test.
* The configuration defines 2 stages for a total test duration of 30 seconds;
* Stage 1 ramps _up_ VUs linearly from the `startVUs` of 0 to the target of 10 over a 20 second duration;
* From the 10 VUs at the end of stage 1, stage 2 then ramps _down_ VUs linearly to the target of 0 over a 10 second duration;
* Each _iteration_ of the `default` function is expected to be roughly 515ms, or ~2/s;
* As the number of VUs changes, the iteration rate directly correlates; each addition of a VU increases the rate by about 2 iters/s, whereas each subtraction of a VU reduces by about 2 iters/s;
* The example performed ~300 iterations over the course of the test.

## Get the stage index

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,8 @@ So it's unnecessary to use a `sleep()` function at the end of the VU code.

## Example

This example starts a constant rate of 30 iterations per second for 30 seconds, allowing k6 to dynamically schedule up to 50 VUs.
This example schedules a constant rate of 30 iterations per second for 30 seconds.
It allocates 50 VUs for k6 to dynamically use as needed.

<CodeGroup labels={[ "constant-arr-rate.js" ]} lineNumbers={[true]}>

Expand All @@ -65,22 +66,18 @@ export const options = {
contacts: {
executor: 'constant-arrival-rate',

// Our test should last 30 seconds in total
// How long the test lasts
duration: '30s',

// It should start 30 iterations per `timeUnit`. Note that iterations starting points
// will be evenly spread across the `timeUnit` period.
// How many iterations per timeUnit
rate: 30,

// It should start `rate` iterations per second
// Start `rate` iterations per second
timeUnit: '1s',

// It should preallocate 2 VUs before starting the test
preAllocatedVUs: 2,
// Pre-allocate VUs
preAllocatedVUs: 50,

// It is allowed to spin up to 50 maximum VUs to sustain the defined
// constant arrival rate.
maxVUs: 50,
},
},
};
Expand All @@ -103,7 +100,7 @@ Based upon our test scenario inputs and results:

* The desired rate of 30 iterations started every 1 second is achieved and maintained for the majority of the test.
* The test scenario runs for the specified 30 second duration.
* Having started with 2 VUs (as specified by the `preAllocatedVUs` option), k6 automatically adjusts the number of VUs to achieve the desired rate, up to the `maxVUs`; for our test, this ended up as 17 VUs.
* Having started with 2 VUs (as specified by the `preAllocatedVUs` option), k6 automatically adjusts the number of VUs to achieve the desired rate, up to the allocated number. For this test, this ended up as 17 VUs.
* Exactly 900 iterations are started in total, `30s * 30 iters/s`.

> As in our example, using too low of a `preAllocatedVUs` setting will reduce the test duration at the desired rate, as resources need to continually be allocated to achieve the rate.
> Using too low of a `preAllocatedVUs` setting will reduce the test duration at the desired rate, as resources need to continually be allocated to achieve the rate.
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,8 @@ It's unnecessary to use a `sleep()` function at the end of the VU code.
## Example

This is an example of a four-stage test.
It initially stays at the defined rate of starting 300 iterations per minute over a one minute period.

It starts at the defined `startRate`, 300 iterations per minute over a one minute period.
After one minute, the iteration rate ramps to 600 iterations started per minute over the next two minutes, and stays at this rate for four more minutes.
In the last two minutes, it ramps down to a target of 60 iterations per minute.

Expand All @@ -68,7 +69,7 @@ export const options = {
contacts: {
executor: 'ramping-arrival-rate',

// Start at 300 iterations per `timeUnit`
// Start iterations per `timeUnit`
startRate: 300,

// Start `startRate` iterations per minute
Expand Down Expand Up @@ -109,14 +110,13 @@ The following graph depicts the performance of the [example](#example) script:

Based upon our test scenario inputs and results:

* We've defined 4 stages for a total test duration of 9 minutes.
* The configuration defines 4 stages for a total test duration of 9 minutes.
* Stage 1 maintains the `startRate` iteration rate at 300 iterations started per minute for 1 minute.
* Stage 2 ramps _up_ the iteration rate linearly from the *stage 1* of 300 iterations started per minute, to the target of 600 iterations started per minute over a 2-minute duration.
* Stage 3 maintains the *stage 2* iteration rate at 600 iterations started per minute over a 4-minute duration.
* Stage 4 ramps _down_ the iteration rate linearly to the target rate of 60 iterations started per minute over the last two minutes duration.
* Changes to the iteration rate are performed by k6, adjusting the number of VUs as necessary from `preAllocatedVUs` to a maximum of `maxVUs`.
* The script waits for a period of time (defined by the `gracefulStop` option) for iterations to finish. It won't start new iterations during the `gracefulStop` period.
* The script will run the `teardown()` function (if specified) before exiting.
* Changes to the iteration rate are performed by k6, adjusting the number of VUs
* The script waits for a period of time (defined by the `gracefulStop` option) for iterations to finish. It doesn't start new iterations during the `gracefulStop` period.
* Our example performed, 4020 iterations over the course of the test.

## Get the stage index
Expand Down

0 comments on commit 7943760

Please sign in to comment.