Skip to content

Commit

Permalink
fix(docs): fixing docs build and variable sets (#3047)
Browse files Browse the repository at this point in the history
* fix(docs): fixing docs build and variable sets

* Fixing test suite id param

* pr comments
  • Loading branch information
xoscar committed Aug 9, 2023
1 parent 6314696 commit 6301dbd
Show file tree
Hide file tree
Showing 22 changed files with 102 additions and 173 deletions.
31 changes: 0 additions & 31 deletions docs/docs/cli/creating-environments.md

This file was deleted.

4 changes: 2 additions & 2 deletions docs/docs/cli/creating-test-outputs.md
@@ -1,10 +1,10 @@
# Defining Test Outputs in Text Files

Outputs are really useful when running [Transactions](../concepts/transactions). They allow for exporting values from a test so they become available in the [Environment Variables](../concepts/variable-sets.md) of the current transaction.
Outputs are really useful when running [Transactions](../concepts/transactions). They allow for exporting values from a test so they become available in the [Variable Sets](../concepts/variable-sets.md) of the current transaction.

## Outputs are Expression Results

An output exports the result of an [Expression](../concepts/expressions) and assigns it to a name, so it can be injected into the environment variables of a running transaction.
An output exports the result of an [Expression](../concepts/expressions) and assigns it to a name, so it can be injected into the variable set of a running transaction.
A `selector` is needed only if the provided expression refers to a/some span/s attribute or meta attributes.

It can be defined using the following YAML definition:
Expand Down
31 changes: 31 additions & 0 deletions docs/docs/cli/creating-variable-sets.md
@@ -0,0 +1,31 @@
# Defining Variable Sets as Text Files

This page showcases how to create and edit variable sets with the CLI.

:::tip
[To read more about variable sets check out variable sets concepts.](../concepts/variable-sets.md)
:::

Just like Data Stores, you can also manage your variable sets using the CLI and definition files.

A definition file looks like the following:

```yaml
type: VariableSet
spec:
name: Production
description: Production env variables
values:
- key: URL
value: https://app-production.company.com
- key: API_KEY
value: mysecret
```

In order to apply this configuration to your Tracetest instance, make sure to have your [CLI configured](./configuring-your-cli.md) and run:

```sh
tracetest apply variableset -f <variableset.yaml>
```

> If the file contains the property `spec.id`, the operation will be considered a variable set update. If you try to apply a variable set and you get the error: `could not apply variableset: 404 Not Found`, it means the provided ID doesn't exist. Either update the ID to reference an existing variable set or remove the property from the file. Tracetest will create a new variable set and a new ID.
16 changes: 8 additions & 8 deletions docs/docs/cli/running-tests.md
Expand Up @@ -186,24 +186,24 @@ The JUnit output file would then contain the JUnit result, for example:
</testsuites>
```

## Running a Test That Uses Environment Variables
## Running a Test That Uses Variable Sets

There are two ways of referencing an environment when running a test.
There are two ways of referencing a variable set when running a test.

You can reference an existing environment using its id. For example, given this defined environment with an id of 'testenv':
You can reference an existing variable set using its id. For example, given this defined variable set with an id of 'testenv':

![testenv](../img/show-environment-definition.png)

We can run a test and specify that environment with this command:
We can run a test and specify that variable set with this command:

```sh
tracetest run test -f path/to/test.yaml -e testenv
```

You can also reference an environment resource file which will be used to create a new environment or update an existing one. For example, if you have a file named local.env with this content:
You can also reference a variable set resource file which will be used to create a new variable set or update an existing one. For example, if you have a file named local.env with this content:

```yaml
type: Environment
type: VariableSet
spec:
id: local.env
name: local.env
Expand All @@ -215,9 +215,9 @@ spec:
```

```sh
tracetest run test -f path/to/test.yaml -e path/to/local.env
tracetest run test -f path/to/test.yaml --vars path/to/local.env
```

If you use the environment resource approach, a new environment will be created in Tracetest.
If you use the variable set resource approach, a new variable set will be created in Tracetest.

The second approach is very useful if you are running tests from a CI pipeline.
18 changes: 9 additions & 9 deletions docs/docs/cli/running-transactions.md
Expand Up @@ -21,24 +21,24 @@ tracetest run transaction -f path/to/transaction.yaml
✔ Pokeshop - List (http://localhost:11633/test/QvPjBH_4g/run/4/test)
```

## Running a Transaction That Uses Environment Variables
## Running a Transaction That Uses Variable Sets

There are two ways of referencing an environment when running a transaction.
There are two ways of referencing a variable set when running a transaction.

You can reference an existing environment using its id. For example, given this defined environment with an id of `'testenv'`:
You can reference an existing variable set using its id. For example, given this defined variable set with an id of `'testenv'`:

![testenv](../img/show-environment-definition.png)

We can run a transaction and specify that environment with this command:
We can run a transaction and specify that variable set with this command:

```sh
tracetest run transaction -f path/to/transaction.yaml -e testenv
tracetest run transaction -f path/to/transaction.yaml --vars testenv
```

You can also reference an environment resource file which will be used to create a new environment or update an existing one. For example, if you have a file named `local.env` with this content:
You can also reference a variable set resource file which will be used to create a new variable set or update an existing one. For example, if you have a file named `local.env` with this content:

```yaml
type: Environment
type: VariableSet
spec:
id: local.env
name: local.env
Expand All @@ -50,9 +50,9 @@ spec:
```

```sh
tracetest run test -f path/to/transaction.yaml -e path/to/local.env
tracetest run test -f path/to/transaction.yaml --vars path/to/local.env
```

If you use the environment resource approach, a new environment will be created in Tracetest.
If you use the variable set resource approach, a new variable set will be created in Tracetest.

The second approach is very useful if you are running tests from a CI pipeline.
18 changes: 9 additions & 9 deletions docs/docs/cli/undefined-variables.md
Expand Up @@ -13,29 +13,29 @@ POKEID:
POKENAME:
```

Undefined variables are dependent on the environment selected and whether or not the variable is defined in the current environment. Select the environment to run the test or transaction by passing it into the test run command.
Undefined variables are dependent on the variable set selected and whether or not the variable is defined in the current variable set. Select the variable set to run the test or transaction by passing it into the test run command.

```sh
tracetest list environment
tracetest list variableset
```

```text title="Output:"
ID NAME DESCRIPTION
--------- --------- -------------
testenv testenv testenv
testvars testvars testvars
```

```sh
tracetest get environment --id testenv
tracetest get variableset --id testvars
```

```text title="Output:"
---
type: Environment
type: VariableSet
spec:
id: testenv
name: testenv
description: testenv
id: testvars
name: testvars
description: testvars
values:
- key: POKEID
value: "42"
Expand All @@ -44,7 +44,7 @@ spec:
```

```sh
tracetest run test -f path/to/test.yaml -e testenv
tracetest run test -f path/to/test.yaml --vars testvars
```

```text title="Output:"
Expand Down
8 changes: 4 additions & 4 deletions docs/docs/concepts/ad-hoc-testing.md
@@ -1,6 +1,6 @@
# Ad-hoc Testing

This page showcases use-cases for undefined variables and how to enable ad-hoc testing by utilizing environments and undefined variabled.
This page showcases use-cases for undefined variables and how to enable ad-hoc testing by utilizing variable sets and undefined variables.

:::tip
[Check out how to configure ad-hoc testing with undefined variables with the **Web UI** here.](../web-ui/undefined-variables.md)
Expand All @@ -24,15 +24,15 @@ In Tracetest, undefined variables can be used in both the UI and CLI.

## **Undefined Variables Transaction with Multiple Tests Example**

1. Create an HTTP Pokemon list test that uses environment variables for hostname and the SKIP query parameter:
1. Create an HTTP Pokemon list test that uses variables for hostname and the SKIP query parameter:

![Create Pokemon List](../img/pokeshop-list.png)

2. Within the test, create test spec assertions that use environment variables for comparators, something like: `http.status_code = "${env:STATUS_CODE}"`:
1. Within the test, create test spec assertions that use variables for comparators, something like: `http.status_code = "${env:STATUS_CODE}"`:

![Create Test Spec Assertionsl](../img/create-test-spec-assertions.png)

3. Create a GRPC Pokemon add test that uses environment variables for hostname and Pokemon name:
1. Create a GRPC Pokemon add test that uses variables for hostname and Pokemon name:

![Create GRPC](../img/create-grpc.png)

Expand Down
66 changes: 0 additions & 66 deletions docs/docs/concepts/environments.md

This file was deleted.

2 changes: 1 addition & 1 deletion docs/docs/concepts/expressions.md
Expand Up @@ -32,7 +32,7 @@ attr:product.stock = attr:product.stok_before_purchase - attr:product.number_bou

### **String Interpolation**

Some tests might require strings to be compared, but maybe you need to generate a dynamic string that relies on a dynamic value. This might be used in an assertion or even in the request body referencing an environment variable.
Some tests might require strings to be compared, but maybe you need to generate a dynamic string that relies on a dynamic value. This might be used in an assertion or even in the request body referencing a variable.

```css
attr:error.message = "Could not withdraw ${attr:withdraw.amount}, your balance is insufficient."
Expand Down
10 changes: 5 additions & 5 deletions docs/docs/concepts/transactions.md
Expand Up @@ -9,13 +9,13 @@ A transaction is defined as a group of steps that are executed in the defined or
The main benefit of using transactions is to chain tests together and use values obtained from a test in a subsequent test.

### How Values are Shared by Tests
When a transaction is run, a context object is created with information about that specific run. One of those pieces of information is an `environment variables` object, which is empty by default. If the transaction is run when referencing an [environment](./environments), all values from the selected environments will be copied to the `environment variables` object.
When a transaction is run, a context object is created with information about that specific run. One of those pieces of information is a `variable set` object, which is empty by default. If the transaction is run when referencing an [variable set](./variable-sets.md), all values from the selected variable sets will be copied to the `variable set` object.

When a test is executed within a transaction, if it generates any outputs, its outputs will be injected into the transaction context environment variables object. After the outputs are injected, all subsequent tests to be run within the transaction will be able to reference those values.
When a test is executed within a transaction, if it generates any outputs, its outputs will be injected into the transaction context variable set object. After the outputs are injected, all subsequent tests to be run within the transaction will be able to reference those values.

> :information_source: Outputs generated by steps don't modify the selected [environment](./environments). It only modifies the transaction run context object.
> :information_source: Outputs generated by steps don't modify the selected [variable set](./variable-sets.md). It only modifies the transaction run context object.
Consider you have 3 tests within a transaction: A, B, and C. Tests A and B generate outputs called A_OUTPUT and B_OUTPUT, respectively. When running the transaction, we provide an environment which contains a `HOST` environment variable. The execution of test A would only be able to reference `env:HOST`. B would be able to reference `env:HOST`, and `env:A_OUTPUT`. While C would be able to reference all three environment variables: `env:HOST`, `env:A_OUTPUT`, `env:B_OUTPUT`.
Consider you have 3 tests within a transaction: A, B, and C. Tests A and B generate outputs called A_OUTPUT and B_OUTPUT, respectively. When running the transaction, we provide a variable set which contains a `HOST` variable. The execution of test A would only be able to reference `env:HOST`. B would be able to reference `env:HOST`, and `env:A_OUTPUT`. While C would be able to reference all three variables: `env:HOST`, `env:A_OUTPUT`, `env:B_OUTPUT`.

> :information_source: A single test can contain as many outputs as you like.
Expand All @@ -29,7 +29,7 @@ outputs:
expression: attr:tracetest.time.start
```

This would create an output called `TIME_CANCEL_SUBSCRIPTION_MESSAGE_OBTAINED` that is obtained by reading the attribute `tracetest.time.start` from the span with `name` equal to `Process request from cancel subscription queue`. This value would would then be injected into the environment variables of that transaction to be accessed by other tests within the same transaction run.
This would create an output called `TIME_CANCEL_SUBSCRIPTION_MESSAGE_OBTAINED` that is obtained by reading the attribute `tracetest.time.start` from the span with `name` equal to `Process request from cancel subscription queue`. This value would then be injected into the variables of that transaction to be accessed by other tests within the same transaction run.

### Transactions Execution Flow

Expand Down
8 changes: 4 additions & 4 deletions docs/docs/concepts/variable-sets.md
Expand Up @@ -8,7 +8,7 @@ Variable sets are objects containing variables that can be referenced by tests.

```yaml
# dev.env
type: Environment
type: VariableSet
spec:
name: dev.env
id: dev.env
Expand All @@ -21,7 +21,7 @@ spec:

```yaml
# staging.env
type: Environment
type: VariableSet
spec:
name: staging.env
id: staging.env
Expand All @@ -34,7 +34,7 @@ spec:

```yaml
# production.env
type: Environment
type: VariableSet
spec:
name: production.env
id: production.env
Expand Down Expand Up @@ -63,4 +63,4 @@ specs:
token: "${env:API_TOKEN}"
```

Both `env:URL` and `env:API_TOKEN` would be replaced by the variables defined in the chosen environment where the test will run. So, if the chosen environment was `dev.env`, its values would be replaced by `https://app-dev.com` and `dev-key`, respectively.
Both `env:URL` and `env:API_TOKEN` would be replaced by the variables defined in the chosen variable set where the test will run. So, if the chosen variable set was `dev.env`, its values would be replaced by `https://app-dev.com` and `dev-key`, respectively.
2 changes: 1 addition & 1 deletion docs/docs/configuration/tracetest-analyzer.md
Expand Up @@ -52,4 +52,4 @@ Here, you can also set the thresholds for `Otel Semantic Conventions`, `Common P

## Tracetest Analyzer in the CLI

You can use Tracetest Analyzer in the CLI to analyze per individual test. Visit the [Creating Transactions](/docs/docs/web-ui/creating-transactions.md) page for details.
You can use Tracetest Analyzer in the CLI to analyze per individual test. Visit the [Creating Transactions](../web-ui/creating-transactions.md) page for details.
3 changes: 0 additions & 3 deletions docs/docs/environment-variables.md

This file was deleted.

Binary file added docs/docs/img/all-vars.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/docs/img/create-vars-form.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/docs/img/data-vars.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/docs/img/definition-vars.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 6301dbd

Please sign in to comment.