Skip to content

Commit

Permalink
feat(docs): update docs and examples with new run command (#2931)
Browse files Browse the repository at this point in the history
  • Loading branch information
schoren committed Jul 13, 2023
1 parent 332a755 commit 06ba21a
Show file tree
Hide file tree
Showing 52 changed files with 631 additions and 658 deletions.
992 changes: 496 additions & 496 deletions docs/docs/ci-cd-automation/github-actions-pipeline.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/docs/ci-cd-automation/overview.md
Expand Up @@ -31,5 +31,5 @@ Use the command below, substituting the following placeholders:
- `file-path` - The path to the saved Tracetest test. Example: `./mytest.yaml`

```bash wordWrap=true
docker run --rm -it -v$(pwd):$(pwd) -w $(pwd) --network host --entrypoint tracetest kubeshop/tracetest:latest -s <your-tracetest-server-url> test run --definition <file-path> --wait-for-result
docker run --rm -it -v$(pwd):$(pwd) -w $(pwd) --network host --entrypoint tracetest kubeshop/tracetest:latest -s <your-tracetest-server-url> run test --file <file-path>
```
2 changes: 1 addition & 1 deletion docs/docs/ci-cd-automation/tekton-pipeline.md
Expand Up @@ -243,7 +243,7 @@ spec:
script: |
# Configure and Run Tracetest CLI
tracetest configure -g --endpoint http://tracetest.tracetest.svc.cluster.local:11633/
tracetest test run -d /workspace/test-api.yaml -w
tracetest run test -f /workspace/test-api.yaml
volumeMounts:
- name: custom
mountPath: /workspace
Expand Down
4 changes: 2 additions & 2 deletions docs/docs/ci-cd-automation/testkube-pipeline.md
Expand Up @@ -314,7 +314,7 @@ Duration:
Getting logs from test job 641885f39922b3e1003dd5b6
Execution completed
🔬 Executing in directory :
$ tracetest test run --server-url http://10.96.93.106:11633 --definition /tmp/test-content737616681 --wait-for-result --output pretty
$ tracetest run test --server-url http://10.96.93.106:11633 --file /tmp/test-content737616681 --output pretty
✘ Pokeshop - List (http://10.96.93.106:11633/test/RUkKQ_aVR/run/2/test)
✘ Database queries less than 500 ms
#2b213392d0e3ff21
Expand Down Expand Up @@ -359,7 +359,7 @@ Duration:
Getting logs from test job 6418873d9922b3e1003dd5b8
Execution completed
🔬 Executing in directory :
$ tracetest test run --server-url http://10.96.93.106:11633 --definition /tmp/test-content1901459587 --wait-for-result --output pretty
$ tracetest run test --server-url http://10.96.93.106:11633 --file /tmp/test-content1901459587 --output pretty
✔ Pokeshop - List (http://10.96.93.106:11633/test/RUkKQ_aVR/run/3/test)
✔ Database queries less than 500 ms
Expand Down
128 changes: 62 additions & 66 deletions docs/docs/cli/configuring-your-cli.md
@@ -1,66 +1,62 @@
# Configuring your CLI

Our web interface makes it easier to visualize your traces and add assertions, but sometimes a CLI is needed for automation. The CLI was developed for users creating tests and executing them each time a change is made in the system, so Tracetest can detect regressions and check service Service Level Objectives (SLOs).

## Available Commands

Here is a list of all available commands and how to use them:

### Configure

Configure your CLI to connect to your Tracetest server.

**How to Use**:

```sh
tracetest configure
```

If you want to set values without having to answer questions from a prompt, you can provide the flag `--endpoint` to define the server endpoint.

```sh
tracetest configure --endpoint http://my-tracetest-server:11633
```

### Test List

Allows you to list all tests.

**How to Use**:

```sh
tracetest list test
```

### Run a Test

Allows you to run a test by referencing a [test definition file](./creating-tests).

> Note: If the definition file contains the field `id`, this command will not create a new test. Instead, it will update the test with that ID. If that test doesn't exist, a new one will be created with that ID on the server.
Every time the test is run, changes are detected and, if any change is introduced, we use Tractest's [versioning](../concepts/versioning) mechanism to ensure that it will not cause problems with previous test runs.

**How to Use**:

```sh
tracetest test run --definition <file-path>
```

**Options**:

`--wait-for-result`: The CLI will only exit after the test run has completed (the trace was retrieved and assertions were executed).

### Running Tracetest CLI From Docker

There are times when it is easier to directly execute the Tracetest CLI from a Docker image rather than installing the CLI on your local machine. This can be convenient when you wish to execute the CLI in a CI/CD environment.

**How to Use**:

Use the command below, substituting the following placeholders:

- `your-tracetest-server-url` - The URL to the running Tracetest server you wish to execute the test on. Example: `http://localhost:11633/`
- `file-path` - The path to the saved Tracetest test. Example: `./mytest.yaml`

```bash wordWrap=true
docker run --rm -it -v$(pwd):$(pwd) -w $(pwd) --network host --entrypoint tracetest kubeshop/tracetest:latest -s <your-tracetest-server-url> test run --definition <file-path> --wait-for-result
```
# Configuring your CLI

Our web interface makes it easier to visualize your traces and add assertions, but sometimes a CLI is needed for automation. The CLI was developed for users creating tests and executing them each time a change is made in the system, so Tracetest can detect regressions and check your Service Level Objectives (SLOs).

## Available Commands

Here is a list of all available commands and how to use them:

### Configure

Configure your CLI to connect to your Tracetest server.

**How to Use**:

```sh
tracetest configure
```

If you want to set values without having to answer questions from a prompt, you can provide the flag `--endpoint` to define the server endpoint.

```sh
tracetest configure --endpoint http://my-tracetest-server:11633
```

### Test List

Allows you to list all tests.

**How to Use**:

```sh
tracetest list test
```

### Run a Test

Allows you to run a test by referencing a [test definition file](./creating-tests).

> Note: If the definition file contains the field `id`, this command will not create a new test. Instead, it will update the test with that ID. If that test doesn't exist, a new one will be created with that ID on the server.
Every time the test is run, changes are detected and, if any change is introduced, we use Tractest's [versioning](../concepts/versioning) mechanism to ensure that it will not cause problems with previous test runs.

**How to Use**:

```sh
tracetest run test --file <file-path>
```

### Running Tracetest CLI from Docker

There are times when it is easier to directly execute the Tracetest CLI from a Docker image rather than installing the CLI on your local machine. This can be convenient when you wish to execute the CLI in a CI/CD environment.

**How to Use**:

Use the command below, substituting the following placeholders:

- `your-tracetest-server-url` - The URL to the running Tracetest server you wish to execute the test on. Example: `http://localhost:11633/`
- `file-path` - The path to the saved Tracetest test. Example: `./mytest.yaml`

```bash wordWrap=true
docker run --rm -it -v$(pwd):$(pwd) -w $(pwd) --network host --entrypoint tracetest kubeshop/tracetest:latest -s <your-tracetest-server-url> run test --file <file-path>
```
24 changes: 7 additions & 17 deletions docs/docs/cli/running-tests.md
Expand Up @@ -4,24 +4,14 @@ Once you have created a test, whether from the Tracetest UI or via a text editor

The documentation for running a test via the CLI can be found here:

- [tracetest test run](./reference/tracetest_test_run.md): This page provides examples of using this command.
- [tracetest run](./reference/tracetest_run.md): This page provides examples of using this command.

## Running Your First Test

To run a test, give the path to the test definition file with the `'-d'` option. This will launch a test and provide a link to the created test run.
To run a test, give the path to the test definition file with the `'-f'` option. This will launch a test and provide a link to the created test run.

```sh
tracetest test run -d path/to/test.yaml
```

```text title="Output:"
✔ Pokeshop - Import (http://localhost:11633/test/4oI08rA4g/run/3/test)
```

Now, let's run the same test but tell the CLI to wait for the test to complete running before returning with the `'-w'` option. This will provide results from the test.

```sh
tracetest test run -d path/to/test.yaml -w
tracetest run test -f path/to/test.yaml
```

```text title="Output:"
Expand All @@ -44,7 +34,7 @@ tracetest test run -d path/to/test.yaml -w
Running the same command with the '-o json' option would change the output from the default of human readable 'pretty' to 'json'. This can be useful when you wish to extract particular data from the response. This would look like:

```sh
tracetest test run -d path/to/test.yaml -w -o json
tracetest run test -f path/to/test.yaml -o json
```

```json title="Output:"
Expand Down Expand Up @@ -173,7 +163,7 @@ tracetest test run -d path/to/test.yaml -w -o json
You can also opt to output the result as JUnit to a file. You would run the command with a -j option and a file name, ie:

```sh
tracetest test run -d path/to/test.yaml -w -j junit.out
tracetest run test -f path/to/test.yaml -j junit.out
```

The JUnit output file would then contain the JUnit result, for example:
Expand Down Expand Up @@ -207,7 +197,7 @@ You can reference an existing environment using its id. For example, given this
We can run a test and specify that environment with this command:

```sh
tracetest test run -d path/to/test.yaml -e testenv -w
tracetest run test -f path/to/test.yaml -e testenv
```

You can also reference an environment resource file which will be used to create a new environment or update an existing one. For example, if you have a file named local.env with this content:
Expand All @@ -225,7 +215,7 @@ spec:
```

```sh
tracetest test run -d path/to/test.yaml -e path/to/local.env -w
tracetest run test -f path/to/test.yaml -e path/to/local.env
```

If you use the environment resource approach, a new environment will be created in Tracetest.
Expand Down
21 changes: 5 additions & 16 deletions docs/docs/cli/running-transactions.md
Expand Up @@ -6,26 +6,15 @@ The command to run a transaction is the same as running a test from the CLI.

The documentation for running a test via the CLI can be found here:

- [tracetest test run](./reference/tracetest_test_run.md): This page provides examples of using this command.
- [tracetest run](./reference/tracetest_run.md): This page provides examples of using this command.

## Running Your First Transaction

To run a transaction, give the path to the transaction definition file with the `'-d'` option. This will launch a transaction, providing us with a link to the created transaction run.
To run a transaction, give the path to the transaction definition file with the `'-f'` option. This will launch a transaction, providing us with a link to the created transaction run.

```sh
tracetest test run -d path/to/transaction.yaml
tracetest run transaction -f path/to/transaction.yaml
```

```text title="Output:"
✔ Pokemon Transaction (http://localhost:11633/transaction/xcGqfHl4g/run/3)
```

Now, let's run the same test but tell the CLI to wait for the test to complete running before returning with the `'-w'` option. This will provide results from the test.

```sh
tracetest test run -d path/to/transaction.yaml -w
```

```text title="Output:"
✔ Pokemon Transaction (http://localhost:11633/transaction/xcGqfHl4g/run/3)
✔ Pokeshop - Import (http://localhost:11633/test/XRHjfH_4R/run/4/test)
Expand All @@ -43,7 +32,7 @@ You can reference an existing environment using its id. For example, given this
We can run a transaction and specify that environment with this command:

```sh
tracetest test run -d path/to/transaction.yaml -e testenv -w
tracetest run transaction -f path/to/transaction.yaml -e testenv
```

You can also reference an environment resource file which will be used to create a new environment or update an existing one. For example, if you have a file named `local.env` with this content:
Expand All @@ -61,7 +50,7 @@ spec:
```

```sh
tracetest test run -d path/to/transaction.yaml -e path/to/local.env -w
tracetest run test -f path/to/transaction.yaml -e path/to/local.env
```

If you use the environment resource approach, a new environment will be created in Tracetest.
Expand Down
4 changes: 2 additions & 2 deletions docs/docs/cli/undefined-variables.md
Expand Up @@ -3,7 +3,7 @@
When a user runs a test or a transaction, any variables that will be needed but are not defined will be prompted for:

```sh
tracetest test run -d path/to/test.yaml
tracetest run test -f path/to/test.yaml
```

```text title="Output:"
Expand Down Expand Up @@ -44,7 +44,7 @@ spec:
```

```sh
tracetest test run -d path/to/test.yaml -e testenv
tracetest run test -f path/to/test.yaml -e testenv
```

```text title="Output:"
Expand Down
Expand Up @@ -279,7 +279,7 @@ Now that all of the required services and infra have been created, you can start

1. From the Terraform output you can copy the `api_endpoint` and replace the `<your_api_endpoint>` placeholder from the `tests/test.yaml` file.
2. Configure the [Tracetest CLI](https://docs.tracetest.io/cli/configuring-your-cli) to point to the public load balancer endpoint with `tracetest configure --endpoint <tracetest_url>`.
3. Run the test YAML file using the CLI `tracetest test run -d tests/test.yaml`.
3. Run the test YAML file using the CLI `tracetest run test -f tests/test.yaml`.
4. Follow the link to find the results.

## Learn More
Expand Down
Expand Up @@ -318,7 +318,7 @@ This file defines a test the same way you would through the Web UI.
To run the test, run this command in the terminal:

```bash
tracetest test run -d ./http-test.yaml -w
tracetest run test -f ./http-test.yaml
```

This test will fail just like the sample above due to the `attr:tracetest.span.duration < 10ms` assertion.
Expand Down
Expand Up @@ -315,7 +315,7 @@ This file defines the a test the same way you would through the Web UI.
To run the test, run this command in the terminal:

```bash
tracetest test run -d ./test-api.yaml -w
tracetest run test -f ./test-api.yaml
```

```bash
Expand Down
Expand Up @@ -300,7 +300,7 @@ This file defines the a test the same way you would through the Web UI.
To run the test, run this command in the terminal:

```bash
tracetest test run -d ./http-test.yaml -w
tracetest run test -f ./http-test.yaml
```

This test will fail just like the sample above due to the `attr:tracetest.span.duration < 50ms` assertion.
Expand Down
Expand Up @@ -309,7 +309,7 @@ This file defines the a test the same way you would through the Web UI.
To run the test, run this command in the terminal:

```bash
tracetest test run -d ./http-test.yaml -w
tracetest run test -f ./http-test.yaml
```

This test will fail just like the sample above due to the `attr:tracetest.span.duration < 50ms` assertion.
Expand Down
Expand Up @@ -161,9 +161,9 @@ Now that all of the required services and infra have been created, you can start
3. Run the tests YAML file using the CLI.

```bash
tracetest test run -d tests/incident.yaml \
tracetest test run -d tests/exam.yaml \
tracetest test run -d tests/transaction.yaml
tracetest run test -f tests/incident.yaml \
tracetest run test -f tests/exam.yaml \
tracetest run transaction -f tests/transaction.yaml
```

4. Follow the link to find the results.
Expand Down
Expand Up @@ -241,7 +241,7 @@ spec:
script: |
# Configure and Run Tracetest CLI
tracetest configure -g --endpoint http://tracetest.tracetest.svc.cluster.local:11633/
tracetest test run -d /workspace/test-api.yaml -w
tracetest run test -f /workspace/test-api.yaml
volumeMounts:
- name: custom
mountPath: /workspace
Expand Down
Expand Up @@ -352,7 +352,7 @@ This file defines the a test the same way you would through the Web UI.
To run the test, run this command in the terminal:

```bash
tracetest test run -d ./test-api.yaml -w
tracetest run test -f ./test-api.yaml
```

This test will fail just like the sample above due to the `attr:books.list.count = 4` assertion.
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/getting-started/open.md
Expand Up @@ -125,7 +125,7 @@ spec:
Using the CLI, trigger a test run.

```bash title="Terminal"
tracetest test run -d pokeshop_import.yaml --wait-for-result -o pretty
tracetest run test -f pokeshop_import.yaml -o pretty
```

### Output
Expand Down
Expand Up @@ -61,7 +61,7 @@ Now, you can validate this entire use case.
To replicate this entire test on Tracetest, you can replicate these steps on our Web UI or using our CLI, saving the following test definition as the file `test-definition.yml` and later running:

```sh
tracetest test -d test-definition.yml --wait-for-results
tracetest run test -f test-definition.yml
```

We are assuming that the Frontend service is exposed on `http://otel-demo-frontend:8080`:
Expand Down
Expand Up @@ -66,7 +66,7 @@ Now you can validate this entire use case.
To replicate this entire test on Tracetest, you can replicate these steps on our Web UI or using our CLI, saving the following test definition as the file `test-definition.yml` and later running:

```sh
tracetest test -d test-definition.yml --wait-for-results
tracetest run test -f test-definition.yml
```

We are assuming that the Frontend service is exposed on `http://otel-demo-frontend:8080`:
Expand Down

0 comments on commit 06ba21a

Please sign in to comment.