diff --git a/docs/docs/ci-cd-automation/github-actions-pipeline.md b/docs/docs/ci-cd-automation/github-actions-pipeline.md index 4270afbf3a..7e456cda7d 100644 --- a/docs/docs/ci-cd-automation/github-actions-pipeline.md +++ b/docs/docs/ci-cd-automation/github-actions-pipeline.md @@ -1,496 +1,496 @@ -# GitHub Actions Pipeline - -:::note -[Check out the source code on GitHub here.](https://github.com/kubeshop/tracetest/tree/main/examples/quick-start-github-actions) -::: - -[Tracetest](https://tracetest.io/) is a testing tool based on [OpenTelemetry](https://opentelemetry.io/) that allows you to test your distributed application. It allows you to use data from distributed traces generated by OpenTelemetry to validate and assert if your application has the desired behavior defined by your test definitions. - -## GitHub Actions Workflow for Running Tracetest tests against a sample Node.js app with OpenTelemetry - -This is a simple quick start on how to configure GitHub Actions to run Tracetest tests against a Node.js app thats uses OpenTelemetry instrumentation with traces. This example includes manual instrumentation and a sample bookstore array that simulates fetching data from a database. - -## GitHub Actions Workflow - -This sample has two workflows. The workflows have one job and a total of 6 steps. The steps are: - -1. Checking out the repo code -2. Starting the sample app with Docker Compose -3. Installing the Tracetest CLI -4. Configuring the Tracetest CLI -5. Running tests with the Tracetest CLI -6. Stop Docker Compose - -The first workflow triggers a pre-merge and merge test run. - -```yaml -# start-and-test-on-main.yaml - -name: Docker Compose Start and Test on push and PR to main - -on: - push: - branches: [ "main" ] - pull_request: - branches: [ "main" ] - -jobs: - start-and-test: - timeout-minutes: 10 - runs-on: ubuntu-latest - steps: - - name: Checkout - uses: actions/checkout@v1 - - - name: Start containers - run: docker compose -f "docker-compose.yaml" -f "tracetest/docker-compose.yaml" up -d --build - - - name: Install Tracetest CLI - shell: bash - run: curl -L https://raw.githubusercontent.com/kubeshop/tracetest/main/install-cli.sh | bash - - - name: Configure Tracetest CLI - run: tracetest configure -g --endpoint http://localhost:11633 - - - name: Run tests via the Tracetest CLI - run: | - tracetest test run -d ./tracetest/tests/test-api.yaml -w - tracetest test run -d ./tracetest/tests/test-api-and-av.yaml -w - tracetest test run -d ./tracetest/tests/transaction-api.yaml -w - - - name: Stop containers - if: always() - run: docker compose -f "docker-compose.yaml" -f "tracetest/docker-compose.yaml" down -v -``` - -And, the other is a scheduled test run. - -```yaml -# start-and-test-on-schedule.yaml - -name: Docker Compose Start and Test Every Hour - -on: - schedule: - - cron: '0 * * * *' - -jobs: - start-and-test: - timeout-minutes: 10 - runs-on: ubuntu-latest - steps: - - name: Checkout - uses: actions/checkout@v1 - - - name: Start containers - run: docker compose -f "docker-compose.yaml" -f "tracetest/docker-compose.yaml" up -d --build - - - name: Install Tracetest CLI - shell: bash - run: curl -L https://raw.githubusercontent.com/kubeshop/tracetest/main/install-cli.sh | bash - - - name: Configure Tracetest CLI - run: tracetest configure -g --endpoint http://localhost:11633 - - - name: Run tests via the Tracetest CLI - run: | - tracetest test run -d ./tracetest/tests/test-api.yaml -w - tracetest test run -d ./tracetest/tests/test-api-and-av.yaml -w - tracetest test run -d ./tracetest/tests/transaction-api.yaml -w - - - name: Stop containers - if: always() - run: docker compose -f "docker-compose.yaml" -f "tracetest/docker-compose.yaml" down -v -``` - -## Project structure - -Let's first explain how the example app is built. It uses Docker Compose, and contains two distinct `docker-compose.yaml` files. As you see in the GitHub Actions Workflow, all services in the example app are started with Docker Compose. - -### 1. Node.js app - -The `docker-compose.yaml` file and `Dockerfile` in the root directory are for the Node.js app. - -### 2. Tracetest - -The `docker-compose.yaml` file, `collector.config.yaml`, `tracetest-provision.yaml`, and `tracetest.config.yaml` in the `tracetest` directory are for the setting up Tracetest and the OpenTelemetry Collector. - -The `tracetest` directory is self-contained and will run all the prerequisites for enabling OpenTelemetry traces and trace-based testing with Tracetest. - -### Docker Compose Network - -All `services` in the `docker-compose.yaml` are on the same network and will be reachable by hostname from within other services. E.g. `tracetest:4317` in the `collector.config.yaml` will map to the `tracetest` service, where the port `4317` is the port where Tracetest accepts traces. - -## Node.js app - -The Node.js app is a simple Express app with two microservices, contained in the `app.js` and `availability.js` files. - -The OpenTelemetry tracing is contained in the `tracing.otel.grpc.js` or `tracing.otel.http.js` files, respectively. -Traces will be sent to the OpenTelemetry Collector. - -Here's the content of the `tracing.otel.grpc.js` file: - -```js -const opentelemetry = require("@opentelemetry/sdk-node"); -const { - getNodeAutoInstrumentations, -} = require("@opentelemetry/auto-instrumentations-node"); -const { - OTLPTraceExporter, -} = require("@opentelemetry/exporter-trace-otlp-grpc"); -const { Resource } = require("@opentelemetry/resources"); -const { - SemanticResourceAttributes, -} = require("@opentelemetry/semantic-conventions"); -const { NodeTracerProvider } = require("@opentelemetry/sdk-trace-node"); -const { BatchSpanProcessor } = require("@opentelemetry/sdk-trace-base"); - -const resource = Resource.default().merge( - new Resource({ - [SemanticResourceAttributes.SERVICE_NAME]: - "quick-start-nodejs-manual-instrumentation", - [SemanticResourceAttributes.SERVICE_VERSION]: "0.0.1", - }) -); - -const provider = new NodeTracerProvider({ resource: resource }); -const exporter = new OTLPTraceExporter({ url: "http://otel-collector:4317" }); -const processor = new BatchSpanProcessor(exporter); -provider.addSpanProcessor(processor); -provider.register(); - -const sdk = new opentelemetry.NodeSDK({ - traceExporter: exporter, - instrumentations: [getNodeAutoInstrumentations()], - serviceName: "quick-start-nodejs-manual-instrumentation", -}); -sdk.start(); -``` - -Depending on which of these you choose, traces will be sent to either the `grpc` or `http` endpoint. - -The hostnames and ports for these are: - -- GRPC: `http://otel-collector:4317` -- HTTP: `http://otel-collector:4318/v1/traces` - -Enabling the tracer is done by preloading the trace file. - -```bash -node -r ./tracing.otel.grpc.js app.js -``` - -In the `package.json` you will see two npm script for running the respective tracers alongside the `app.js`. - -```json -"scripts": { - "app-with-grpc-tracer": "node -r ./tracing.otel.grpc.js app.js", - "app-with-http-tracer": "node -r ./tracing.otel.http.js app.js", - "availability-with-grpc-tracer": "node -r ./tracing.otel.grpc.js availability.js", - "availability-with-http-tracer": "node -r ./tracing.otel.http.js availability.js" -}, -``` - -To start the `app.js` Express server you run this command. - -```bash -npm run app-with-grpc-tracer -# or -npm run app-with-http-tracer -``` - -To start the `availability.js` Express server you run this command. - -```bash -npm run availability-with-grpc-tracer -# or -npm run availability-with-http-tracer -``` - -As you can see the `Dockerfile` does not have a `CMD` section. - -```Dockerfile -FROM node:slim -WORKDIR /usr/src/app -COPY package*.json ./ -RUN npm install -COPY . . -EXPOSE 8080 -``` - -Instead, the `docker-compose.yaml` contains the `CMD` section for both services. - -```yaml -version: "3" -services: - app: - image: quick-start-nodejs - build: . - command: npm run app-with-grpc-tracer - ports: - - "8080:8080" - availability: - image: quick-start-nodejs-availability - build: . - command: npm run availability-with-grpc-tracer - ports: - - "8080" -``` - -To start it, run this command: - -```bash -docker compose build # optional if you haven't already built the image -docker compose up -``` - -This will start the Node.js app. But, you're not sending the traces anywhere. - -Let's fix this by configuring Tracetest and OpenTelemetry Collector. - -## Tracetest - -The `docker-compose.yaml` in the `tracetest` directory is configured with three services. - -- **Postgres** - Postgres is a prerequisite for Tracetest to work. It stores trace data when running the trace-based tests. -- [**OpenTelemetry Collector**](https://opentelemetry.io/docs/collector/) - A vendor-agnostic implementation of how to receive, process and export telemetry data. -- [**Tracetest**](https://tracetest.io/) - Trace-based testing that generates end-to-end tests automatically from traces. - -```yaml -version: "3" -services: - tracetest: - image: kubeshop/tracetest:latest - platform: linux/amd64 - volumes: - - type: bind - source: ./tracetest/tracetest-config.yaml - target: /app/tracetest.yaml - - type: bind - source: ./tracetest/tracetest-provision.yaml - target: /app/provisioning.yaml - ports: - - 11633:11633 - command: --provisioning-file /app/provisioning.yaml - depends_on: - postgres: - condition: service_healthy - otel-collector: - condition: service_started - healthcheck: - test: ["CMD", "wget", "--spider", "localhost:11633"] - interval: 1s - timeout: 3s - retries: 60 - environment: - TRACETEST_DEV: ${TRACETEST_DEV} - - postgres: - image: postgres:14 - environment: - POSTGRES_PASSWORD: postgres - POSTGRES_USER: postgres - healthcheck: - test: pg_isready -U "$$POSTGRES_USER" -d "$$POSTGRES_DB" - interval: 1s - timeout: 5s - retries: 60 - - otel-collector: - image: otel/opentelemetry-collector-contrib:0.59.0 - command: - - "--config" - - "/otel-local-config.yaml" - volumes: - - ./tracetest/collector.config.yaml:/otel-local-config.yaml -``` - -Tracetest depends on both Postgres and the OpenTelemetry Collector. Both Tracetest and the OpenTelemetry Collector require config files to be loaded via a volume. The volumes are mapped from the root directory into the `tracetest` directory and the respective config files. - -The `tracetest-config.yaml` file contains the basic setup of connecting Tracetest to the Postgres instance. - -```yaml -postgres: - host: postgres - user: postgres - password: postgres - port: 5432 - dbname: postgres - params: sslmode=disable -``` - -The `tracetest-provision.yaml` file provisions the trace data store and polling to store in the Postgres database. The data store is set to OTLP meaning the traces will be stored in Tracetest itself. - -```yaml ---- -type: PollingProfile -spec: - name: Default - strategy: periodic - default: true - periodic: - retryDelay: 5s - timeout: 10m - ---- -type: DataStore -spec: - name: OpenTelemetry Collector - type: otlp - default: true -``` - -But how are traces sent to Tracetest? - -The `collector.config.yaml` explains that. It receives traces via either `grpc` or `http`. Then, exports them to Tracetest's otlp endpoint `tracetest:4317`. - -```yaml -receivers: - otlp: - protocols: - grpc: - http: - -processors: - batch: - timeout: 100ms - -exporters: - logging: - loglevel: debug - otlp/1: - endpoint: tracetest:4317 - # Send traces to Tracetest. - # Read more in docs here: https://docs.tracetest.io/configuration/connecting-to-data-stores/opentelemetry-collector - tls: - insecure: true - -service: - pipelines: - traces/1: - receivers: [otlp] - processors: [batch] - exporters: [otlp/1] -``` - -## Run both the Node.js app and Tracetest - -To start both the Node.js services and Tracetest we will run this command: - -```bash -docker-compose -f docker-compose.yaml -f tracetest/docker-compose.yaml up # add --build if the images are not built already -``` - -This will start your Tracetest instance on `http://localhost:11633/`. - -## Run Tracetest tests with the Tracetest CLI - -First, [install the CLI](https://docs.tracetest.io/getting-started/installation#install-the-tracetest-cli). -Then, configure the CLI: - -```bash -tracetest configure --endpoint http://localhost:11633 -``` - -Once configured, you can run a test against the Tracetest instance via the terminal. - -Check out the `test-api.yaml` file in the `./tracetest/tests` directory. - -```yaml -type: Test -spec: - id: W656Q0c4g - name: Books List - description: List of books - trigger: - type: http - httpRequest: - url: http://app:8080/books - method: GET - headers: - - key: Content-Type - value: application/json - specs: - - selector: span[tracetest.span.type="http" name="GET /books" http.target="/books" http.method="GET"] - assertions: - - attr:http.status_code = 200 - - selector: span[tracetest.span.type="general" name="Books List"] - assertions: - - attr:books.list.count = 4 -``` - -To run the test, run this command in the terminal: - -```bash -tracetest test run -d ./tracetest/tests/test-api.yaml -w -``` - -This test will fail just like the sample above due to the `attr:books.list.count = 4` assertion. - -``` -✘ http://app:8080 (http://localhost:11633/test/W656Q0c4g/run/5/test) - ✔ span[tracetest.span.type="http" name="GET /books" http.target="/books" http.method="GET"] - ✔ #994c63e0ea35e632 - ✔ attr:http.status_code = 200 (200) - ✘ span[tracetest.span.type="general" name="Books List"] - ✘ #5ab1856c32b0d5c8 - ✘ attr:books.list.count = 4 (3) (http://localhost:11633/test/W656Q0c4g/run/5/test?selectedAssertion=1&selectedSpan=5ab1856c32b0d5c8) -``` - -The tests will pass if you change the assertion to: - -```css -attr: books.list.count = 3; -``` - -There are two more files in the `./tracetest/tests` directory that we use in the GitHub Actions Workflow. - -The test `test-api-and-av.yaml` also includes assertions for the `availability` service. - -```yaml -# ./tracetest/tests/test-api-and-av.yaml - -type: Test -spec: - id: phAZcrT4B - name: Books list with availability - description: Testing the books list and availability check - trigger: - type: http - httpRequest: - url: http://app:8080/books - method: GET - headers: - - key: Content-Type - value: application/json - specs: - - selector: span[tracetest.span.type="http" name="GET /books" http.target="/books" - http.method="GET"] - assertions: - - attr:tracetest.span.duration < 500ms - - selector: span[tracetest.span.type="general" name="Books List"] - assertions: - - attr:books.list.count = 3 - - selector: span[tracetest.span.type="http" name="GET /availability/:bookId" http.method="GET"] - assertions: - - attr:http.host = "availability:8080" - - selector: span[tracetest.span.type="general" name="Availablity check"] - assertions: - - attr:isAvailable = "true" -``` - -The transaction `transaction-api.yaml` will run both the tests above. - -```yaml -# ./tracetest/tests/transaction-api.yaml - -type: Transaction -spec: - id: 3YIB7rPVg - name: All Tests for the Books List API - steps: - - phAZcrT4W - - phAZcrT4B -``` - -Feel free to check out our [docs](https://docs.tracetest.io/), and join our [Discord Community](https://discord.gg/8MtcMrQNbX) for more info! +# GitHub Actions Pipeline + +:::note +[Check out the source code on GitHub here.](https://github.com/kubeshop/tracetest/tree/main/examples/quick-start-github-actions) +::: + +[Tracetest](https://tracetest.io/) is a testing tool based on [OpenTelemetry](https://opentelemetry.io/) that allows you to test your distributed application. It allows you to use data from distributed traces generated by OpenTelemetry to validate and assert if your application has the desired behavior defined by your test definitions. + +## GitHub Actions Workflow for Running Tracetest tests against a sample Node.js app with OpenTelemetry + +This is a simple quick start on how to configure GitHub Actions to run Tracetest tests against a Node.js app thats uses OpenTelemetry instrumentation with traces. This example includes manual instrumentation and a sample bookstore array that simulates fetching data from a database. + +## GitHub Actions Workflow + +This sample has two workflows. The workflows have one job and a total of 6 steps. The steps are: + +1. Checking out the repo code +2. Starting the sample app with Docker Compose +3. Installing the Tracetest CLI +4. Configuring the Tracetest CLI +5. Running tests with the Tracetest CLI +6. Stop Docker Compose + +The first workflow triggers a pre-merge and merge test run. + +```yaml +# start-and-test-on-main.yaml + +name: Docker Compose Start and Test on push and PR to main + +on: + push: + branches: [ "main" ] + pull_request: + branches: [ "main" ] + +jobs: + start-and-test: + timeout-minutes: 10 + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v1 + + - name: Start containers + run: docker compose -f "docker-compose.yaml" -f "tracetest/docker-compose.yaml" up -d --build + + - name: Install Tracetest CLI + shell: bash + run: curl -L https://raw.githubusercontent.com/kubeshop/tracetest/main/install-cli.sh | bash + + - name: Configure Tracetest CLI + run: tracetest configure -g --endpoint http://localhost:11633 + + - name: Run tests via the Tracetest CLI + run: | + tracetest run test -f ./tracetest/tests/test-api.yaml + tracetest run test -f ./tracetest/tests/test-api-and-av.yaml + tracetest run transaction -f ./tracetest/tests/transaction-api.yaml + + - name: Stop containers + if: always() + run: docker compose -f "docker-compose.yaml" -f "tracetest/docker-compose.yaml" down -v +``` + +And, the other is a scheduled test run. + +```yaml +# start-and-test-on-schedule.yaml + +name: Docker Compose Start and Test Every Hour + +on: + schedule: + - cron: '0 * * * *' + +jobs: + start-and-test: + timeout-minutes: 10 + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v1 + + - name: Start containers + run: docker compose -f "docker-compose.yaml" -f "tracetest/docker-compose.yaml" up -d --build + + - name: Install Tracetest CLI + shell: bash + run: curl -L https://raw.githubusercontent.com/kubeshop/tracetest/main/install-cli.sh | bash + + - name: Configure Tracetest CLI + run: tracetest configure -g --endpoint http://localhost:11633 + + - name: Run tests via the Tracetest CLI + run: | + tracetest run test -f ./tracetest/tests/test-api.yaml + tracetest run test -f ./tracetest/tests/test-api-and-av.yaml + tracetest run transaction -f ./tracetest/tests/transaction-api.yaml + + - name: Stop containers + if: always() + run: docker compose -f "docker-compose.yaml" -f "tracetest/docker-compose.yaml" down -v +``` + +## Project structure + +Let's first explain how the example app is built. It uses Docker Compose, and contains two distinct `docker-compose.yaml` files. As you see in the GitHub Actions Workflow, all services in the example app are started with Docker Compose. + +### 1. Node.js app + +The `docker-compose.yaml` file and `Dockerfile` in the root directory are for the Node.js app. + +### 2. Tracetest + +The `docker-compose.yaml` file, `collector.config.yaml`, `tracetest-provision.yaml`, and `tracetest.config.yaml` in the `tracetest` directory are for the setting up Tracetest and the OpenTelemetry Collector. + +The `tracetest` directory is self-contained and will run all the prerequisites for enabling OpenTelemetry traces and trace-based testing with Tracetest. + +### Docker Compose Network + +All `services` in the `docker-compose.yaml` are on the same network and will be reachable by hostname from within other services. E.g. `tracetest:4317` in the `collector.config.yaml` will map to the `tracetest` service, where the port `4317` is the port where Tracetest accepts traces. + +## Node.js app + +The Node.js app is a simple Express app with two microservices, contained in the `app.js` and `availability.js` files. + +The OpenTelemetry tracing is contained in the `tracing.otel.grpc.js` or `tracing.otel.http.js` files, respectively. +Traces will be sent to the OpenTelemetry Collector. + +Here's the content of the `tracing.otel.grpc.js` file: + +```js +const opentelemetry = require("@opentelemetry/sdk-node"); +const { + getNodeAutoInstrumentations, +} = require("@opentelemetry/auto-instrumentations-node"); +const { + OTLPTraceExporter, +} = require("@opentelemetry/exporter-trace-otlp-grpc"); +const { Resource } = require("@opentelemetry/resources"); +const { + SemanticResourceAttributes, +} = require("@opentelemetry/semantic-conventions"); +const { NodeTracerProvider } = require("@opentelemetry/sdk-trace-node"); +const { BatchSpanProcessor } = require("@opentelemetry/sdk-trace-base"); + +const resource = Resource.default().merge( + new Resource({ + [SemanticResourceAttributes.SERVICE_NAME]: + "quick-start-nodejs-manual-instrumentation", + [SemanticResourceAttributes.SERVICE_VERSION]: "0.0.1", + }) +); + +const provider = new NodeTracerProvider({ resource: resource }); +const exporter = new OTLPTraceExporter({ url: "http://otel-collector:4317" }); +const processor = new BatchSpanProcessor(exporter); +provider.addSpanProcessor(processor); +provider.register(); + +const sdk = new opentelemetry.NodeSDK({ + traceExporter: exporter, + instrumentations: [getNodeAutoInstrumentations()], + serviceName: "quick-start-nodejs-manual-instrumentation", +}); +sdk.start(); +``` + +Depending on which of these you choose, traces will be sent to either the `grpc` or `http` endpoint. + +The hostnames and ports for these are: + +- GRPC: `http://otel-collector:4317` +- HTTP: `http://otel-collector:4318/v1/traces` + +Enabling the tracer is done by preloading the trace file. + +```bash +node -r ./tracing.otel.grpc.js app.js +``` + +In the `package.json` you will see two npm script for running the respective tracers alongside the `app.js`. + +```json +"scripts": { + "app-with-grpc-tracer": "node -r ./tracing.otel.grpc.js app.js", + "app-with-http-tracer": "node -r ./tracing.otel.http.js app.js", + "availability-with-grpc-tracer": "node -r ./tracing.otel.grpc.js availability.js", + "availability-with-http-tracer": "node -r ./tracing.otel.http.js availability.js" +}, +``` + +To start the `app.js` Express server you run this command. + +```bash +npm run app-with-grpc-tracer +# or +npm run app-with-http-tracer +``` + +To start the `availability.js` Express server you run this command. + +```bash +npm run availability-with-grpc-tracer +# or +npm run availability-with-http-tracer +``` + +As you can see the `Dockerfile` does not have a `CMD` section. + +```Dockerfile +FROM node:slim +WORKDIR /usr/src/app +COPY package*.json ./ +RUN npm install +COPY . . +EXPOSE 8080 +``` + +Instead, the `docker-compose.yaml` contains the `CMD` section for both services. + +```yaml +version: "3" +services: + app: + image: quick-start-nodejs + build: . + command: npm run app-with-grpc-tracer + ports: + - "8080:8080" + availability: + image: quick-start-nodejs-availability + build: . + command: npm run availability-with-grpc-tracer + ports: + - "8080" +``` + +To start it, run this command: + +```bash +docker compose build # optional if you haven't already built the image +docker compose up +``` + +This will start the Node.js app. But, you're not sending the traces anywhere. + +Let's fix this by configuring Tracetest and OpenTelemetry Collector. + +## Tracetest + +The `docker-compose.yaml` in the `tracetest` directory is configured with three services. + +- **Postgres** - Postgres is a prerequisite for Tracetest to work. It stores trace data when running the trace-based tests. +- [**OpenTelemetry Collector**](https://opentelemetry.io/docs/collector/) - A vendor-agnostic implementation of how to receive, process and export telemetry data. +- [**Tracetest**](https://tracetest.io/) - Trace-based testing that generates end-to-end tests automatically from traces. + +```yaml +version: "3" +services: + tracetest: + image: kubeshop/tracetest:latest + platform: linux/amd64 + volumes: + - type: bind + source: ./tracetest/tracetest-config.yaml + target: /app/tracetest.yaml + - type: bind + source: ./tracetest/tracetest-provision.yaml + target: /app/provisioning.yaml + ports: + - 11633:11633 + command: --provisioning-file /app/provisioning.yaml + depends_on: + postgres: + condition: service_healthy + otel-collector: + condition: service_started + healthcheck: + test: ["CMD", "wget", "--spider", "localhost:11633"] + interval: 1s + timeout: 3s + retries: 60 + environment: + TRACETEST_DEV: ${TRACETEST_DEV} + + postgres: + image: postgres:14 + environment: + POSTGRES_PASSWORD: postgres + POSTGRES_USER: postgres + healthcheck: + test: pg_isready -U "$$POSTGRES_USER" -d "$$POSTGRES_DB" + interval: 1s + timeout: 5s + retries: 60 + + otel-collector: + image: otel/opentelemetry-collector-contrib:0.59.0 + command: + - "--config" + - "/otel-local-config.yaml" + volumes: + - ./tracetest/collector.config.yaml:/otel-local-config.yaml +``` + +Tracetest depends on both Postgres and the OpenTelemetry Collector. Both Tracetest and the OpenTelemetry Collector require config files to be loaded via a volume. The volumes are mapped from the root directory into the `tracetest` directory and the respective config files. + +The `tracetest-config.yaml` file contains the basic setup of connecting Tracetest to the Postgres instance. + +```yaml +postgres: + host: postgres + user: postgres + password: postgres + port: 5432 + dbname: postgres + params: sslmode=disable +``` + +The `tracetest-provision.yaml` file provisions the trace data store and polling to store in the Postgres database. The data store is set to OTLP meaning the traces will be stored in Tracetest itself. + +```yaml +--- +type: PollingProfile +spec: + name: Default + strategy: periodic + default: true + periodic: + retryDelay: 5s + timeout: 10m + +--- +type: DataStore +spec: + name: OpenTelemetry Collector + type: otlp + default: true +``` + +But how are traces sent to Tracetest? + +The `collector.config.yaml` explains that. It receives traces via either `grpc` or `http`. Then, exports them to Tracetest's otlp endpoint `tracetest:4317`. + +```yaml +receivers: + otlp: + protocols: + grpc: + http: + +processors: + batch: + timeout: 100ms + +exporters: + logging: + loglevel: debug + otlp/1: + endpoint: tracetest:4317 + # Send traces to Tracetest. + # Read more in docs here: https://docs.tracetest.io/configuration/connecting-to-data-stores/opentelemetry-collector + tls: + insecure: true + +service: + pipelines: + traces/1: + receivers: [otlp] + processors: [batch] + exporters: [otlp/1] +``` + +## Run both the Node.js app and Tracetest + +To start both the Node.js services and Tracetest we will run this command: + +```bash +docker-compose -f docker-compose.yaml -f tracetest/docker-compose.yaml up # add --build if the images are not built already +``` + +This will start your Tracetest instance on `http://localhost:11633/`. + +## Run Tracetest tests with the Tracetest CLI + +First, [install the CLI](https://docs.tracetest.io/getting-started/installation#install-the-tracetest-cli). +Then, configure the CLI: + +```bash +tracetest configure --endpoint http://localhost:11633 +``` + +Once configured, you can run a test against the Tracetest instance via the terminal. + +Check out the `test-api.yaml` file in the `./tracetest/tests` directory. + +```yaml +type: Test +spec: + id: W656Q0c4g + name: Books List + description: List of books + trigger: + type: http + httpRequest: + url: http://app:8080/books + method: GET + headers: + - key: Content-Type + value: application/json + specs: + - selector: span[tracetest.span.type="http" name="GET /books" http.target="/books" http.method="GET"] + assertions: + - attr:http.status_code = 200 + - selector: span[tracetest.span.type="general" name="Books List"] + assertions: + - attr:books.list.count = 4 +``` + +To run the test, run this command in the terminal: + +```bash +tracetest run test -f ./tracetest/tests/test-api.yaml +``` + +This test will fail just like the sample above due to the `attr:books.list.count = 4` assertion. + +``` +✘ http://app:8080 (http://localhost:11633/test/W656Q0c4g/run/5/test) + ✔ span[tracetest.span.type="http" name="GET /books" http.target="/books" http.method="GET"] + ✔ #994c63e0ea35e632 + ✔ attr:http.status_code = 200 (200) + ✘ span[tracetest.span.type="general" name="Books List"] + ✘ #5ab1856c32b0d5c8 + ✘ attr:books.list.count = 4 (3) (http://localhost:11633/test/W656Q0c4g/run/5/test?selectedAssertion=1&selectedSpan=5ab1856c32b0d5c8) +``` + +The tests will pass if you change the assertion to: + +```css +attr: books.list.count = 3; +``` + +There are two more files in the `./tracetest/tests` directory that we use in the GitHub Actions Workflow. + +The test `test-api-and-av.yaml` also includes assertions for the `availability` service. + +```yaml +# ./tracetest/tests/test-api-and-av.yaml + +type: Test +spec: + id: phAZcrT4B + name: Books list with availability + description: Testing the books list and availability check + trigger: + type: http + httpRequest: + url: http://app:8080/books + method: GET + headers: + - key: Content-Type + value: application/json + specs: + - selector: span[tracetest.span.type="http" name="GET /books" http.target="/books" + http.method="GET"] + assertions: + - attr:tracetest.span.duration < 500ms + - selector: span[tracetest.span.type="general" name="Books List"] + assertions: + - attr:books.list.count = 3 + - selector: span[tracetest.span.type="http" name="GET /availability/:bookId" http.method="GET"] + assertions: + - attr:http.host = "availability:8080" + - selector: span[tracetest.span.type="general" name="Availablity check"] + assertions: + - attr:isAvailable = "true" +``` + +The transaction `transaction-api.yaml` will run both the tests above. + +```yaml +# ./tracetest/tests/transaction-api.yaml + +type: Transaction +spec: + id: 3YIB7rPVg + name: All Tests for the Books List API + steps: + - phAZcrT4W + - phAZcrT4B +``` + +Feel free to check out our [docs](https://docs.tracetest.io/), and join our [Discord Community](https://discord.gg/8MtcMrQNbX) for more info! diff --git a/docs/docs/ci-cd-automation/overview.md b/docs/docs/ci-cd-automation/overview.md index 787f602156..8b3a6e7a9e 100644 --- a/docs/docs/ci-cd-automation/overview.md +++ b/docs/docs/ci-cd-automation/overview.md @@ -31,5 +31,5 @@ Use the command below, substituting the following placeholders: - `file-path` - The path to the saved Tracetest test. Example: `./mytest.yaml` ```bash wordWrap=true -docker run --rm -it -v$(pwd):$(pwd) -w $(pwd) --network host --entrypoint tracetest kubeshop/tracetest:latest -s test run --definition --wait-for-result +docker run --rm -it -v$(pwd):$(pwd) -w $(pwd) --network host --entrypoint tracetest kubeshop/tracetest:latest -s run test --file ``` diff --git a/docs/docs/ci-cd-automation/tekton-pipeline.md b/docs/docs/ci-cd-automation/tekton-pipeline.md index 414858feba..63fcffaf48 100644 --- a/docs/docs/ci-cd-automation/tekton-pipeline.md +++ b/docs/docs/ci-cd-automation/tekton-pipeline.md @@ -243,7 +243,7 @@ spec: script: | # Configure and Run Tracetest CLI tracetest configure -g --endpoint http://tracetest.tracetest.svc.cluster.local:11633/ - tracetest test run -d /workspace/test-api.yaml -w + tracetest run test -f /workspace/test-api.yaml volumeMounts: - name: custom mountPath: /workspace diff --git a/docs/docs/ci-cd-automation/testkube-pipeline.md b/docs/docs/ci-cd-automation/testkube-pipeline.md index f12ffd20e2..350068686e 100644 --- a/docs/docs/ci-cd-automation/testkube-pipeline.md +++ b/docs/docs/ci-cd-automation/testkube-pipeline.md @@ -314,7 +314,7 @@ Duration: Getting logs from test job 641885f39922b3e1003dd5b6 Execution completed 🔬 Executing in directory : - $ tracetest test run --server-url http://10.96.93.106:11633 --definition /tmp/test-content737616681 --wait-for-result --output pretty + $ tracetest run test --server-url http://10.96.93.106:11633 --file /tmp/test-content737616681 --output pretty ✘ Pokeshop - List (http://10.96.93.106:11633/test/RUkKQ_aVR/run/2/test) ✘ Database queries less than 500 ms ✘ #2b213392d0e3ff21 @@ -359,7 +359,7 @@ Duration: Getting logs from test job 6418873d9922b3e1003dd5b8 Execution completed 🔬 Executing in directory : - $ tracetest test run --server-url http://10.96.93.106:11633 --definition /tmp/test-content1901459587 --wait-for-result --output pretty + $ tracetest run test --server-url http://10.96.93.106:11633 --file /tmp/test-content1901459587 --output pretty ✔ Pokeshop - List (http://10.96.93.106:11633/test/RUkKQ_aVR/run/3/test) ✔ Database queries less than 500 ms diff --git a/docs/docs/cli/configuring-your-cli.md b/docs/docs/cli/configuring-your-cli.md index 07b06cbe83..3a97a6482e 100644 --- a/docs/docs/cli/configuring-your-cli.md +++ b/docs/docs/cli/configuring-your-cli.md @@ -1,66 +1,62 @@ -# Configuring your CLI - -Our web interface makes it easier to visualize your traces and add assertions, but sometimes a CLI is needed for automation. The CLI was developed for users creating tests and executing them each time a change is made in the system, so Tracetest can detect regressions and check service Service Level Objectives (SLOs). - -## Available Commands - -Here is a list of all available commands and how to use them: - -### Configure - -Configure your CLI to connect to your Tracetest server. - -**How to Use**: - -```sh -tracetest configure -``` - -If you want to set values without having to answer questions from a prompt, you can provide the flag `--endpoint` to define the server endpoint. - -```sh -tracetest configure --endpoint http://my-tracetest-server:11633 -``` - -### Test List - -Allows you to list all tests. - -**How to Use**: - -```sh -tracetest list test -``` - -### Run a Test - -Allows you to run a test by referencing a [test definition file](./creating-tests). - -> Note: If the definition file contains the field `id`, this command will not create a new test. Instead, it will update the test with that ID. If that test doesn't exist, a new one will be created with that ID on the server. - -Every time the test is run, changes are detected and, if any change is introduced, we use Tractest's [versioning](../concepts/versioning) mechanism to ensure that it will not cause problems with previous test runs. - -**How to Use**: - -```sh -tracetest test run --definition -``` - -**Options**: - -`--wait-for-result`: The CLI will only exit after the test run has completed (the trace was retrieved and assertions were executed). - -### Running Tracetest CLI From Docker - -There are times when it is easier to directly execute the Tracetest CLI from a Docker image rather than installing the CLI on your local machine. This can be convenient when you wish to execute the CLI in a CI/CD environment. - -**How to Use**: - -Use the command below, substituting the following placeholders: - -- `your-tracetest-server-url` - The URL to the running Tracetest server you wish to execute the test on. Example: `http://localhost:11633/` -- `file-path` - The path to the saved Tracetest test. Example: `./mytest.yaml` - -```bash wordWrap=true -docker run --rm -it -v$(pwd):$(pwd) -w $(pwd) --network host --entrypoint tracetest kubeshop/tracetest:latest -s test run --definition --wait-for-result -``` +# Configuring your CLI + +Our web interface makes it easier to visualize your traces and add assertions, but sometimes a CLI is needed for automation. The CLI was developed for users creating tests and executing them each time a change is made in the system, so Tracetest can detect regressions and check your Service Level Objectives (SLOs). + +## Available Commands + +Here is a list of all available commands and how to use them: + +### Configure + +Configure your CLI to connect to your Tracetest server. + +**How to Use**: + +```sh +tracetest configure +``` + +If you want to set values without having to answer questions from a prompt, you can provide the flag `--endpoint` to define the server endpoint. + +```sh +tracetest configure --endpoint http://my-tracetest-server:11633 +``` + +### Test List + +Allows you to list all tests. + +**How to Use**: + +```sh +tracetest list test +``` + +### Run a Test + +Allows you to run a test by referencing a [test definition file](./creating-tests). + +> Note: If the definition file contains the field `id`, this command will not create a new test. Instead, it will update the test with that ID. If that test doesn't exist, a new one will be created with that ID on the server. + +Every time the test is run, changes are detected and, if any change is introduced, we use Tractest's [versioning](../concepts/versioning) mechanism to ensure that it will not cause problems with previous test runs. + +**How to Use**: + +```sh +tracetest run test --file +``` + +### Running Tracetest CLI from Docker + +There are times when it is easier to directly execute the Tracetest CLI from a Docker image rather than installing the CLI on your local machine. This can be convenient when you wish to execute the CLI in a CI/CD environment. + +**How to Use**: + +Use the command below, substituting the following placeholders: + +- `your-tracetest-server-url` - The URL to the running Tracetest server you wish to execute the test on. Example: `http://localhost:11633/` +- `file-path` - The path to the saved Tracetest test. Example: `./mytest.yaml` + +```bash wordWrap=true +docker run --rm -it -v$(pwd):$(pwd) -w $(pwd) --network host --entrypoint tracetest kubeshop/tracetest:latest -s run test --file +``` diff --git a/docs/docs/cli/running-tests.md b/docs/docs/cli/running-tests.md index b34e5a0f67..56b876595e 100644 --- a/docs/docs/cli/running-tests.md +++ b/docs/docs/cli/running-tests.md @@ -4,24 +4,14 @@ Once you have created a test, whether from the Tracetest UI or via a text editor The documentation for running a test via the CLI can be found here: -- [tracetest test run](./reference/tracetest_test_run.md): This page provides examples of using this command. +- [tracetest run](./reference/tracetest_run.md): This page provides examples of using this command. ## Running Your First Test -To run a test, give the path to the test definition file with the `'-d'` option. This will launch a test and provide a link to the created test run. +To run a test, give the path to the test definition file with the `'-f'` option. This will launch a test and provide a link to the created test run. ```sh -tracetest test run -d path/to/test.yaml -``` - -```text title="Output:" -✔ Pokeshop - Import (http://localhost:11633/test/4oI08rA4g/run/3/test) -``` - -Now, let's run the same test but tell the CLI to wait for the test to complete running before returning with the `'-w'` option. This will provide results from the test. - -```sh -tracetest test run -d path/to/test.yaml -w +tracetest run test -f path/to/test.yaml ``` ```text title="Output:" @@ -44,7 +34,7 @@ tracetest test run -d path/to/test.yaml -w Running the same command with the '-o json' option would change the output from the default of human readable 'pretty' to 'json'. This can be useful when you wish to extract particular data from the response. This would look like: ```sh -tracetest test run -d path/to/test.yaml -w -o json +tracetest run test -f path/to/test.yaml -o json ``` ```json title="Output:" @@ -173,7 +163,7 @@ tracetest test run -d path/to/test.yaml -w -o json You can also opt to output the result as JUnit to a file. You would run the command with a -j option and a file name, ie: ```sh -tracetest test run -d path/to/test.yaml -w -j junit.out +tracetest run test -f path/to/test.yaml -j junit.out ``` The JUnit output file would then contain the JUnit result, for example: @@ -207,7 +197,7 @@ You can reference an existing environment using its id. For example, given this We can run a test and specify that environment with this command: ```sh -tracetest test run -d path/to/test.yaml -e testenv -w +tracetest run test -f path/to/test.yaml -e testenv ``` You can also reference an environment resource file which will be used to create a new environment or update an existing one. For example, if you have a file named local.env with this content: @@ -225,7 +215,7 @@ spec: ``` ```sh -tracetest test run -d path/to/test.yaml -e path/to/local.env -w +tracetest run test -f path/to/test.yaml -e path/to/local.env ``` If you use the environment resource approach, a new environment will be created in Tracetest. diff --git a/docs/docs/cli/running-transactions.md b/docs/docs/cli/running-transactions.md index 91abfb388d..252d104fda 100644 --- a/docs/docs/cli/running-transactions.md +++ b/docs/docs/cli/running-transactions.md @@ -6,26 +6,15 @@ The command to run a transaction is the same as running a test from the CLI. The documentation for running a test via the CLI can be found here: -- [tracetest test run](./reference/tracetest_test_run.md): This page provides examples of using this command. +- [tracetest run](./reference/tracetest_run.md): This page provides examples of using this command. ## Running Your First Transaction -To run a transaction, give the path to the transaction definition file with the `'-d'` option. This will launch a transaction, providing us with a link to the created transaction run. +To run a transaction, give the path to the transaction definition file with the `'-f'` option. This will launch a transaction, providing us with a link to the created transaction run. ```sh -tracetest test run -d path/to/transaction.yaml +tracetest run transaction -f path/to/transaction.yaml ``` - -```text title="Output:" -✔ Pokemon Transaction (http://localhost:11633/transaction/xcGqfHl4g/run/3) -``` - -Now, let's run the same test but tell the CLI to wait for the test to complete running before returning with the `'-w'` option. This will provide results from the test. - -```sh -tracetest test run -d path/to/transaction.yaml -w -``` - ```text title="Output:" ✔ Pokemon Transaction (http://localhost:11633/transaction/xcGqfHl4g/run/3) ✔ Pokeshop - Import (http://localhost:11633/test/XRHjfH_4R/run/4/test) @@ -43,7 +32,7 @@ You can reference an existing environment using its id. For example, given this We can run a transaction and specify that environment with this command: ```sh -tracetest test run -d path/to/transaction.yaml -e testenv -w +tracetest run transaction -f path/to/transaction.yaml -e testenv ``` You can also reference an environment resource file which will be used to create a new environment or update an existing one. For example, if you have a file named `local.env` with this content: @@ -61,7 +50,7 @@ spec: ``` ```sh -tracetest test run -d path/to/transaction.yaml -e path/to/local.env -w +tracetest run test -f path/to/transaction.yaml -e path/to/local.env ``` If you use the environment resource approach, a new environment will be created in Tracetest. diff --git a/docs/docs/cli/undefined-variables.md b/docs/docs/cli/undefined-variables.md index f14c760c89..8a2072f1e6 100644 --- a/docs/docs/cli/undefined-variables.md +++ b/docs/docs/cli/undefined-variables.md @@ -3,7 +3,7 @@ When a user runs a test or a transaction, any variables that will be needed but are not defined will be prompted for: ```sh -tracetest test run -d path/to/test.yaml +tracetest run test -f path/to/test.yaml ``` ```text title="Output:" @@ -44,7 +44,7 @@ spec: ``` ```sh -tracetest test run -d path/to/test.yaml -e testenv +tracetest run test -f path/to/test.yaml -e testenv ``` ```text title="Output:" diff --git a/docs/docs/examples-tutorials/recipes/running-tracetest-with-aws-terraform.md b/docs/docs/examples-tutorials/recipes/running-tracetest-with-aws-terraform.md index 3ce925a229..f482d309b7 100644 --- a/docs/docs/examples-tutorials/recipes/running-tracetest-with-aws-terraform.md +++ b/docs/docs/examples-tutorials/recipes/running-tracetest-with-aws-terraform.md @@ -279,7 +279,7 @@ Now that all of the required services and infra have been created, you can start 1. From the Terraform output you can copy the `api_endpoint` and replace the `` placeholder from the `tests/test.yaml` file. 2. Configure the [Tracetest CLI](https://docs.tracetest.io/cli/configuring-your-cli) to point to the public load balancer endpoint with `tracetest configure --endpoint `. -3. Run the test YAML file using the CLI `tracetest test run -d tests/test.yaml`. +3. Run the test YAML file using the CLI `tracetest run test -f tests/test.yaml`. 4. Follow the link to find the results. ## Learn More diff --git a/docs/docs/examples-tutorials/recipes/running-tracetest-with-datadog.md b/docs/docs/examples-tutorials/recipes/running-tracetest-with-datadog.md index aa1579a03d..2f1c7f31cd 100644 --- a/docs/docs/examples-tutorials/recipes/running-tracetest-with-datadog.md +++ b/docs/docs/examples-tutorials/recipes/running-tracetest-with-datadog.md @@ -318,7 +318,7 @@ This file defines a test the same way you would through the Web UI. To run the test, run this command in the terminal: ```bash -tracetest test run -d ./http-test.yaml -w +tracetest run test -f ./http-test.yaml ``` This test will fail just like the sample above due to the `attr:tracetest.span.duration < 10ms` assertion. diff --git a/docs/docs/examples-tutorials/recipes/running-tracetest-with-honeycomb.md b/docs/docs/examples-tutorials/recipes/running-tracetest-with-honeycomb.md index 3d6734b127..098981f7f5 100644 --- a/docs/docs/examples-tutorials/recipes/running-tracetest-with-honeycomb.md +++ b/docs/docs/examples-tutorials/recipes/running-tracetest-with-honeycomb.md @@ -315,7 +315,7 @@ This file defines the a test the same way you would through the Web UI. To run the test, run this command in the terminal: ```bash -tracetest test run -d ./test-api.yaml -w +tracetest run test -f ./test-api.yaml ``` ```bash diff --git a/docs/docs/examples-tutorials/recipes/running-tracetest-with-lightstep.md b/docs/docs/examples-tutorials/recipes/running-tracetest-with-lightstep.md index 58754781dd..dd64382aa9 100644 --- a/docs/docs/examples-tutorials/recipes/running-tracetest-with-lightstep.md +++ b/docs/docs/examples-tutorials/recipes/running-tracetest-with-lightstep.md @@ -300,7 +300,7 @@ This file defines the a test the same way you would through the Web UI. To run the test, run this command in the terminal: ```bash -tracetest test run -d ./http-test.yaml -w +tracetest run test -f ./http-test.yaml ``` This test will fail just like the sample above due to the `attr:tracetest.span.duration < 50ms` assertion. diff --git a/docs/docs/examples-tutorials/recipes/running-tracetest-with-new-relic.md b/docs/docs/examples-tutorials/recipes/running-tracetest-with-new-relic.md index 18ebf24aa2..d73af65c03 100644 --- a/docs/docs/examples-tutorials/recipes/running-tracetest-with-new-relic.md +++ b/docs/docs/examples-tutorials/recipes/running-tracetest-with-new-relic.md @@ -309,7 +309,7 @@ This file defines the a test the same way you would through the Web UI. To run the test, run this command in the terminal: ```bash -tracetest test run -d ./http-test.yaml -w +tracetest run test -f ./http-test.yaml ``` This test will fail just like the sample above due to the `attr:tracetest.span.duration < 50ms` assertion. diff --git a/docs/docs/examples-tutorials/recipes/running-tracetest-with-step-functions-terraform.md b/docs/docs/examples-tutorials/recipes/running-tracetest-with-step-functions-terraform.md index e68ab7c38a..cab3ca27a7 100644 --- a/docs/docs/examples-tutorials/recipes/running-tracetest-with-step-functions-terraform.md +++ b/docs/docs/examples-tutorials/recipes/running-tracetest-with-step-functions-terraform.md @@ -161,9 +161,9 @@ Now that all of the required services and infra have been created, you can start 3. Run the tests YAML file using the CLI. ```bash - tracetest test run -d tests/incident.yaml \ - tracetest test run -d tests/exam.yaml \ - tracetest test run -d tests/transaction.yaml + tracetest run test -f tests/incident.yaml \ + tracetest run test -f tests/exam.yaml \ + tracetest run transaction -f tests/transaction.yaml ``` 4. Follow the link to find the results. diff --git a/docs/docs/examples-tutorials/recipes/running-tracetest-with-tekton.md b/docs/docs/examples-tutorials/recipes/running-tracetest-with-tekton.md index fd3a60bbf3..d6cad18367 100644 --- a/docs/docs/examples-tutorials/recipes/running-tracetest-with-tekton.md +++ b/docs/docs/examples-tutorials/recipes/running-tracetest-with-tekton.md @@ -241,7 +241,7 @@ spec: script: | # Configure and Run Tracetest CLI tracetest configure -g --endpoint http://tracetest.tracetest.svc.cluster.local:11633/ - tracetest test run -d /workspace/test-api.yaml -w + tracetest run test -f /workspace/test-api.yaml volumeMounts: - name: custom mountPath: /workspace diff --git a/docs/docs/examples-tutorials/recipes/running-tracetest-without-a-trace-data-store-with-manual-instrumentation.md b/docs/docs/examples-tutorials/recipes/running-tracetest-without-a-trace-data-store-with-manual-instrumentation.md index 24e2e6954d..87bfc0e12c 100644 --- a/docs/docs/examples-tutorials/recipes/running-tracetest-without-a-trace-data-store-with-manual-instrumentation.md +++ b/docs/docs/examples-tutorials/recipes/running-tracetest-without-a-trace-data-store-with-manual-instrumentation.md @@ -352,7 +352,7 @@ This file defines the a test the same way you would through the Web UI. To run the test, run this command in the terminal: ```bash -tracetest test run -d ./test-api.yaml -w +tracetest run test -f ./test-api.yaml ``` This test will fail just like the sample above due to the `attr:books.list.count = 4` assertion. diff --git a/docs/docs/getting-started/open.md b/docs/docs/getting-started/open.md index 0793c0f858..91331c0f84 100644 --- a/docs/docs/getting-started/open.md +++ b/docs/docs/getting-started/open.md @@ -125,7 +125,7 @@ spec: Using the CLI, trigger a test run. ```bash title="Terminal" -tracetest test run -d pokeshop_import.yaml --wait-for-result -o pretty +tracetest run test -f pokeshop_import.yaml -o pretty ``` ### Output diff --git a/docs/docs/live-examples/opentelemetry-store/use-cases/add-item-into-shopping-cart.md b/docs/docs/live-examples/opentelemetry-store/use-cases/add-item-into-shopping-cart.md index c4c8572d87..feda9da4e3 100644 --- a/docs/docs/live-examples/opentelemetry-store/use-cases/add-item-into-shopping-cart.md +++ b/docs/docs/live-examples/opentelemetry-store/use-cases/add-item-into-shopping-cart.md @@ -61,7 +61,7 @@ Now, you can validate this entire use case. To replicate this entire test on Tracetest, you can replicate these steps on our Web UI or using our CLI, saving the following test definition as the file `test-definition.yml` and later running: ```sh -tracetest test -d test-definition.yml --wait-for-results +tracetest run test -f test-definition.yml ``` We are assuming that the Frontend service is exposed on `http://otel-demo-frontend:8080`: diff --git a/docs/docs/live-examples/opentelemetry-store/use-cases/check-shopping-cart-contents.md b/docs/docs/live-examples/opentelemetry-store/use-cases/check-shopping-cart-contents.md index 1f7cb01927..68e13a2acd 100644 --- a/docs/docs/live-examples/opentelemetry-store/use-cases/check-shopping-cart-contents.md +++ b/docs/docs/live-examples/opentelemetry-store/use-cases/check-shopping-cart-contents.md @@ -66,7 +66,7 @@ Now you can validate this entire use case. To replicate this entire test on Tracetest, you can replicate these steps on our Web UI or using our CLI, saving the following test definition as the file `test-definition.yml` and later running: ```sh -tracetest test -d test-definition.yml --wait-for-results +tracetest run test -f test-definition.yml ``` We are assuming that the Frontend service is exposed on `http://otel-demo-frontend:8080`: diff --git a/docs/docs/live-examples/opentelemetry-store/use-cases/checkout.md b/docs/docs/live-examples/opentelemetry-store/use-cases/checkout.md index a6a2124a6e..841b9ddfb7 100644 --- a/docs/docs/live-examples/opentelemetry-store/use-cases/checkout.md +++ b/docs/docs/live-examples/opentelemetry-store/use-cases/checkout.md @@ -91,7 +91,7 @@ Now you can validate this entire use case. To replicate this entire test on Tracetest, you can replicate these steps on our Web UI or using our CLI, saving the following test definition as the file `test-definition.yml` and later running: ```sh -tracetest test -d test-definition.yml --wait-for-results +tracetest run test -f test-definition.yml ``` We are assuming that the Frontend service is exposed on `http://otel-demo-frontend:8080`: diff --git a/docs/docs/live-examples/opentelemetry-store/use-cases/get-recommended-products.md b/docs/docs/live-examples/opentelemetry-store/use-cases/get-recommended-products.md index b7245ed0cb..3223d25615 100644 --- a/docs/docs/live-examples/opentelemetry-store/use-cases/get-recommended-products.md +++ b/docs/docs/live-examples/opentelemetry-store/use-cases/get-recommended-products.md @@ -58,7 +58,7 @@ Now you can validate this entire use case. To replicate this entire test on Tracetest, you can replicate these steps on our Web UI or using our CLI by saving the following test definition as the file `test-definition.yml` and later running: ```sh -tracetest test -d test-definition.yml --wait-for-results +tracetest run test -f test-definition.yml ``` We are assuming that the Frontend service is exposed on `http://otel-demo-frontend:8080`: diff --git a/docs/docs/live-examples/opentelemetry-store/use-cases/user-purchasing-products.md b/docs/docs/live-examples/opentelemetry-store/use-cases/user-purchasing-products.md index ea043153b7..26c97b7e79 100644 --- a/docs/docs/live-examples/opentelemetry-store/use-cases/user-purchasing-products.md +++ b/docs/docs/live-examples/opentelemetry-store/use-cases/user-purchasing-products.md @@ -175,7 +175,7 @@ spec: By having the test, transaction and environment files in the same directory, we can call the CLI and execute this transaction: ```sh -tracetest test run -d transaction.yaml -e user-buying-products.env --wait-for-result +tracetest run transaction -f transaction.yaml -e user-buying-products.env ``` The result should be an output like this: diff --git a/docs/docs/live-examples/pokeshop/use-cases/add-pokemon.md b/docs/docs/live-examples/pokeshop/use-cases/add-pokemon.md index e88bc130d2..b664ea8194 100644 --- a/docs/docs/live-examples/pokeshop/use-cases/add-pokemon.md +++ b/docs/docs/live-examples/pokeshop/use-cases/add-pokemon.md @@ -69,7 +69,7 @@ Now you can validate this entire use case. If you want to replicate this entire test on Tracetest, you can replicate these steps on our Web UI or using our CLI, saving the following test definition as the file `test-definition.yml` and later running: ```sh -tracetest test run -d test-definition.yml --wait-for-results +tracetest run test -f test-definition.yml ``` ```yaml diff --git a/docs/docs/live-examples/pokeshop/use-cases/get-pokemon-by-id.md b/docs/docs/live-examples/pokeshop/use-cases/get-pokemon-by-id.md index b3dbcf9110..105100daa0 100644 --- a/docs/docs/live-examples/pokeshop/use-cases/get-pokemon-by-id.md +++ b/docs/docs/live-examples/pokeshop/use-cases/get-pokemon-by-id.md @@ -87,7 +87,7 @@ Now you can validate this entire use case. If you want to replicate those tests on Tracetest, you can replicate these steps on our Web UI or using our CLI, saving one of the test definitions as the file `test-definition.yml` and running: ```sh -tracetest test -d test-definition.yml --wait-for-results +tracetest run test -f test-definition.yml ``` #### Cache Miss Scenario diff --git a/docs/docs/live-examples/pokeshop/use-cases/import-pokemon.md b/docs/docs/live-examples/pokeshop/use-cases/import-pokemon.md index 07397ae23c..7f295e09db 100644 --- a/docs/docs/live-examples/pokeshop/use-cases/import-pokemon.md +++ b/docs/docs/live-examples/pokeshop/use-cases/import-pokemon.md @@ -97,7 +97,7 @@ Now you can validate this entire use case. If you want to replicate this entire test on Tracetest, you can replicate these steps on our Web UI or using our CLI, saving the following test definition as the file `test-definition.yml` and later running: ```sh -tracetest test -d test-definition.yml --wait-for-results +tracetest run test -f test-definition.yml ``` ```yaml diff --git a/docs/docs/live-examples/pokeshop/use-cases/list-pokemon.md b/docs/docs/live-examples/pokeshop/use-cases/list-pokemon.md index cdf49cc1ee..8440363735 100644 --- a/docs/docs/live-examples/pokeshop/use-cases/list-pokemon.md +++ b/docs/docs/live-examples/pokeshop/use-cases/list-pokemon.md @@ -69,7 +69,7 @@ Now you can validate this entire use case. If you want to replicate this entire test on Tracetest, you can replicate these steps on our Web UI or using our CLI, saving the following test definition as the file `test-definition.yml` and later running: ```sh -tracetest test -d test-definition.yml --wait-for-results +tracetest run test -f test-definition.yml ``` ```yaml diff --git a/docs/docs/tools-and-integrations/k6.md b/docs/docs/tools-and-integrations/k6.md index 27a63e79d7..fea7d8c3d2 100644 --- a/docs/docs/tools-and-integrations/k6.md +++ b/docs/docs/tools-and-integrations/k6.md @@ -96,7 +96,7 @@ spec: And create the test using the following command: ```bash -tracetest test run -d +tracetest run test -f ``` ### Extension Features diff --git a/docs/docs/tools-and-integrations/keptn.md b/docs/docs/tools-and-integrations/keptn.md index 5abcf92f8e..0d2193bc4b 100644 --- a/docs/docs/tools-and-integrations/keptn.md +++ b/docs/docs/tools-and-integrations/keptn.md @@ -25,9 +25,8 @@ actions: - /keptn/data/tracetest-cli-config.yaml - test - run - - --definition + - --file - /keptn/data/test-definition.yaml - - --wait-for-result ``` @@ -144,9 +143,8 @@ actions: - /keptn/data/tracetest-cli-config.yaml - test - run - - --definition + - --file - /keptn/data/test-definition.yaml - - --wait-for-result ``` This job will run Tracetest every time a `test` event happens, listening to the event `sh.keptn.event.test.triggered` (event emitted by the `test` task on the `validate-pokeshop` sequence). diff --git a/examples/observability-to-the-rescue/README.md b/examples/observability-to-the-rescue/README.md index fcb458520a..041003e094 100644 --- a/examples/observability-to-the-rescue/README.md +++ b/examples/observability-to-the-rescue/README.md @@ -72,11 +72,11 @@ curl --location 'http://localhost:10013/executePaymentOrder' \ There are two tests that you can do to check how these APIs are working, one is the `test-with-error`, that calls `your-api` passing the `yearsAsACustomer` field as zero, causing a error propagation into the API calls: ```sh -tracetest test run -w -d ./tracetest/tests/test-with-error.yaml +tracetest run test -f ./tracetest/tests/test-with-error.yaml ``` The second one is `test-with-success`, with the field `yearsAsACustomer` greater than 0, causing the services to behave normally: ```sh -tracetest test run -w -d ./tracetest/tests/test-with-success.yaml +tracetest run test -f ./tracetest/tests/test-with-success.yaml ``` diff --git a/examples/quick-start-github-actions/.github/workflows/start-and-test-on-main.yaml b/examples/quick-start-github-actions/.github/workflows/start-and-test-on-main.yaml index 7acdd03f1e..710b0069ba 100644 --- a/examples/quick-start-github-actions/.github/workflows/start-and-test-on-main.yaml +++ b/examples/quick-start-github-actions/.github/workflows/start-and-test-on-main.yaml @@ -28,7 +28,7 @@ jobs: run: | tracetest run test -f ./tracetest/tests/test-api.yaml tracetest run test -f ./tracetest/tests/test-api-and-av.yaml - tracetest run test -f ./tracetest/tests/transaction-api.yaml + tracetest run transaction test -f ./tracetest/tests/transaction-api.yaml - name: Stop containers if: always() diff --git a/examples/quick-start-github-actions/.github/workflows/start-and-test-on-schedule.yaml b/examples/quick-start-github-actions/.github/workflows/start-and-test-on-schedule.yaml index fbdd065ab6..7350f8cb69 100644 --- a/examples/quick-start-github-actions/.github/workflows/start-and-test-on-schedule.yaml +++ b/examples/quick-start-github-actions/.github/workflows/start-and-test-on-schedule.yaml @@ -26,7 +26,7 @@ jobs: run: | tracetest run test -f ./tracetest/tests/test-api.yaml tracetest run test -f ./tracetest/tests/test-api-and-av.yaml - tracetest run test -f ./tracetest/tests/transaction-api.yaml + tracetest run transaction -f ./tracetest/tests/transaction-api.yaml - name: Stop containers if: always() diff --git a/examples/quick-start-net-core/README.md b/examples/quick-start-net-core/README.md index d106519d60..9d7664e025 100644 --- a/examples/quick-start-net-core/README.md +++ b/examples/quick-start-net-core/README.md @@ -9,6 +9,6 @@ This is a simple quick start on how to configure a .NET Core API to use OpenTele 1. [Install the tracetest CLI](https://github.com/kubeshop/tracetest/blob/main/docs/installing.md#cli-installation) 2. Run `tracetest configure --endpoint http://localhost:11633` on a terminal to configure the CLI to send all commands to that address 3. Run the project by using docker-compose: `docker-compose up -d` (Linux) or `docker compose up -d` (Mac) -4. Test if it works by running: `tracetest test run -d tests/test.yaml`. This would execute a test against the .NET Core API that will send spans to Jaeger to be fetched from the Tracetest server. +4. Test if it works by running: `tracetest run test -f tests/test.yaml`. This would execute a test against the .NET Core API that will send spans to Jaeger to be fetched from the Tracetest server. Feel free to check out the [docs](https://docs.tracetest.io/), and join our [Discord Community](https://discord.gg/8MtcMrQNbX) for more info! diff --git a/examples/tracetest-amazon-x-ray-adot/README.md b/examples/tracetest-amazon-x-ray-adot/README.md index 3f7231890c..5e16287e9e 100644 --- a/examples/tracetest-amazon-x-ray-adot/README.md +++ b/examples/tracetest-amazon-x-ray-adot/README.md @@ -10,4 +10,4 @@ This repository objective is to show how you can configure your Tracetest instan 2. Run `tracetest configure --endpoint http://localhost:11633` on a terminal 3. Update the `.env` file adding a valid set of AWS credentials 4. Run the project by using docker-compose: `docker-compose up -d` (Linux) or `docker compose up -d` (Mac) -5. Test if it works by running: `tracetest test run -d tests/test.yaml`. This would trigger a test that will send and retrieve spans from the X-Ray instance that is running on your machine. +5. Test if it works by running: `tracetest run test -f tests/test.yaml`. This would trigger a test that will send and retrieve spans from the X-Ray instance that is running on your machine. diff --git a/examples/tracetest-amazon-x-ray-pokeshop/README.md b/examples/tracetest-amazon-x-ray-pokeshop/README.md index 5b1abddc6d..f7427ce8e7 100644 --- a/examples/tracetest-amazon-x-ray-pokeshop/README.md +++ b/examples/tracetest-amazon-x-ray-pokeshop/README.md @@ -11,4 +11,4 @@ This repository objective is to show how you can configure your Tracetest instan 3. Update the `.env` file adding a valid set of AWS credentials 4. Update the `tracetest.provision.yaml` file adding a valid set of AWS credentials 5. Run the project by using docker-compose: `docker-compose up -d` (Linux) or `docker compose up -d` (Mac) -6. Test if it works by running: `tracetest test run -d tests/test.yaml`. This would trigger a test that will send and retrieve spans from the X-Ray instance that is running on your machine. +6. Test if it works by running: `tracetest run test -f tests/test.yaml`. This would trigger a test that will send and retrieve spans from the X-Ray instance that is running on your machine. diff --git a/examples/tracetest-amazon-x-ray/README.md b/examples/tracetest-amazon-x-ray/README.md index e00e3909b7..e1ff145569 100644 --- a/examples/tracetest-amazon-x-ray/README.md +++ b/examples/tracetest-amazon-x-ray/README.md @@ -11,4 +11,4 @@ This repository objective is to show how you can configure your Tracetest instan 3. Update the `.env` file adding a valid set of AWS credentials 4. Update the `tracetest.provision.yaml` file adding a valid set of AWS credentials 5. Run the project by using docker-compose: `docker-compose up -d` (Linux) or `docker compose up -d` (Mac) -6. Test if it works by running: `tracetest test run -d tests/test.yaml`. This would trigger a test that will send and retrieve spans from the X-Ray instance that is running on your machine. +6. Test if it works by running: `tracetest run test -f tests/test.yaml`. This would trigger a test that will send and retrieve spans from the X-Ray instance that is running on your machine. diff --git a/examples/tracetest-aws-step-functions/README.md b/examples/tracetest-aws-step-functions/README.md index 87bfaac275..35ee599aeb 100644 --- a/examples/tracetest-aws-step-functions/README.md +++ b/examples/tracetest-aws-step-functions/README.md @@ -14,9 +14,9 @@ This is a simple quick start on how to configure a .NET State Machine (AWS Step 7. Inject the Tests and Transactions definitions to the tracetest server using the following: ```bash -tracetest test run -d tests/incident.yaml \ -tracetest test run -d tests/exam.yaml \ -tracetest test run -d tests/transaction.yaml +tracetest run test -f tests/incident.yaml \ +tracetest run test -f tests/exam.yaml \ +tracetest run transaction -f tests/transaction.yaml ``` Feel free to check out the [docs](https://docs.tracetest.io/), and join our [Discord Community](https://discord.gg/8MtcMrQNbX) for more info! diff --git a/examples/tracetest-aws-step-functions/infra/README.md b/examples/tracetest-aws-step-functions/infra/README.md index db7fab625a..e752a4c9e5 100644 --- a/examples/tracetest-aws-step-functions/infra/README.md +++ b/examples/tracetest-aws-step-functions/infra/README.md @@ -10,6 +10,6 @@ This is a simple quick start on how to configure a Node.js lambda function API t 2. Run `terraform init`, `terraform apply` and accept the changes 3. From the terraform outputs, grab the `tracetes_url` and run `tracetest configure --endpoint ` on a terminal to configure the CLI to send all commands to that address 4. From the terraform outputs, grab the `api_endpoint` and update the `` section from `test/test.yaml` -5. Test if it works by running: `tracetest test run -d tests/test.yaml`. This would execute a test against the Node.js API Gateway endpoint that will send spans to Jaeger to be fetched from the Tracetest server. +5. Test if it works by running: `tracetest run test -f tests/test.yaml`. This would execute a test against the Node.js API Gateway endpoint that will send spans to Jaeger to be fetched from the Tracetest server. Feel free to check out the [docs](https://docs.tracetest.io/), and join our [Discord Community](https://discord.gg/8MtcMrQNbX) for more info! diff --git a/examples/tracetest-aws-terraform-serverless/README.md b/examples/tracetest-aws-terraform-serverless/README.md index d604606eaf..beaadb5581 100644 --- a/examples/tracetest-aws-terraform-serverless/README.md +++ b/examples/tracetest-aws-terraform-serverless/README.md @@ -8,6 +8,6 @@ This is a simple quick start on how to configure a Node.js lambda function API t 2. Run `terraform init`, `terraform apply` and accept the changes 3. From the terraform outputs, grab the `tracetes_url` and run `tracetest configure --endpoint ` on a terminal to configure the CLI to send all commands to that address 4. From the terraform outputs, grab the `api_endpoint` and update the `` section from `tests/test.yaml` -5. Test if it works by running: `tracetest test run -d tests/test.yaml`. This would execute a test against the Node.js API Gateway endpoint that will send spans to Jaeger to be fetched from the Tracetest server. +5. Test if it works by running: `tracetest run test -f tests/test.yaml`. This would execute a test against the Node.js API Gateway endpoint that will send spans to Jaeger to be fetched from the Tracetest server. Feel free to check out the [docs](https://docs.tracetest.io/), and join our [Discord Community](https://discord.gg/8MtcMrQNbX) for more info! diff --git a/examples/tracetest-azure-app-insights-collector/README.md b/examples/tracetest-azure-app-insights-collector/README.md index 7b39710df9..bc87b1ccda 100644 --- a/examples/tracetest-azure-app-insights-collector/README.md +++ b/examples/tracetest-azure-app-insights-collector/README.md @@ -10,4 +10,4 @@ This repository objective is to show how you can configure your Tracetest instan 2. Run `tracetest configure --endpoint http://localhost:11633` on a terminal 3. Update the `.env` file adding a valid set the valid App Insights Instrumentation Key 4. Run the project by using docker-compose: `docker compose -f ./docker-compose.yaml -f ./tracetest/docker-compose.yaml up -d` -5. Test if it works by running: `tracetest test run -d tests/test.yaml`. This would trigger a test that will send spans to Azure Monitor API and directly to Tracetest that is running on your machine. +5. Test if it works by running: `tracetest run test -f tests/test.yaml`. This would trigger a test that will send spans to Azure Monitor API and directly to Tracetest that is running on your machine. diff --git a/examples/tracetest-azure-app-insights-pokeshop/README.md b/examples/tracetest-azure-app-insights-pokeshop/README.md index ed18579a87..852f366442 100644 --- a/examples/tracetest-azure-app-insights-pokeshop/README.md +++ b/examples/tracetest-azure-app-insights-pokeshop/README.md @@ -11,4 +11,4 @@ This repository objective is to show how you can configure your Tracetest instan 3. Update the `.env` file adding a valid set the valid App Insights Instrumentation Key 4. Update the `tracetest.provision.yaml` file adding a valid set the Azure ARM Id and secret token 5. Run the project by using docker-compose: `docker compose -f ./docker-compose.yaml -f ./tracetest/docker-compose.yaml up -d` -6. Test if it works by running: `tracetest test run -d tests/test.yaml`. This would trigger a test that will send and retrieve spans from the Azure Monitor API instance that is running on your machine. +6. Test if it works by running: `tracetest run test -f tests/test.yaml`. This would trigger a test that will send and retrieve spans from the Azure Monitor API instance that is running on your machine. diff --git a/examples/tracetest-azure-app-insights/README.md b/examples/tracetest-azure-app-insights/README.md index a4426e325c..a7f9cc55c2 100644 --- a/examples/tracetest-azure-app-insights/README.md +++ b/examples/tracetest-azure-app-insights/README.md @@ -11,4 +11,4 @@ This repository objective is to show how you can configure your Tracetest instan 3. Update the `.env` file adding a valid set the valid App Insights Connection String 4. Update the `tracetest.provision.yaml` file adding a valid set the Azure ARM Id and secret token 5. Run the project by using docker-compose: `docker compose -f ./docker-compose.yaml -f ./tracetest/docker-compose.yaml up -d` -6. Test if it works by running: `tracetest test run -d tests/test.yaml`. This would trigger a test that will send and retrieve spans from the Azure Monitor API instance that is running on your machine. +6. Test if it works by running: `tracetest run test -f tests/test.yaml`. This would trigger a test that will send and retrieve spans from the Azure Monitor API instance that is running on your machine. diff --git a/examples/tracetest-grafana-tempo-pokeshop/README.md b/examples/tracetest-grafana-tempo-pokeshop/README.md index 2de80b0821..882a8b2a6f 100644 --- a/examples/tracetest-grafana-tempo-pokeshop/README.md +++ b/examples/tracetest-grafana-tempo-pokeshop/README.md @@ -12,7 +12,7 @@ This examples' objective is to show how you can: 1. [Install the tracetest CLI](https://docs.tracetest.io/installing/) 2. Run `tracetest configure --endpoint http://localhost:11633` on a terminal 3. Run the project by using docker-compose: `docker-compose up -d` (Linux) or `docker compose up -d` (Mac) -4. Test if it works by running: `tracetest test run -d tests/test.yaml`. This would trigger a test that will send and retrieve spans from the Grafana Tempo instance that is running on your machine. View the test on `http://localhost:11633`. +4. Test if it works by running: `tracetest run test -f tests/test.yaml`. This would trigger a test that will send and retrieve spans from the Grafana Tempo instance that is running on your machine. View the test on `http://localhost:11633`. 5. View traces in Grafana on `http://localhost:3000`. Use this TraceQL query: ```yaml diff --git a/examples/tracetest-jaeger/README.md b/examples/tracetest-jaeger/README.md index 729bee4f32..454790476f 100644 --- a/examples/tracetest-jaeger/README.md +++ b/examples/tracetest-jaeger/README.md @@ -7,4 +7,4 @@ This repository objective is to show how you can configure your tracetest instan 1. [Install the tracetest CLI](https://docs.tracetest.io/installing/) 2. Run `tracetest configure --endpoint http://localhost:11633` on a terminal 3. Run the project by using docker-compose: `docker-compose up` (Linux) or `docker compose up` (Mac) -4. Test if it works by running: `tracetest test run -d tests/list-tests.yaml`. This would trigger a test that will send and retrieve spans from the opensearch instance that is running on your machine. +4. Test if it works by running: `tracetest run test -f tests/list-tests.yaml`. This would trigger a test that will send and retrieve spans from the opensearch instance that is running on your machine. diff --git a/examples/tracetest-k6/README.md b/examples/tracetest-k6/README.md index f4af410249..89f1cb45d1 100644 --- a/examples/tracetest-k6/README.md +++ b/examples/tracetest-k6/README.md @@ -9,7 +9,7 @@ For more detailed information about the K6 Tracetest Binary take a look a the [d 1. [Install the Tracetest CLI](https://docs.tracetest.io/installing/) 2. Run `tracetest configure --endpoint http://localhost:11633` on a terminal 3. Run the project by using docker-compose: `docker-compose up -d` (Linux) or `docker compose up -d` (Mac) -4. Test if it works by running: `tracetest test run -d tests/test.yaml`. This will create and run a test with trace id as trigger +4. Test if it works by running: `tracetest run test -f tests/test.yaml`. This will create and run a test with trace id as trigger 5. In as separate folder outside of the the Tracetest repo, build the k6 binary with the extension by using `xk6 build v0.42.0 --with github.com/kubeshop/xk6-tracetest` 6. Now you are ready to run your load test, you can achieve this by running the following command: `path/to/binary/k6 run import-pokemon.js -o xk6-tracetest` 7. After the load test finishes you should be able to see an output like the following: diff --git a/examples/tracetest-lightstep-otel-demo/README.md b/examples/tracetest-lightstep-otel-demo/README.md index cfc955d596..881963709e 100644 --- a/examples/tracetest-lightstep-otel-demo/README.md +++ b/examples/tracetest-lightstep-otel-demo/README.md @@ -273,7 +273,7 @@ This file defines the a test the same way you would through the Web UI. To run the test, run this command in the terminal: ```bash -tracetest test run -d ./http-test.yaml -w +tracetest run test -f ./http-test.yaml ``` This test will fail just like the sample above due to the `attr:tracetest.span.duration < 50ms` assertion. diff --git a/examples/tracetest-new-relic-otel-demo/README.md b/examples/tracetest-new-relic-otel-demo/README.md index f22ae31ba9..34d8a1bb20 100644 --- a/examples/tracetest-new-relic-otel-demo/README.md +++ b/examples/tracetest-new-relic-otel-demo/README.md @@ -274,7 +274,7 @@ This file defines the a test the same way you would through the Web UI. To run the test, run this command in the terminal: ```bash -tracetest test run -d ./tracetest/e2e/http-test.yaml -w +tracetest run test -f ./tracetest/e2e/http-test.yaml ``` This test will fail just like the sample above due to the `attr:tracetest.span.duration < 50ms` assertion. diff --git a/examples/tracetest-opensearch/README.md b/examples/tracetest-opensearch/README.md index f2e9541db8..3ef41617cf 100644 --- a/examples/tracetest-opensearch/README.md +++ b/examples/tracetest-opensearch/README.md @@ -7,7 +7,7 @@ This repository objective is to show how you can configure your Tracetest instan 1. [Install Tracetest CLI](https://docs.tracetest.io/installing/) 2. Run `tracetest configure --endpoint http://localhost:11633` on a terminal to configure the CLI to send all commands to that address 3. Run the project by using docker-compose: `docker-compose up` (Linux) or `docker compose up` (Mac) -4. Test if it works by running: `tracetest test run -d tests/list-tests.yaml`. This would trigger a test that will send and retrieve spans from the OpenSearch instance that is running on your machine. +4. Test if it works by running: `tracetest run test -f tests/list-tests.yaml`. This would trigger a test that will send and retrieve spans from the OpenSearch instance that is running on your machine. > :warning: Note: The OpenSearch configuration used for this example is not meant to be used in production. diff --git a/examples/tracetest-provisioning-env/README.md b/examples/tracetest-provisioning-env/README.md index 04ea221661..693515c23e 100644 --- a/examples/tracetest-provisioning-env/README.md +++ b/examples/tracetest-provisioning-env/README.md @@ -7,4 +7,4 @@ This repository objective is to show how you can provision a Tracetest instance 1. [Install the Tracetest CLI](https://docs.tracetest.io/installing/) 2. Run `tracetest configure --endpoint http://localhost:11633` on a terminal 3. Run the project by using docker-compose: `docker-compose up` (Linux) or `docker compose up` (Mac) -4. Test if it works by running: `tracetest test run -d tests/list-tests.yaml`. This would trigger a test that will send and retrieve spans from the Jaeger instance that is running on your machine. +4. Test if it works by running: `tracetest run test -f tests/list-tests.yaml`. This would trigger a test that will send and retrieve spans from the Jaeger instance that is running on your machine. diff --git a/examples/tracetest-signalfx/README.md b/examples/tracetest-signalfx/README.md index 7721ab238a..4081f9bfe5 100644 --- a/examples/tracetest-signalfx/README.md +++ b/examples/tracetest-signalfx/README.md @@ -8,4 +8,4 @@ This repository objective is to show how you can configure your tracetest instan 2. Run `tracetest configure --endpoint http://localhost:11633` on a terminal to configure the CLI to send all commands to that address 3. Update the `collector.config.yaml` and `tracetest-config.yaml` with the `token` and `realm` of your SignalFX account. 4. Run the project by using docker-compose: `docker-compose up` (Linux) or `docker compose up` (Mac) -5. Test if it works by running: `tracetest test run -d tests/list-tests.yaml`. This would trigger a test that will send and retrieve spans from the opensearch instance that is running on your machine. +5. Test if it works by running: `tracetest run test -f tests/list-tests.yaml`. This would trigger a test that will send and retrieve spans from the opensearch instance that is running on your machine. diff --git a/examples/tracetest-synthetic-monitoring/.github/workflows/synthetic-monitoring.yaml b/examples/tracetest-synthetic-monitoring/.github/workflows/synthetic-monitoring.yaml index 5c4d4562f5..d000986446 100644 --- a/examples/tracetest-synthetic-monitoring/.github/workflows/synthetic-monitoring.yaml +++ b/examples/tracetest-synthetic-monitoring/.github/workflows/synthetic-monitoring.yaml @@ -43,7 +43,7 @@ jobs: - name: Run syntethic monitoring tests id: monitoring run: | - tracetest test run -f test-api.yaml + tracetest run test -f test-api.yaml - name: Send custom JSON data to Slack workflow if: ${{ failure() }} diff --git a/examples/tracetest-tempo/README.md b/examples/tracetest-tempo/README.md index b0d1202b58..51472e8987 100644 --- a/examples/tracetest-tempo/README.md +++ b/examples/tracetest-tempo/README.md @@ -7,6 +7,6 @@ This repository objective is to show how you can configure your tracetest instan 1. [Install the tracetest CLI](https://github.com/kubeshop/tracetest/blob/main/docs/installing.md#cli-installation) 2. Run `tracetest configure --endpoint http://localhost:11633` on a terminal to configure the CLI to send all commands to that address 3. Run the project by using docker-compose: `docker-compose up` (Linux) or `docker compose up` (Mac) -4. Test if it works by running: `tracetest test run -d tests/list-tests.yaml`. This would trigger a test that will send and retrieve spans from the opensearch instance that is running on your machine. +4. Test if it works by running: `tracetest run test -f tests/list-tests.yaml`. This would trigger a test that will send and retrieve spans from the opensearch instance that is running on your machine. > :warning: Note: The Tempo configuration used for this example is not meant to be used in production. diff --git a/testing/cli-e2etest/README.md b/testing/cli-e2etest/README.md index ae0e4bcd05..f4261c20ec 100644 --- a/testing/cli-e2etest/README.md +++ b/testing/cli-e2etest/README.md @@ -30,12 +30,12 @@ The main idea is to test every CLI command against the Tracetest server with dif | CLI Command | Test scenarios | | ------------------------------------------------------------------ | -------------- | -| `test run -d [test-definition]` | [RunTestWithGrpcTrigger](./testscenarios/test/run_test_with_grpc_trigger_test.go) | -| `test run -d [test-definition] -e [environment-id]` | [RunTestWithHttpTriggerAndEnvironmentFile](./testscenarios/test/run_test_with_http_trigger_and_environment_file_test.go) | -| `test run -d [test-definition] -e [environment-definition]` | [RunTestWithHttpTriggerAndEnvironmentFile](./testscenarios/test/run_test_with_http_trigger_and_environment_file_test.go) | -| `test run -d [transaction-definition]` | [RunTransaction](./testscenarios/transaction//run_transaction_test.go) | -| `test run -d [transaction-definition] -e [environment-id]` | | -| `test run -d [transaction-definition] -e [environment-definition]` | | +| `run test -f [test-definition]` | [RunTestWithGrpcTrigger](./testscenarios/test/run_test_with_grpc_trigger_test.go) | +| `run test -f [test-definition] -e [environment-id]` | [RunTestWithHttpTriggerAndEnvironmentFile](./testscenarios/test/run_test_with_http_trigger_and_environment_file_test.go) | +| `run test -f [test-definition] -e [environment-definition]` | [RunTestWithHttpTriggerAndEnvironmentFile](./testscenarios/test/run_test_with_http_trigger_and_environment_file_test.go) | +| `run transaction -f [transaction-definition]` | [RunTransaction](./testscenarios/transaction//run_transaction_test.go) | +| `run transaction -f [transaction-definition] -e [environment-id]` | | +| `run transaction -f [transaction-definition] -e [environment-definition]` | | ### Resources: Config