diff --git a/README.md b/README.md index 4ccb20acd5..dae18bbf06 100644 --- a/README.md +++ b/README.md @@ -1,28 +1,154 @@ -

- Tracetest Logo Light - Tracetest Logo Dark -

+ +

+ Tracetest Logo Light + Tracetest Logo Dark +

+
+ +---

- Tracetest - Generate end-to-end tests automatically from your traces. For QA, Dev, & Ops. + Build integration and end-to-end tests in minutes, instead of days, using OpenTelemetry and trace-based testing.

+ +

+ + Get Started!  👉  + +

+
+ + +

+ Docs |  + Tutorials |  + Recipes |  + Examples |  + Discord |  + Blog |  + Website +

+
+ +

+ + Tracetest is released under the MIT License + + + + + + + + + + PRs welcome! + + + Help us reach 1k stars! + + + Join our Community! + + + Talk to us on Discord! + + + + + Tracetest Twitter + +

+

- - - Install |  - Documentation |  - Twitter |  - Discord |  - Blog + Tracetest lets you build integration and end-to-end tests 98% faster with disitrbuted traces.

- - - + No plumbing, no mocks, no fakes. Test against real data.

+You can: + +- **Assert** against both the **response and trace data** at every point of a request transaction. +- **Assert** on the **timing of trace spans**. + - Eg. A database span executes within `100ms`. +- **Wildcard assertions** across common types of activities. + - Eg. All gRPC return codes should be `0`. + - Eg. All database calls should happen in less than `100ms`. +- **Assert** against **side-effects** in your distributed system. + - Eg. Message queues, async API calls, external APIs, etc. +- **Integrate** with your **existing distributed tracing solution**. +- Define multiple test triggers: + - HTTP requests + - gRPC requests + - Trace IDs + - and many more... +- Save and run tests manually and via CI build jobs. +- Verify and analyze the quality of your OpenTelemetry instrumentation to enforce rules and standards. +- Test long-running processes. + +**Build tests in minutes**. + +- **Visually** - in the Web UI +

+ +

+ +- **Programmatically** - in YAML + + ```yaml + type: Test + spec: + id: Yg9sN-94g + name: Pokeshop - Import + description: Import a Pokemon + trigger: + type: http + httpRequest: + url: http://demo-api:8081/pokemon/import + method: POST + headers: + - key: Content-Type + value: application/json + body: '{"id":52}' + specs: + - name: 'All Database Spans: Processing time is less than 100ms' + selector: span[tracetest.span.type="database"] + assertions: + - attr:tracetest.span.duration < 100ms + ``` + +# 🔥 Features + +- Works out of the box with your existing OpenTelemetry instrumentation, supporting [numerous trace data stores](https://docs.tracetest.io/configuration/overview/#supported-trace-data-stores), including: + - Jeager + - Grafana Tempo + - OpenSearch + - Elastic + - And, many more... + - Tell us which other trace data stores you want supported! +- Works out of the box by adding Tracetest as an [additional pipeline](https://docs.tracetest.io/getting-started/supported-backends#using-tracetest-without-a-backend) via your OpenTelemetry Collector config. +- Supporting multiple ways of creating a test, including HTTP, GRPC and Postman Collections. +- Visualize the changes you are making to your trace as you develop, enabling Observability-Driven Development. +- [Add assertions](https://docs.tracetest.io/using-tracetest/adding-assertions) based on response data from the trigger request and all trace data contained in the spans of your distributed trace. +- Specify which spans to check in assertions via the [selector language](https://docs.tracetest.io/concepts/selectors). +- Define checks against the attributes in these spans, including properties, return status, or timing. +- Create tests visually in the Tracetest Web UI or programatically via [YAML-based test definition files](https://docs.tracetest.io/cli/test-definition-file). +- Use test definition files and the Tracetest CLI to [enable Gitops flows and CI/CD automation](https://docs.tracetest.io/ci-cd-automation/overview). +- [Tracetest CLI](https://docs.tracetest.io/cli/cli-installation-reference) allows importing & exporting tests, running tests, and more. +- [Version tests](https://docs.tracetest.io/concepts/versioning/) as the definition of the test is altered. +- The [guided install](https://docs.tracetest.io/getting-started/installation) can include [an example "Pokeshop" microservice](https://docs.tracetest.io/live-examples/pokeshop/overview) that is instrumented with OpenTelemetry to use as an example application under test. +- Create [environment variables](https://docs.tracetest.io/concepts/environments) to assert the same behavior across multiple environments (dev, staging, and production, for example) +- Create [test outputs](https://docs.tracetest.io/web-ui/creating-test-outputs/) by defining a variable based on the information contained in a particular span's attributes. +- Run [ad-hoc tests](https://docs.tracetest.io/concepts/ad-hoc-testing) by using undefined variables to enable supplying variables at runtime. +- Define [test suites/transactions](https://docs.tracetest.io/concepts/transactions) to chain tests together and use variables obtained from a test in a subsequent test. These variables can also be loaded from the environment. +- Run comprehensive [trace analysis and validation](https://docs.tracetest.io/analyzer/concepts) to adhere to OpenTelemetry rules and standards. +- Configure [test runner](https://docs.tracetest.io/configuration/test-runner) behavior with required gates used when executing your tests to determine whether to mark the test as passed or failed. + +# 🚀 Getting Started +

@@ -32,63 +158,54 @@

-# Tracetest +## 1️⃣ Install the Tracetest CLI -Tracetest is a OpenTelemetry based tool that helps you develop and test your distributed applications. It assists you in the development process by enabling you to trigger your code and see the trace as you add OTel instrumentation. It also empowers you to create trace-based tests based on the data contained in your OpenTelemetry trace. You can verify against both the triggering transactions response AND any of the information contained deep in a span in your trace. This can include: +```bash +curl -L https://raw.githubusercontent.com/kubeshop/tracetest/main/install-cli.sh | bash -s +``` -- verify the quality of your OpenTelemetry instrumentation and enforce standards. -- Testing events that occur on 'the other side' of an async message queue, even though the original async call has returned earlier. -- Assertions based on the timing of different steps in your process, such as specifying maximum return time for a SQL query or an external HTTP call. -- Wildcard assertions across common types of activities, ie all gRPC return codes should be 0, all database calls should happen in less than 100ms. -- Testing long running processes instrumented with OpenTelemetry tracing to assert proper operation deep in the process. +> [:gear: Read the CLI installation docs for more options and instructions.](https://docs.tracetest.io/getting-started/installation#install-the-tracetest-cli) -# Features +## 2️⃣ Install the Tracetest Server -- Works out of the box with your existing OTel instrumentation, supporting [numerous backend trace datastores](https://docs.tracetest.io/getting-started/supported-backends), including Jeager and Grafana Tempo. In addition, supports adding Tracetest as an [additional pipeline](https://docs.tracetest.io/getting-started/supported-backends#using-tracetest-without-a-backend) via your OpenTelemetry Collector config. Tell us others backend datastores you want supported! -- Supporting multiple ways of creating a test, including HTTP, GRPC and Postman Collections. -- Visualize the changes you are making to your trace as you develop, enabling Observability-Driven Development. -- [Add assertions](https://docs.tracetest.io/using-tracetest/adding-assertions) based on return data from trigger call and/or data contained in the spans in your distributed trace. -- Specify which spans to check in assertions via the [selector language](https://docs.tracetest.io/concepts/selectors). -- Define checks against the attributes in these spans, including properties, return status, or timing. -- Create tests via graphical UI or via [YAML-based test definition file](https://docs.tracetest.io/cli/test-definition-file). -- Use the test definition file to [enable Gitops flows](https://tracetest.io/blog/integrating-tracetest-with-github-actions-in-a-ci-pipeline). -- [Tracetest CLI](https://docs.tracetest.io/cli/command-line-tool) allows importing & exporting tests, running tests, and more. -- [Version tests](https://docs.tracetest.io/using-tracetest/versioning) as the definition of the test is altered. -- Install can include [an example microservice](https://kubeshop.github.io/tracetest/pokeshop/) that is instrumented with OpenTelemetry to use as an example application under test. +```bash +tracetest server install +``` -# Getting Started +This command will launch an install wizard that automatically installs Tracetest and a [sample Pokeshop Demo app](https://docs.tracetest.io/live-examples/pokeshop/overview) into either Docker or Kubernetes based on your selection. -You can install tracetest by running: +Or, install Tracetest with Helm. The Tracetest Helm charts are located [here](https://github.com/kubeshop/helm-charts/tree/main/charts/tracetest). -```sh -curl -L https://raw.githubusercontent.com/kubeshop/tracetest/main/install-cli.sh | bash -s -tracetest server install +```bash +helm install tracetest kubeshop/tracetest --namespace=tracetest --create-namespace ``` -> :gear: To customize your Tracetest installation. Go to our [installation guide](https://docs.tracetest.io/getting-started/installation) for more information. +> [:gear: Read the Server installation docs for more options and instructions.](https://docs.tracetest.io/getting-started/installation#install-the-tracetest-server) + +## 3️⃣ Open Tracetest -Installation only takes a few minutes and is done with via a Helm command. After installing, take a look at the -[Accessing the Dashboard](https://docs.tracetest.io/getting-started/accessing-dashboard) guide to access the Tracetest Dashboard and -create and run your first test. +Once you've installed Tracetest Server, access the Tracetest Web UI on `http://localhost:11633`. -# How does Tracetest work? +Check out the [Opening Tracetest guide](https://docs.tracetest.io/getting-started/open) to start creating and running tests! + +# 🤔 How does Tracetest work? 1. Pick an endpoint to test. 2. Run a test, and get the trace. 3. The trace is the blueprint of your system under test. It shows all the steps the system has taken to execute the request. -4. Use this blueprint to define assertions through Tracetest UI. +4. Use this blueprint to define assertions in the Tracetest Web UI. 5. Add assertions on different services, checking return statuses, data, or even execution times of a system. 6. Run the tests. Once the test is built, it can be run automatically as part of a build process. Every test has a trace attached, allowing you to immediately see what worked, and what did not, reducing the need to reproduce the problem to see the underlying issue. -# What does the test definition file look like? +# 📂 What does the test definition file look like? The Tracetest [test definition files](https://docs.tracetest.io/cli/test-definition-file) are written in a simple YAML format. You can write them directly or build them graphically via the UI. Here is an example of a test which: -- executes POST against the pokemon/import endpoint. -- verifies that the HTTP blocks return a 200 status code. -- verifies all database calls execute in less than 50ms. +- Executes `POST` against the `pokemon/import` endpoint. +- Verifies that the HTTP blocks return a `200` status code. +- Verifies all database calls execute in less than `50ms`. ```yaml type: Test @@ -125,26 +242,36 @@ spec: - tracetest.span.duration < "50ms" ``` -# Feedback +# 🤖 How to run an automated test? -Have an idea to improve Tracetest? Please [create an issue here](https://github.com/kubeshop/tracetest/issues/new/choose) or join our community on [Discord](https://discord.gg/eBvEQRVyKX). +Save a test definition file above as `pokeshop_import.yaml`. Use the CLI to run a test. -Follow us on [Twitter at @tracetest_io](https://twitter.com/tracetest_io) for updates. +```bash +tracetest run test --file /path/to/pokeshop_import.yaml +``` -Give us a star on Github if you're interested in the project! +Check out the [CI/CD docs](https://docs.tracetest.io/ci-cd-automation/overview) to learn more about test automation. -# Documentation +# Tests -Is available at [https://docs.tracetest.io/](https://docs.tracetest.io/) +We strive to produce quality code and improve Tracetest rapidly and safely. Therefore, we have a full suite of both front-end and back-end tests. Cypress tests are running against the front-end code and (surprise, surprise) Tracetest against the back-end code. You can see the [test runs here](https://github.com/kubeshop/tracetest/actions/workflows/pull-request.yaml), and a blog post describing our [testing pipelines here](https://tracetest.io/blog/50-faster-ci-pipelines-with-github-actions). -# Tests +# 🎤 Feedback + +Have an idea to improve Tracetest? + +You can: -We strive to produce quality code and improve Tracetest rapidly and safely. Therefore, we have a full suite of both frontend and backend tests. We are using Cypress to test our frontend code and (surprise, surprise) Tracetest for our backend code. You can see the [test runs here](https://github.com/kubeshop/tracetest/actions/workflows/pull-request.yaml), and a blog post describing our [testing pipelines here](https://tracetest.io/blog/integrating-tracetest-with-github-actions-in-a-ci-pipeline). +- [Create an issue here](https://github.com/kubeshop/tracetest/issues/new/choose)! +- Join our [Discord](https://discord.gg/eBvEQRVyKX), and ask us any questions there. +- Follow us on [Twitter at @tracetest_io](https://twitter.com/tracetest_io) for updates. +- Give us a ⭐️ on Github if you like what we're doing! -# Contributing +# 🌱 Contributing & Community -Want to contribute with Tracetest? See our contributing guidelines [here](./CONTRIBUTING.md). +Whether it's big or small, we love contributions. -# License +Not sure where to get started? You can: -[MIT License](/LICENSE) +- Visit our [Community Page](https://tracetest.io/community). +- See our contributing guidelines [here](./CONTRIBUTING.md). diff --git a/docs/docs/ci-cd-automation/overview.md b/docs/docs/ci-cd-automation/overview.md index 8b3a6e7a9e..007750ac70 100644 --- a/docs/docs/ci-cd-automation/overview.md +++ b/docs/docs/ci-cd-automation/overview.md @@ -4,9 +4,9 @@ This section contains a general overview of running Tracetest in CI/CD pipelines You can find guides for: -- [GitHub Actions](./github-actions-pipeline) -- [Testkube](./testkube-pipeline) -- [Tekton](./tekton-pipeline) +- [GitHub Actions](./github-actions-pipeline.md) +- [Testkube](./testkube-pipeline.md) +- [Tekton](./tekton-pipeline.md) :::note If you want to see more examples with other CI/CD tools, let us know by [opening an issue in GitHub](https://github.com/kubeshop/tracetest/issues/new/choose)! @@ -21,7 +21,7 @@ To read more about integrating Tracetest with CI/CD tools, check out tutorials i ## Running Tracetest CLI from Docker -Many integrations with CI/CD tools can be accomplished by running the [Tracetest CLI](../cli/configuring-your-cli) to execute a test against a remote Tracetest server. If you do not want to install the Tracetest CLI in your environment, you can choose to directly execute it from a Docker image. +Many integrations with CI/CD tools can be accomplished by running the [Tracetest CLI](../cli/configuring-your-cli.md) to execute a test against a remote Tracetest server. If you do not want to install the Tracetest CLI in your environment, you can choose to directly execute it from a Docker image. **How to Use**: diff --git a/docs/docs/configuration/overview.md b/docs/docs/configuration/overview.md index 6eaaf77d96..6857199ae6 100644 --- a/docs/docs/configuration/overview.md +++ b/docs/docs/configuration/overview.md @@ -27,7 +27,7 @@ Currently, Tracetest supports the following data stores. Click on the respective Another option is to send traces directly to Tracetest using the OpenTelemetry Collector. And, you don't have to change your existing pipelines to do so. -View [configuration for OpenTelemetry Collector](./connecting-to-data-stores/opentelemetry-collector) for more details. +View [configuration for OpenTelemetry Collector](./connecting-to-data-stores/opentelemetry-collector.md) for more details. ## Trace Data Store Configuration Examples