Skip to content

Commit

Permalink
readme: add initial overhaul (#2977)
Browse files Browse the repository at this point in the history 
* readme: add initial overhaul

* readme: second improvement

* docs(typos): fix links and typos

* readme: fix formatting

* readme: formatting
  • Loading branch information
@adnanrahic
adnanrahic committed Jul 25, 2023
1 parent 6fc605c commit a6ba0b4
Show file tree
Hide file tree
Showing 3 changed files with 193 additions and 66 deletions.
249 changes: 188 additions & 61 deletions README.md
View file
@@ -1,28 +1,154 @@
<p align="center">
<img style="width:66%" src="assets/tracetest-logo-color-w-white-text.svg#gh-dark-mode-only" alt="Tracetest Logo Light"/>
<img style="width:66%" src="assets/tracetest-logo-color-w-black-text.svg#gh-light-mode-only" alt="Tracetest Logo Dark" />
</p>
<a href="https://tracetest.io">
<p align="center">
<img style="width:66%" src="assets/tracetest-logo-color-w-white-text.svg#gh-dark-mode-only" alt="Tracetest Logo Light"/>
<img style="width:66%" src="assets/tracetest-logo-color-w-black-text.svg#gh-light-mode-only" alt="Tracetest Logo Dark" />
</p>
</a>

---

<p align="center">
Tracetest - Generate end-to-end tests automatically from your traces. For QA, Dev, & Ops.
Build integration and end-to-end tests in minutes, instead of days, using OpenTelemetry and trace-based testing.
</p>

<b>
<p align="center">
<a href="https://docs.tracetest.io/getting-started/installation">
Get Started! &nbsp;馃憠&nbsp;
</a>
</p>
</b>

<b>
<p align="center">
<a href="https://docs.tracetest.io/">Docs</a>&nbsp;|&nbsp;
<a href="https://docs.tracetest.io/examples-tutorials/overview">Tutorials</a>&nbsp;|&nbsp;
<a href="https://docs.tracetest.io/examples-tutorials/recipes">Recipes</a>&nbsp;|&nbsp;
<a href="https://github.com/kubeshop/tracetest/tree/main/examples">Examples</a>&nbsp;|&nbsp;
<a href="https://discord.gg/eBvEQRVyKX">Discord</a>&nbsp;|&nbsp;
<a href="https://tracetest.io/blog">Blog</a>&nbsp;|&nbsp;
<a href="https://tracetest.io">Website</a>
</p>
</b>

<h4 align="center">
<a href="https://github.com/kubeshop/tracetest/blob/main/LICENSE">
<img src="https://img.shields.io/badge/Licence-MIT-blue" alt="Tracetest is released under the MIT License">
</a>
<a href="https://kubeshop.io/"><img src="https://img.shields.io/website?url=https://kubeshop.io/&up_message=Kubeshop&up_color=%232635F1&label=Accelerator&down_color=%232635F1&down_message=Kubeshop"></a>
<a href="https://github.com/kubeshop/tracetest/releases">
<img title="Release" src="https://img.shields.io/github/v/release/kubeshop/tracetest"/>
</a>
<a href="https://github.com/kubeshop/tracetest/releases">
<img title="Release date" src="https://img.shields.io/github/release-date/kubeshop/tracetest"/>
</a>
<a href="https://github.com/kubeshop/tracetest/blob/main/CONTRIBUTING.md">
<img src="https://img.shields.io/badge/PRs-Welcome-brightgreen?logo=github" alt="PRs welcome!" />
</a>
<a href="https://github.com/kubeshop/tracetest/issues">
<img src="https://img.shields.io/github/stars/kubeshop/tracetest?color=%23EAC54F&logo=github&label=Help us reach 1k stars! Now at:" alt="Help us reach 1k stars!" />
</a>
<a href="https://tracetest.io/community">
<img src="https://img.shields.io/badge/Join-Community!-purple" alt="Join our Community!" />
</a>
<a href="https://discord.gg/8MtcMrQNbX">
<img src="https://img.shields.io/badge/Chat-on Discord!-red" alt="Talk to us on Discord!" />
</a>
<a href="https://hub.docker.com/r/kubeshop/tracetest"><img title="Docker Pulls" src="https://img.shields.io/docker/pulls/kubeshop/tracetest?logo=docker"/></a>
<a href="https://hub.docker.com/r/kubeshop/tracetest/tags?page=1&name=latest"><img title="Docker Image Size" src="https://img.shields.io/docker/image-size/kubeshop/tracetest/latest?logo=docker&label='kubeshop/tracetest:latest' image size"/></a>
<a href="https://twitter.com/tracetest_io">
<img src="https://img.shields.io/badge/follow-%40tracetest__io-1DA1F2?logo=twitter&style=social" alt="Tracetest Twitter" />
</a>
</h4>

<p align="center">
<!--<a href="https://tracetest.io">Website</a>&nbsp;|&nbsp; -->
<!--<a href="https://github.com/kubeshop/tracetest#try-the-demo--give-us-feedback">Live Demo</a>&nbsp;|&nbsp;-->
<a href="https://docs.tracetest.io/getting-started/installation">Install</a>&nbsp;|&nbsp;
<a href="https://docs.tracetest.io/">Documentation</a>&nbsp;|&nbsp;
<a href="https://twitter.com/tracetest_io">Twitter</a>&nbsp;|&nbsp;
<a href="https://discord.gg/eBvEQRVyKX">Discord</a>&nbsp;|&nbsp;
<a href="https://tracetest.io/blog">Blog</a>
Tracetest lets you build integration and end-to-end tests 98% faster with disitrbuted traces.
</p>

<p align="center">
<a href="https://github.com/kubeshop/tracetest/releases"><img title="Release" src="https://img.shields.io/github/v/release/kubeshop/tracetest"/></a>
<a href=""><img title="Docker builds" src="https://img.shields.io/docker/automated/kubeshop/tracetest"/></a>
<a href="https://github.com/kubeshop/tracetest/releases"><img title="Release date" src="https://img.shields.io/github/release-date/kubeshop/tracetest"/></a>
No plumbing, no mocks, no fakes. Test against real data.
</p>

You can:

- **Assert** against both the **response and trace data** at every point of a request transaction.
- **Assert** on the **timing of trace spans**.
- Eg. A database span executes within `100ms`.
- **Wildcard assertions** across common types of activities.
- Eg. All gRPC return codes should be `0`.
- Eg. All database calls should happen in less than `100ms`.
- **Assert** against **side-effects** in your distributed system.
- Eg. Message queues, async API calls, external APIs, etc.
- **Integrate** with your **existing distributed tracing solution**.
- Define multiple test triggers:
- HTTP requests
- gRPC requests
- Trace IDs
- and many more...
- Save and run tests manually and via CI build jobs.
- Verify and analyze the quality of your OpenTelemetry instrumentation to enforce rules and standards.
- Test long-running processes.

**Build tests in minutes**.

- **Visually** - in the Web UI
<p align="center">
<img src="https://res.cloudinary.com/djwdcmwdz/image/upload/v1688476657/docs/screely-1688476653521_omxe4r.png" style="width:66%;height:auto">
</p>

- **Programmatically** - in YAML

```yaml
type: Test
spec:
id: Yg9sN-94g
name: Pokeshop - Import
description: Import a Pokemon
trigger:
type: http
httpRequest:
url: http://demo-api:8081/pokemon/import
method: POST
headers:
- key: Content-Type
value: application/json
body: '{"id":52}'
specs:
- name: 'All Database Spans: Processing time is less than 100ms'
selector: span[tracetest.span.type="database"]
assertions:
- attr:tracetest.span.duration < 100ms
```

# 馃敟 Features

- Works out of the box with your existing OpenTelemetry instrumentation, supporting [numerous trace data stores](https://docs.tracetest.io/configuration/overview/#supported-trace-data-stores), including:
- Jeager
- Grafana Tempo
- OpenSearch
- Elastic
- And, many more...
- Tell us which other trace data stores you want supported!
- Works out of the box by adding Tracetest as an [additional pipeline](https://docs.tracetest.io/getting-started/supported-backends#using-tracetest-without-a-backend) via your OpenTelemetry Collector config.
- Supporting multiple ways of creating a test, including HTTP, GRPC and Postman Collections.
- Visualize the changes you are making to your trace as you develop, enabling Observability-Driven Development.
- [Add assertions](https://docs.tracetest.io/using-tracetest/adding-assertions) based on response data from the trigger request and all trace data contained in the spans of your distributed trace.
- Specify which spans to check in assertions via the [selector language](https://docs.tracetest.io/concepts/selectors).
- Define checks against the attributes in these spans, including properties, return status, or timing.
- Create tests visually in the Tracetest Web UI or programatically via [YAML-based test definition files](https://docs.tracetest.io/cli/test-definition-file).
- Use test definition files and the Tracetest CLI to [enable Gitops flows and CI/CD automation](https://docs.tracetest.io/ci-cd-automation/overview).
- [Tracetest CLI](https://docs.tracetest.io/cli/cli-installation-reference) allows importing & exporting tests, running tests, and more.
- [Version tests](https://docs.tracetest.io/concepts/versioning/) as the definition of the test is altered.
- The [guided install](https://docs.tracetest.io/getting-started/installation) can include [an example "Pokeshop" microservice](https://docs.tracetest.io/live-examples/pokeshop/overview) that is instrumented with OpenTelemetry to use as an example application under test.
- Create [environment variables](https://docs.tracetest.io/concepts/environments) to assert the same behavior across multiple environments (dev, staging, and production, for example)
- Create [test outputs](https://docs.tracetest.io/web-ui/creating-test-outputs/) by defining a variable based on the information contained in a particular span's attributes.
- Run [ad-hoc tests](https://docs.tracetest.io/concepts/ad-hoc-testing) by using undefined variables to enable supplying variables at runtime.
- Define [test suites/transactions](https://docs.tracetest.io/concepts/transactions) to chain tests together and use variables obtained from a test in a subsequent test. These variables can also be loaded from the environment.
- Run comprehensive [trace analysis and validation](https://docs.tracetest.io/analyzer/concepts) to adhere to OpenTelemetry rules and standards.
- Configure [test runner](https://docs.tracetest.io/configuration/test-runner) behavior with required gates used when executing your tests to determine whether to mark the test as passed or failed.

# 馃殌 Getting Started

<p align="center">
<a target="_new" href="https://kubeshop.wistia.com/medias/dw06408oqz">
<img src="/assets/introvideo.png" style="width:66%;height:auto">
Expand All @@ -32,63 +158,54 @@
</a>
</p>

# Tracetest
## 1锔忊儯 Install the Tracetest CLI

Tracetest is a OpenTelemetry based tool that helps you develop and test your distributed applications. It assists you in the development process by enabling you to trigger your code and see the trace as you add OTel instrumentation. It also empowers you to create trace-based tests based on the data contained in your OpenTelemetry trace. You can verify against both the triggering transactions response AND any of the information contained deep in a span in your trace. This can include:
```bash
curl -L https://raw.githubusercontent.com/kubeshop/tracetest/main/install-cli.sh | bash -s
```

- verify the quality of your OpenTelemetry instrumentation and enforce standards.
- Testing events that occur on 'the other side' of an async message queue, even though the original async call has returned earlier.
- Assertions based on the timing of different steps in your process, such as specifying maximum return time for a SQL query or an external HTTP call.
- Wildcard assertions across common types of activities, ie all gRPC return codes should be 0, all database calls should happen in less than 100ms.
- Testing long running processes instrumented with OpenTelemetry tracing to assert proper operation deep in the process.
> [:gear: Read the CLI installation docs for more options and instructions.](https://docs.tracetest.io/getting-started/installation#install-the-tracetest-cli)
# Features
## 2锔忊儯 Install the Tracetest Server

- Works out of the box with your existing OTel instrumentation, supporting [numerous backend trace datastores](https://docs.tracetest.io/getting-started/supported-backends), including Jeager and Grafana Tempo. In addition, supports adding Tracetest as an [additional pipeline](https://docs.tracetest.io/getting-started/supported-backends#using-tracetest-without-a-backend) via your OpenTelemetry Collector config. Tell us others backend datastores you want supported!
- Supporting multiple ways of creating a test, including HTTP, GRPC and Postman Collections.
- Visualize the changes you are making to your trace as you develop, enabling Observability-Driven Development.
- [Add assertions](https://docs.tracetest.io/using-tracetest/adding-assertions) based on return data from trigger call and/or data contained in the spans in your distributed trace.
- Specify which spans to check in assertions via the [selector language](https://docs.tracetest.io/concepts/selectors).
- Define checks against the attributes in these spans, including properties, return status, or timing.
- Create tests via graphical UI or via [YAML-based test definition file](https://docs.tracetest.io/cli/test-definition-file).
- Use the test definition file to [enable Gitops flows](https://tracetest.io/blog/integrating-tracetest-with-github-actions-in-a-ci-pipeline).
- [Tracetest CLI](https://docs.tracetest.io/cli/command-line-tool) allows importing & exporting tests, running tests, and more.
- [Version tests](https://docs.tracetest.io/using-tracetest/versioning) as the definition of the test is altered.
- Install can include [an example microservice](https://kubeshop.github.io/tracetest/pokeshop/) that is instrumented with OpenTelemetry to use as an example application under test.
```bash
tracetest server install
```

# Getting Started
This command will launch an install wizard that automatically installs Tracetest and a [sample Pokeshop Demo app](https://docs.tracetest.io/live-examples/pokeshop/overview) into either Docker or Kubernetes based on your selection.

You can install tracetest by running:
Or, install Tracetest with Helm. The Tracetest Helm charts are located [here](https://github.com/kubeshop/helm-charts/tree/main/charts/tracetest).

```sh
curl -L https://raw.githubusercontent.com/kubeshop/tracetest/main/install-cli.sh | bash -s
tracetest server install
```bash
helm install tracetest kubeshop/tracetest --namespace=tracetest --create-namespace
```

> :gear: To customize your Tracetest installation. Go to our [installation guide](https://docs.tracetest.io/getting-started/installation) for more information.
> [:gear: Read the Server installation docs for more options and instructions.](https://docs.tracetest.io/getting-started/installation#install-the-tracetest-server)
## 3锔忊儯 Open Tracetest

Installation only takes a few minutes and is done with via a Helm command. After installing, take a look at the
[Accessing the Dashboard](https://docs.tracetest.io/getting-started/accessing-dashboard) guide to access the Tracetest Dashboard and
create and run your first test.
Once you've installed Tracetest Server, access the Tracetest Web UI on `http://localhost:11633`.

# How does Tracetest work?
Check out the [Opening Tracetest guide](https://docs.tracetest.io/getting-started/open) to start creating and running tests!

# 馃 How does Tracetest work?

1. Pick an endpoint to test.
2. Run a test, and get the trace.
3. The trace is the blueprint of your system under test. It shows all the steps the system has taken to execute the request.
4. Use this blueprint to define assertions through Tracetest UI.
4. Use this blueprint to define assertions in the Tracetest Web UI.
5. Add assertions on different services, checking return statuses, data, or even execution times of a system.
6. Run the tests.

Once the test is built, it can be run automatically as part of a build process. Every test has a trace attached, allowing you to immediately see what worked, and what did not, reducing the need to reproduce the problem to see the underlying issue.

# What does the test definition file look like?
# 馃搨 What does the test definition file look like?

The Tracetest [test definition files](https://docs.tracetest.io/cli/test-definition-file) are written in a simple YAML format. You can write them directly or build them graphically via the UI. Here is an example of a test which:

- executes POST against the pokemon/import endpoint.
- verifies that the HTTP blocks return a 200 status code.
- verifies all database calls execute in less than 50ms.
- Executes `POST` against the `pokemon/import` endpoint.
- Verifies that the HTTP blocks return a `200` status code.
- Verifies all database calls execute in less than `50ms`.

```yaml
type: Test
Expand Down Expand Up @@ -125,26 +242,36 @@ spec:
- tracetest.span.duration < "50ms"
```

# Feedback
# 馃 How to run an automated test?

Have an idea to improve Tracetest? Please [create an issue here](https://github.com/kubeshop/tracetest/issues/new/choose) or join our community on [Discord](https://discord.gg/eBvEQRVyKX).
Save a test definition file above as `pokeshop_import.yaml`. Use the CLI to run a test.

Follow us on [Twitter at @tracetest_io](https://twitter.com/tracetest_io) for updates.
```bash
tracetest run test --file /path/to/pokeshop_import.yaml
```

Give us a star on Github if you're interested in the project!
Check out the [CI/CD docs](https://docs.tracetest.io/ci-cd-automation/overview) to learn more about test automation.

# Documentation
# Tests

Is available at [https://docs.tracetest.io/](https://docs.tracetest.io/)
We strive to produce quality code and improve Tracetest rapidly and safely. Therefore, we have a full suite of both front-end and back-end tests. Cypress tests are running against the front-end code and (surprise, surprise) Tracetest against the back-end code. You can see the [test runs here](https://github.com/kubeshop/tracetest/actions/workflows/pull-request.yaml), and a blog post describing our [testing pipelines here](https://tracetest.io/blog/50-faster-ci-pipelines-with-github-actions).

# Tests
# 馃帳 Feedback

Have an idea to improve Tracetest?

You can:

We strive to produce quality code and improve Tracetest rapidly and safely. Therefore, we have a full suite of both frontend and backend tests. We are using Cypress to test our frontend code and (surprise, surprise) Tracetest for our backend code. You can see the [test runs here](https://github.com/kubeshop/tracetest/actions/workflows/pull-request.yaml), and a blog post describing our [testing pipelines here](https://tracetest.io/blog/integrating-tracetest-with-github-actions-in-a-ci-pipeline).
- [Create an issue here](https://github.com/kubeshop/tracetest/issues/new/choose)!
- Join our [Discord](https://discord.gg/eBvEQRVyKX), and ask us any questions there.
- Follow us on [Twitter at @tracetest_io](https://twitter.com/tracetest_io) for updates.
- Give us a 猸愶笍 on Github if you like what we're doing!

# Contributing
# 馃尡 Contributing & Community

Want to contribute with Tracetest? See our contributing guidelines [here](./CONTRIBUTING.md).
Whether it's big or small, we love contributions.

# License
Not sure where to get started? You can:

[MIT License](/LICENSE)
- Visit our [Community Page](https://tracetest.io/community).
- See our contributing guidelines [here](./CONTRIBUTING.md).
8 changes: 4 additions & 4 deletions docs/docs/ci-cd-automation/overview.md
View file
Expand Up @@ -4,9 +4,9 @@ This section contains a general overview of running Tracetest in CI/CD pipelines

You can find guides for:

- [GitHub Actions](./github-actions-pipeline)
- [Testkube](./testkube-pipeline)
- [Tekton](./tekton-pipeline)
- [GitHub Actions](./github-actions-pipeline.md)
- [Testkube](./testkube-pipeline.md)
- [Tekton](./tekton-pipeline.md)

:::note
If you want to see more examples with other CI/CD tools, let us know by [opening an issue in GitHub](https://github.com/kubeshop/tracetest/issues/new/choose)!
Expand All @@ -21,7 +21,7 @@ To read more about integrating Tracetest with CI/CD tools, check out tutorials i

## Running Tracetest CLI from Docker

Many integrations with CI/CD tools can be accomplished by running the [Tracetest CLI](../cli/configuring-your-cli) to execute a test against a remote Tracetest server. If you do not want to install the Tracetest CLI in your environment, you can choose to directly execute it from a Docker image.
Many integrations with CI/CD tools can be accomplished by running the [Tracetest CLI](../cli/configuring-your-cli.md) to execute a test against a remote Tracetest server. If you do not want to install the Tracetest CLI in your environment, you can choose to directly execute it from a Docker image.

**How to Use**:

Expand Down

0 comments on commit a6ba0b4

Please sign in to comment.