Document how system tests work #727
Conversation
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
🤖 GitHub commentsTo re-run your PR in the CI, just comment with:
|
docs/howto/system_testing.md
Outdated
|
|
||
| ## Stack version | ||
|
|
||
| The tests use the [default version](https://github.com/elastic/elastic-package/blob/main/internal/install/stack_version.go#L9) `elastic-package` provides. |
There was a problem hiding this comment.
What do you mean by that? Tests don't start the cluster, so can't have an influence on its version. In the Integrations repository, we select the stack based on Kibana version constraints.
If you're referring to the elastic-package repository, please move this point to the "Continuous Integration" section, as this file contains generic information also valid for Integrations repository.
docs/howto/system_testing.md
Outdated
|
|
||
| **NOTE**: Terraform variables passed as environment variables require to be prefixed with `TF_VAR_`. These variables are not available in test case definitions because they are [not injected](https://github.com/elastic/elastic-package/blob/f5312b6022e3527684e591f99e73992a73baafcf/internal/testrunner/runners/system/servicedeployer/terraform_env.go#L43) in the test environment. | ||
|
|
||
| #### Cloud Provider access |
There was a problem hiding this comment.
Please rename this point as CI support. It isn't a mandatory step for regular users, it's more for maintainers.
docs/howto/system_testing.md
Outdated
| Placeholders used in the `test-<test_name>-config.yml` must be enclosed in `{{` and `}}` delimiters, per Handlebars syntax. | ||
| **NOTE**: Terraform variable environment variable form (prefixed with `TF_VAR_`) is not injected and cannot be used as placeholders (their value would always be empty). |
There was a problem hiding this comment.
nit: Terraform variable environment variable
nit: as placeholder
docs/howto/system_testing.md
Outdated
| ## Continuous Integration | ||
| `elastic-package` runs a set of system tests on some dummy packages to ensure it's functionalities work as expected. This allows to test changes affecting package testing within `elastic-package` before merging and releasing the changes. |
There was a problem hiding this comment.
Maybe link the dummy packages with a directory in Github.
docs/howto/system_testing.md
Outdated
| The tests use the [default version](https://github.com/elastic/elastic-package/blob/main/internal/install/stack_version.go#L9) `elastic-package` provides. | ||
| You can override this value by changing it in your PR if needed. To update the default version always created a dedicated PR. |
docs/howto/system_testing.md
Outdated
| `elastic-package` runs a set of system tests on some [dummy packages](https://github.com/elastic/elastic-package/tree/main/test/packages) to ensure it's functionalities work as expected. This allows to test changes affecting package testing within `elastic-package` before merging and releasing the changes. | ||
| Tests uses a set of environment variables that are set at the beginning of the `Jenkinsfile`. |
docs/howto/system_testing.md
Outdated
| Tests uses a set of environment variables that are set at the beginning of the `Jenkinsfile`. | ||
| The exposed environment variables are passed to the test runners through service deployer specific configurations (refer to the service deployer section for further details). |
There was a problem hiding this comment.
nit: service deployer specific configurations
Still very WIP, I want to document how system tests work and some nuances.