Skip to content

FedeAlonso/e2e-test-suite

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

E2E Test Suite

TestNG based Java test suite focused on e2e testing application services running in the cloud.

Requirements

  • java jdk 11
  • maven >= 3.8.1
  • kcat (only to run Kcat tests)
  • python 3

Build and check checkstyle

mvn install -DskipTests

Executing tests

Running full test suite

mvn test

Running single test

mvn verify -Dit.test=ClassName

Running subset tests defined by maven profile

mvn verify -Psmoke

Writing tests

  • Test method must be annotated with annotation @Test.
  • Every new test class must extend TestBase
  • Environment variables are stored in static Environment class
  • Every new and variable used in testsuite must be added to table or env variables
  • The test class must be included in one of the TestNG suites in suites/

Environment Variables

This environment variables are used to configure tests behaviour, the ENVs marked with the double asterisk (**) are required to run all tests. But not all tests require all ENVs, if you want to run a single test you can check ahead the required variables in the test sourcecode javadoc.

Environment variables can also be configured in the config.json file.

Name Description Default value
CONFIG_FILE relative or absolute path to the config.json $(pwd)/config.json
LOG_DIR path where the test suite will store additional collected logs and artifacts $(pwd)/target/logs
PRIMARY_USERNAME ** https://sso.redhat.com username with quota to provision the tested service (See Create test users if you don't have one)
PRIMARY_PASSWORD ** primary user password
<USER>_OFFLINE_TOKEN ** offline token used to authenticate and authorize user to perform operations against data, control plane. Example user: PRIMARY_OFFLINE_TOKEN
SECONDARY_USERNAME ** https://sso.redhat.com username in the same organization as the primary user
SECONDARY_PASSWORD ** secondary user password
ALIEN_USERNAME ** https://sso.redhat.com username in a different organization respect the primary user with no quota
ALIEN_PASSWORD ** alien user password
DIFF_ORG_USERNAME ** https://sso.redhat.com username in a different organization respect the primary user and alien user with quota: 1 SKU
DIFF_ORG_PASSWORD ** diff_org user password
OPENSHIFT_API_URI the base URI for the application services mgmt APIs (See Test Environments) https://api.stage.openshift.com
REDHAT_SSO_URI users authentication endpoint for application services mgmt APIs https://sso.redhat.com
REDHAT_SSO_LOGIN_FORM_ID HTML id value of the login <form> the SSO application will present after redirect #rh-password-verification-form
OBSERVATORIUM_CLIENT_ID client id for Observatorium client (with token refreshing container running in locally)
OBSERVATORIUM_CLIENT_SECRET client secret for Obserevatorium client
OBSERVATORIUM_OIDC_ISSUER_URL sso issue url for observatorium client
OBSERVATORIUM_URL Observatorium's URL
OPENSHIFT_IDENTITY_URI users authentication endpoint for application services instances APIs (See Test Environments) https://identity.api.stage.openshift.com
OPENSHIFT_IDENTITY_LOGIN_FORM_ID HTML id value of the login <form> the SSO application will present after redirect #rh-password-verification-form
DEV_CLUSTER_SERVER ** the API server URI of a OpenShift cluster with the binding operator installed
DEV_CLUSTER_TOKEN ** the cluster user or service account token
DEV_CLUSTER_NAMESPACE the namespace where to create test resources (See Create test namespace on the dev cluster) mk-e2e-tests
CLI_VERSION the CLI version to download from the app-services-cli repo latest
CLI_PLATFORM windows/macOS/linux auto-detect
CLI_ARCH the CLI arch and os to download from the app-services-cli repo amd64
CLI_EXCLUDE_VERSIONS a regex that if match will exclude the the CLI versions matching it while looking for the latest CLI release alpha
LAUNCH_KEY A string key used to identify the current configuration and owner which is used to generate unique name and identify the launch change-me
SKIP_TEARDOWN Skip the whole test teardown in most tests, although some of them will need top re-enable it to succeed false
SKIP_KAFKA_TEARDOWN Skip only the Kafka instance cleanup teardown in the tests that don't require a new instance for each run to speed the local development false
DEFAULT_KAFKA_REGION Change the default region where kafka instances will be provisioned if the test suite doesn't decide otherwise us-east-1
KAFKA_INSECURE_TLS Boolean value to indicate whether the Kafka and Admin REST API TLS is insecure (for self-signed certificates) false
KAFKA_INSTANCE_API_TEMPLATE URL template for the Kafka Admin REST API. May be used to specify plain-text HTTP or an alternate port https://admin-server-%s/rest
AWS_DATA_PLANE_ACCESS_TOKEN Service account's token with permission t access and manipulate all necessary resources located in AWS data plane cluster
STRATOSPHERE_PASSWORD Password used for all of stratosphere users
STRATOSPHERE_SCENARIO_1_AWS_ACCOUNT_ID aws cloud account' id linked to the org where stratospehere1 user resides
STRATOSPHERE_SCENARIO_1_USER stratosphere1: user in organization with 1 aws cloud account linked and without standard quota
STRATOSPHERE_SCENARIO_2_AWS_ACCOUNT_ID aws cloud account' id linked to the org where stratospehere2 user resides
STRATOSPHERE_SCENARIO_2_USER stratosphere2: user in organization with 1 aws cloud account linked and with standard quota
STRATOSPHERE_SCENARIO_3_AWS_ACCOUNT_ID aws cloud account' id linked to the org where stratospehere3 user resides
STRATOSPHERE_SCENARIO_3_RHM_ACCOUNT_ID rhm cloud account' id linked to the org where stratospehere3 user resides
STRATOSPHERE_SCENARIO_3_USER stratosphere3: user in organization with 1 aws cloud account linked and 1 rhm cloud account linked
STRATOSPHERE_SCENARIO_4_AWS_ACCOUNT_ID aws cloud account' id linked to the org where stratospehere4 user resides
STRATOSPHERE_SCENARIO_3_USER stratosphere4: user in organization with 2 aws cloud account linked

Config File

All Environment variables can also be configured in a JSON file, by default they are loaded from the config.json file located in the project root, but an alternative JSON file can be configured used the the CONFIG_FILE env.

Config file example:

{
  "PRIMARY_USERNAME": "my-primary-user",
  "PRIMARY_PASSWORD": "password",
  "SECONDARY_USERNAME": "my-secondary-user",
  "SECONDARY_PASSWORD": "password",
  "ALIEN_USERNAME": "my-alien-user",
  "ALIEN_PASSWORD": "password",
  "DEV_CLUSTER_SERVER": "https://api.my.cluster:6443",
  "DEV_CLUSTER_TOKEN": "token",
  "DEV_CLUSTER_NAMESPACE": "a-custom-namespace-if-local",
  "LAUNCH_KEY": "my-username-if-local"
}

You can create multiple local *.config.json file that will be ignored by git (ex: prod.config.json) and you can easily switch between them in IntelliJ using the EnvFile plugin and setting the CONFIG_FILE env in the .env file in the project root.

Test Environments

The default targeted environment is the application services stage env.

Stage

Env Value
OPENSHIFT_API_URI https://api.stage.openshift.com
OPENSHIFT_IDENTITY_URI https://identity.api.stage.openshift.com

Production

Env Value
OPENSHIFT_API_URI https://api.openshift.com
OPENSHIFT_IDENTITY_URI https://identity.api.openshift.com

Profiles

Name Description
default run kafka, registry, devexp and quickstarts test suites
integration run test which are part of integration pipeline
kafka run test related to testing managed kafka as such
pr-check run test which are run to check each PR to this repository
registry run test related to testing service registries
sandbox run the sandbox test suite to test the openshift sandbox cluster
smoke run smoke tests, literally making sure basic functions work
quickstarts run the cucumber quickstarts test suite

Short guides

Create test users

The test users needs to be manually created before running the test suites, but not all users are required for all tests.

Organization

  1. Go to https://cloud.redhat.com/ and click on Create an account
  2. You can choose Personal or Corporate
  3. Fill all the required and optional data fields and create the account

This will automatically create a new Organization where the user is the organization owner.

Primary user

  1. Go to https://www.redhat.com/wapps/ugc/protected/usermgt/userList.html and login with the organization owner.
  2. Click on Add New User and fill all the required and optional data fields for the primary user

By default, the account will be able to create evaluation instances which will be enough to run single tests

Secondary user

  1. Go to https://www.redhat.com/wapps/ugc/protected/usermgt/userList.html and login with the organization owner.
  2. Click on Add New User and fill all the required and optional data fields for the secondary user

The secondary user is exactly like the primary user, but by default it wouldn't have access to instances created by the primary user

Alien user

  1. Go to https://cloud.redhat.com/ and click on Create an account
  2. You can choose Personal or Corporate
  3. Fill all the required and optional data fields for the alien user and create the account

This will create the Alien user in a new organization

Diff org user

  1. Go to https://cloud.redhat.com/ and click on Create an account
  2. You can choose Personal or Corporate
  3. Fill all the required and optional data fields for the alien user and create the account
  4. Get the org id for the user created
     ocm get /api/accounts_mgmt/v1/organizations/$(ocm account orgs --columns ID | tail -n1) | jq -r .external_id
     ```
    
  5. Create a MR on ocm-resources repo to add quota for that org example MR

This will create the Diff org user in a new organization and add quota to create standard Kafka Instances

Create test namespace on the dev cluster

  1. Login to the dev cluster as kubeadmin
    oc login --token=KUBE_ADMIN_TOKEN --server=DEV_CLUSTER_SERVER
    
  2. Execute the ./hack/bootstrap-mk-e2e-tests-namespace.sh script
    ./hack/bootstrap-mk-e2e-tests-namespace.sh
    
  3. Update the staging Vault with the new dev cluster token

Recreate the service account for the sandbox cluster

  1. Login to the sandbox cluster with the sandbox user
    oc login --token=KUBE_ADMIN_TOKEN --server=DEV_CLUSTER_SERVER
    
  2. Execute the ./hack/bootstrap-mk-e2e-tests-namespace.sh script
    SANDBOX=true NAMESPACE=THE_SANDBOX_NAMESPACE ./hack/bootstrap-mk-e2e-tests-namespace.sh
    
  3. Update the sandbox Vault with the new dev cluster token

Update the rhoas-model dependency

Pick the latest available version folder from: https://github.com/redhat-developer/app-services-operator/tree/main/olm/olm-catalog/rhoas-operator

and update the property rhoas-model.version in pom.xml.

start token refresher cotainer to access observatorium

Class BillingMetricTest queries Observatorium. To do so it needs to obtain token which is obtained from local container. To start this process locally, you need a container runtime (Docker) and run following scripts with described environment variables:

  • OBSERVATORIUM_CLIENT_ID
  • OBSERVATORIUM_CLIENT_SECRET
  • OBSERVATORIUM_OIDC_ISSUER_URL
  • OBSERVATORIUM_URL
./hack/run-token-refresher.sh

Maintainers

About

The E2E Test Suite for the Kafka service

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Java 96.1%
  • Gherkin 2.9%
  • Shell 1.0%