OCM-7958 | ci: Update readme for tests folder to document how to prep…

…are cluster and how to run day2 cases

update

enhance

update

update

tests/README.md

@@ -14,26 +14,57 @@ tests
 |    |____exec                                    ---- exec contains the different services to run the commands
 |    |    |____rosacli                            ---- ROSA CLI specific services/commands
 |    |____log                                     ---- tests logger
+|    |____profilehandler                          ---- package which support create cluster by profile configuration
 |       
 |____prow_ci.sh
 ```
 
 ## Contibute to ROSA CLI tests
 
 Please read the structure and contribute code to the correct place
+### Contribute to day1
+1. Enable the configuration in _[profiles](./ci/data/profiles)_
+2. Mapping the configuration in _[ClusterConfig](./utils/profilehandler/interface.go)_
+3. Define the userdata preparation function in _[data_preparation](./utils/profilehandler/data_preparation.go)_
+4. Call the functions in the _[GenerateClusterCreateFlags](./utils/profilehandler/profile_handler.go)_
 
 ### Contribute to day2
 
-* Create the case in rosa/tests/e2e/<feature name>_test.go
+* Create the case in _rosa/tests/e2e/{feature name}_test.go_
 * Label the case with ***CI.Day2***
 * Label the case with importance ***CI.Critical*** or ***CI.High***
 * Don't need to run creation step, just in BeforeEach step call function  ***config.GetClusterID()*** it will load the clusterID prepared from env or cluster_id file
 * Code for day2 actions and check step
 * Every case need to recover the cluster after the case run finished unless it's un-recoverable
+* Case format should follow 
+  * _main feature description in Describe level_ at the same time _testing purpose in It level_.
+  * `id` of the case is included which will follow the fmt of _[id:<id>]_
+  * Use `By("")` to describe the steps
+  * An example as below
+```golang
+var _ = Describe("Create Machine Pool", func() {
+  It("to hosted cluster with additional security group IDs will work - [id:72195]",func(){
+    By("Prepare security groups")
+    // security groups preparation code
+
+    By("Create machinepool with security groups configured")
+    // machinepool creation code
+
+    By("Verify the machinepool is created with security groups")
+    // machinepool security groups verification code
+  })
+}
+```
+  * The commit and PR should follow
+    * Only one commit is allowed per PR, if multiple commits created please squash them with command 
+    `git rebase -i HEAD~N`(_N_ is the commits number you would squashed)
+    * The commit and PR title should follow rule of [contributing-to-rosa](../CONTRIBUTING.md#contributing-to-rosa)
+    * Case id must be included in the PR/commit title if new automated or updated. Comma-separated if multiple included in same PR/commit. For example
+    `<card id> | test: automated cases id:123456,123457`
 
 ### Labels
 
-* Label your case with the ***CI.Feature<feature name>*** defined in rosa/tests/ci/labels/features.go
+* Label your case with the ***CI.Feature{feature name}*** defined in rosa/tests/ci/labels/features.go
 * Label your case with importance defined in rosa/tests/ci/labels/importance.go
 * Label your case with ***CI.Day1Post/CI.Day2/CI.Day3*** defined in rosa/tests/ci/labels/runtime.go, according to the case runtime
 * Label your case with ***CI.Exclude*** if it fails CI all  the time and you can't fix it in time
@@ -44,31 +75,68 @@ Please read the structure and contribute code to the correct place
 ### Prerequisite
 
 Please read repo's [README.md](../README.md)
-For the test cases, we need `make install` to make the rosa command line installed to local
-
-#### Users and Tokens
-
-Please login ocm and aws cli prior to launching the tests.
-
-#### Global variables
-
-To declare the cluster id, use the below variable::
-* export CLUSTER_ID = <cluster_id>
-
-### Running a local CI simulation
-
+For the test cases, we need `$ make install` to make the rosa command line installed to local
+
+### Users and Tokens
+
+1. Make local aws configuration finishing to launching the tests.
+2. Please login rosacli with the token:
+  * `$ rosa login --env staging --token $ROSA_USER_TOKEN`
+3. Run rosa init to check all configurations are working well:
+  * `$ rosa init`
+
+### Day1 cluster preparation
+
+1. Pick a profile for the cluster creation according to the configurations from 
+  - [rosa-classic profiles](./ci/data/profiles/rosa-classic.yaml)
+  - [rosa-hcp profiles](./ci/data/profiles/rosa-hcp.yaml)
+  - [external team profiles](./ci/data/profiles/external.yaml)
+
+2. Export the profile name as an environment variable
+  * `$ export TEST_PROFILE=<PROFILE NAME>`
+
+3. Create cluster according to the profile configuration
+  * `$ ginkgo run --label-filter day1-prepare tests/rosa --timeout 2h`
+
+4. Wait for the cluster preparation finished
+
+> [!CAUTION]
+> **The profiles with a _TODO_ is not supported yet**
+
+> [!NOTE]
+> Supported environment variables to override the profile configurations
+> * **SHARED_DIR** if you have the env variable set, all output files will be put under it, otherwise it will create a dir _output/${TEST_PROFILE}_
+> * **ARTIFACT_DIR** if you configured the env variable, files need to be archived will be recorded in it. Otherwise it will be recorded to dir _output/${TEST_PROFILE}_
+> * **CHANNEL_GROUP** if it is set, the *channel_group* in profile configuration will be override
+> * **VERSION** if it is set, the _version_ in profile configuration will be override. Supported values 
+>    - _`4.15`_ it will pick the latest z-stream version in minor release of _4.15_
+>    - _`latest`_ it will pick the latest version in current channel_group
+>    - _`4.16.0-rc.0`_ will match the exact version set
+>    - _`y-1`_ will pick a minor stream upgrade version
+>    - _`z-1`_ will pick a optional stream upgrade version
+> * **REGION** if it is set, the _region_ in profile configuration will be override. NOTE: rosa cluster with proxy will fail on region `us-east-1`. It's a known issue.
+> * **PROVISION_SHARD** if it is set, a provision shard will be specified for cluster provision
+> * **NAME_PREFIX** if it is set, all resources will be generated based with the name prefix to identify the created cluster created by you. Otherwise _`rosacli-ci`_ will be used. For local testing, we should have it be set with your alias
+> * **CLUSTER_TIMEOUT** if it is set, the process will exit if cluster cannot be ready in setting time. Unit is minute
+
+### Running a local CI test simulation
 This feature allows for running tests through a case filter to simulate CI. Anyone can customize the case label filter to select the specific cases that would be run. 
 
-* Run day2 or day1-post cases with profile
-  * Run ginkgo run command
-    * `ginkgo run --label-filter '(Critical,High)&&(day1-post,day2)&&!Exclude' tests/e2e`
-* Run a specified case to debug
-  * `ginkgo -focus <case id> tests/e2e`
+1. To declare the cluster id, use the below variable
+  * ```$ export CLUSTER_ID=<cluster_id>```
 
-### Set log level
+2. Run cases with the profile
+    * Running cases based on label filter which to simulate CI jobs
+      * `$ ginkgo run --label-filter '(Critical,High)&&(day1-post,day2)&&!Exclude' tests/e2e`
+    * Run a specified case to debug
+      * `$ ginkgo run -focus <case id> tests/e2e`
 
-* Log level defined in rosa/tests/utils/log/logger.go
+### Resources destroy
 
-```golang
-Logger.logger.SetLevel()
-```
+## Additional configuration
+> [!TIP]
+> Set log level
+> Log level defined in rosa/tests/utils/log/logger.go
+> ```golang
+> Logger.logger.SetLevel()
+> ```

tests/ci/data/profiles/rosa-classic.yaml

@@ -69,7 +69,7 @@ profiles:
   account-role:
     path: "/test/"
     permission_boundary: "arn:aws:iam::aws:policy/AdministratorAccess"
-- as: rosa-shared-vpc
+- as: rosa-shared-vpc # TODO it is not supported right now.
   version: latest
   channel_group: candidate
   region: "us-east-1"