Document our config files and structure

conf/README.md

@@ -0,0 +1,117 @@
+# Config directory
+
+In this directory we store all the configuration for cluster and OCSCI config
+files.
+
+During the execution we are loading different config files with different
+sections (DEFAULTS, ENV_DATA, RUN, etc).
+We are using [jinja2](http://jinja.pocoo.org/docs/2.10/templates/#variables)
+template's variables which you can reuse among different configs from
+DEFAULT and different sections.
+
+This gives us the ability to easily reuse variables from context of the
+yaml file itself.
+
+For more information please read the rest of the documentation.
+
+
+## OCS CI Config
+
+We moved all the OCSCI framework related config under
+[ocsci folder](./ocsci/). We have defined currently this file:
+
+* [default_config.yaml](ocsci/default_config.yaml) - this is the default
+    config file for OCS CI which is loaded automatically (don't pass it to
+    --ocsci-conf). It contains multiple sections which user can overwrite in
+    custom file and provide as an option to `--oscci-conf`.
+* In the future we will have more config files in this folder which you can
+    reuse.
+
+## Cluster config
+
+If you would like to overwrite cluster default data you can create file
+similar to [this example](./examples/ocs_basic_install.yml). Example shown
+overwrites below ENV data:
+
+* `platform` - currently we support only AWS, but in future we will cover
+    more.
+* `worker_replicas` - number of replicas of worker.
+* `rook_image` - which rook image can be used.
+* and many more - look at the deployment templates under
+    [../templates/ocs-deployment/](../templates/ocs-deployment/) and you can
+    see the variables in templates. In the deploment we are using ENV_DATA
+    section as data for rendering the templates which allow us to add new
+    values just in config and we don't have to also update the code on all
+    places to take specific variable.
+
+If you define cluster config you have to put all the data under `ENV_DATA`
+section like in the mentioned example. Actually you can add another section as
+well but in cluster config file we recommend to define just `ENV_DATA` section
+or one more additional section which is `DEFAULTS` for propagation those
+defaults among all configs as well.
+
+### Sections in our configs
+
+All of the below sections, except for `DEFAULTS`, will be available from
+[ocsci/config.py](../ocsci/config.py) module. `DEFAULTS` section is exposed to
+[ocs/defaults.py](../ocs/defaults.py) module as described below.
+
+#### DEFAULTS
+
+Default values which are used among different configs and sections.
+
+All variables defined in this section are exposed to
+[../ocs/defaults.py](../ocs/defaults.py) module during the start of the
+execution. So you can access those default data from `ocs/defaults.py` module
+directly from your tests or modules.
+
+#### RUN
+
+Framework RUN related config parameters. If the paremeter is for whole run
+it belongs here.
+
+#### DEPLOYMENT
+
+Deployment related parameters. Only deployment related params not used
+anywhere else.
+
+#### REPORTING
+
+Reporting related config. (Do not store secret data in the repository!).
+
+#### ENV_DATA
+
+Environment specific data. This section is meant to be overwritten by own
+cluster config file, but can be overwritten also here (But cluster config has
+higher priority).
+
+## Example of accessing config/default data
+
+```python
+from ocsci.config import RUN, ENV_DATA
+from ocs import defaults
+
+# From you code you can access those data like
+
+Taking data from ENV_DATA will always use right cluster_namespace passed via
+`--ocsci-conf` config file or default one defined in `default_config.yaml`.
+funciton_which_taking_namespace(namespace=ENV_DATA['cluster_namespace'])
+
+# Defaults data you can access like in this example:
+print(f"Printing some default data like API version: {defaults.API_VERSION}")
+```
+
+## Priority of loading configs:
+
+Lower number == higher priority
+
+1) **CLI args** - sometime we can pass some variables by CLI parameters, in
+    this case those arguments should overwrite everything and have the highest
+    priority.
+2) **cluster config file** - yaml file passed by `--cluster-conf` parameter
+3) **ocsci config file** - ocsci related config passed by `--ocsci-conf`
+    parameter.
+4) **default configuration** - default values and the lowest priority. You can
+    see [default config here](ocsci/default_config.yaml).
+5) **ocsci/config.py** - module data are with lowest priority and can be
+    overwritten with data mentioned above.

conf/examples/ocs_basic_install.yml

@@ -0,0 +1,29 @@
+---
+# This is example of the config for deployment which allows you to overwrite
+# the default values from templates.
+
+# In default section you can overwrite data which will be then exposed to all
+# configs like ocsci/default_config.yaml. As you can see we are using there
+# default ENV_DATA section and you can see for example:
+# ceph_image: '{{ DEFAULTS["CEPH_IMAGE"] }}' which means is take by default
+# from DEFAULTS section from CEPH_IMAGE variable define in this section.
+DEFAULTS:
+  CEPH_IMAGE: "quay.io/rhceph-dev/rhceph-4.0-rhel-8"
+  # this will be used in ENV_DATA section as well cause of example mentioned
+  # above!
+
+ENV_DATA:
+  # take a look at deployment templates variables, here you is the place you
+  # can overwrite all the vars you want.
+  platform: 'aws'
+  worker_replicas: 6  # here you can define diff number of workers
+  master_replicas: 3  # no effect here as default is also 3
+  # you can overwrite default rook image right here. But you will redefine it
+  # just for ENV_DATA section. PPL who will access this ROOK Image from
+  # ocs/defaults.py will still get default one from ocsci/default_config.yaml
+  # from it's DEFAULTS section.
+  rook_image: "quay.io/rhceph-dev/rook"
+
+  # We don't define ceph_image: here so it will be taken from
+  # ocsci/default_config.yaml and as you can see from DEFAULTS which we
+  # re-defining in DEFAULTS section above.

conf/ocsci/README.md

@@ -0,0 +1,6 @@
+# OCSCI related config directory
+
+In this directory we will have common config files which we will use in our
+automation for example for different platforms like AWS or VmWare and so on.
+
+See the documentation from [conf dir](../README.md) for more details!

conf/ocsci/default_config.yaml

@@ -17,25 +17,26 @@
 # This is the place for common default variables used in multiple places.
 # Values from this section are then available from tests from ocs.defaults
 # module.
+#
+# Put here mostly default variables reused just for related config sections
+# below. The default values which are related just to some modules/tests which
+# are not suppose to be used from sections below put directly in
+# ocs/defaults.py module.
 DEFAULTS:
   AWS_REGION: "us-east-2"
-  ROOK_CLUSTER_NAMESPACE: "openshift-storage"
-  KUBECONFIG_LOCATION: "auth/kubeconfig"  # relative from cluster_dir
   API_VERSION: "v1"
-  CEPH_IMAGE: "ceph/ceph:v14.2.0-20190410"
-  ROOK_IMAGE: "rook/ceph:master"
-  DEPLOYMENT_PLATFORM: "AWS"
-
-
 
 # in this RUN section we will keep default parameters for run of OCS-CI
 RUN:
   log_dir: "/tmp"
   run_id: null  # this will be redefined in the execution
-  kubeconfig_location: '{{ DEFAULTS["KUBECONFIG_LOCATION"] }}'
+  kubeconfig_location: 'auth/kubeconfig' # relative from cluster_dir
   cli_params: {}  # this will be filled with CLI parameters data
+  client_version: '{{ DEPLOYMENT["installer_version"] }}'
+  bin_dir: './bin'
 
-# In this section we are storing all deployment related configuration
+# In this section we are storing all deployment related configuration but not
+# the environment related data as those are defined in ENV_DATA section.
 DEPLOYMENT:
   installer_version: "4.1.0-rc.3"
 
@@ -46,10 +47,10 @@ REPORTING:
 
 # This is the default information about environment. Will be overwritten with
 # --cluster-conf file.yaml data you will pass to the pytest.
-ENV_DATA: 
+ENV_DATA:
   platform: '{{ DEFAULTS["DEPLOYMENT_PLATFORM"] }}'
   cluster_name: "ocs-ci"  # needs to be changed in ocscilib plugin
-  cluster_namespace: '{{ DEFAULTS["ROOK_CLUSTER_NAMESPACE"] }}'
+  cluster_namespace: 'openshift-storage'
   region: '{{ DEFAULTS["AWS_REGION"] }}'
-  ceph_image: '{{ DEFAULTS["CEPH_IMAGE"] }}'
-  rook_image: '{{ DEFAULTS["ROOK_IMAGE"] }}'
+  ceph_image: 'ceph/ceph:v14.2.0-20190410'
+  rook_image: 'rook/ceph:master'

docs/README.md

@@ -11,15 +11,12 @@ Rest of the information you can find in those sections
 * [Code review and contribution](./code_review.md) - information about code
     review best practices for the OCS-CI project and tips and rules for
     contribution.
-
 * [Code guidelines](./coding_guidelines.md) - information about code conventions and
     best practices for writing code in the OCS-CI project.
     **Has to be followed for new PRs!**
-
-* [Getting started](./getting_started.md) - Getting started with our testing framework OCS-CI
-
-* [Usage](./usage.md) - information about usage of OCS-CI framework
-
-* [Results](./results.md) - information about results publishing
-
+* [Getting started](./getting_started.md) - Getting started with our testing framework
+    OCS-CI.
+* [Usage](./usage.md) - information about usage of OCS-CI framework.
+* [Config files](../conf/README.md) - information about all our config files.
+* [Results](./results.md) - information about results publishing.
 * [Writing tests](./writing_tests.md) - information about how to write tests.

docs/usage.md

@@ -2,10 +2,11 @@
 
 **Work in progress**
 
-`run.py` is the main script for ocs-ci. You can view the full usage details by
-passing in the `--help` argument.
 
-For pytest run py.test --help
+`run.py` - (**deprecated**) is the main script for ocs-ci. You can view the
+full usage details by passing in the `--help` argument.
+
+For pytest run python -m run_ocsci --help
 
 ```bash
 python run.py --help
@@ -30,7 +31,7 @@ There are a few arguments that are required ocs test execution:
 ## Useful pytest arguments
 
 Some non-required arguments that we end up using a lot. You can use
-`py.test --help` to see all the parameters and description which you can pass
+`python -m run_ocsci --help` to see all the parameters and description which you can pass
 to the pytest.
 
 * `--capture=no` - when using pdb or ipdb you have to turn of capture mode
@@ -82,7 +83,8 @@ python run.py --suite suites/ocs_basic_install.yml --log-level info
 
 ```bash
 python run.py --cluster-name=my-testing-cluster \
-    --suite=suites/custom-test.yml --cluster-path=/home/your_login/my-testing-cluster \
+    --suite=suites/custom-test.yml \
+    --cluster-path=/home/your_login/my-testing-cluster \
     --no-email
 ```
 
@@ -109,7 +111,7 @@ on existing cluster with second execution of `pytest`!
 Deployment is moved already to pytest. If you would like to deploy new cluster
 you can run folowing command:
 ```bash
-py.test -m deployment  --ocsci-conf conf/ocsci/custom_config.yaml \
+python -m run_ocsci -m deployment --ocsci-conf conf/ocsci/custom_config.yaml \
     --cluster-conf conf/ocs_basic_install.yml \
     --cluster-name kerberos_ID-ocs-deployment \
     --cluster-path /home/my_user/my-ocs-dir tests/
@@ -121,6 +123,7 @@ values.
 #### Runing tests on deployed environment
 
 ```bash
-py.test -m "tier1 and manage"  --cluster-name kerberos_ID-ocs-deployment \
+python -m run_ocsci -m "tier1 and manage" \
+    --cluster-name kerberos_ID-ocs-deployment \
     --cluster-path /home/my_user/my-ocs-dir tests/
  ```

docs/writing_tests.md

@@ -9,16 +9,23 @@ a lot!
 ## Pytest marks
 
 We have predefined some of pytest marks you can use to decorate your tests.
-You can find them defined in [pytest.ini](../pytest.ini) where we inform pytest
-about this markers.
+You can find them defined in [pytest.ini](../pytest.ini) where we inform
+pytest about those marks.
 
 We have markers defined in pytest_customization package under
-[marks.py](../pytest_customization/marks.py) plugin. From your tests you can
-import directly from ocsci package with `from ocsci import tier1` for example.
+[marks.py](../ocsci/pytest_customization/marks.py) plugin. From your tests you
+can import directly from `ocsci.testlib` module with this statement:
+`from ocsci.testlib import tier1` for example.
 
 
 ## Base test calsses for teams
 
 Those are located in [testlib.py](../ocsci/testlib.py) which you can also
-import directly from ocsci package with `from ocsci import manage` which is
-base test class for manage team.
+import from `ocsci.testlib` module with statement:
+`from ocsci.testlib import manage` which is base test class for manage team.
+
+
+## Other notes
+
+Of course you can import in one line both team base class and marker with
+statement: `from ocsci.testlib import manage, tier1`

ocs/defaults.py

@@ -8,6 +8,20 @@
 TOP_DIR = os.path.dirname(THIS_DIR)
 TEMPLATE_DIR = os.path.join(TOP_DIR, "templates/ocs-deployment/")
 
+# This module is automatically loaded with variables defined in
+# conf/ocsci/default_config.yaml in its DEFAULTS section.
+# If the variable can be used in some config file sections from ocsci/config.py
+# module, plese put your defaults rather to mentioned default_config.yaml file!
+
+# See the documentation in conf/README.md file to understand this config file.
+
+# Those variables are duplicate at the moment from default_config.yaml and once
+# we drop support for old runner we will remove those variables from here and
+# will have them defined only on one place.
+
+# Be aware that variables defined below are not used anywhere in th config
+# files and their sections when we rendering config!
+
 INSTALLER_VERSION = '4.1.0-rc.3'
 CLIENT_VERSION = INSTALLER_VERSION
 AWS_REGION = 'us-east-2'
@@ -18,8 +32,10 @@
 CEPH_IMAGE = "ceph/ceph:v14.2.0-20190410"
 ROOK_IMAGE = "rook/ceph:master"
 DEPLOYMENT_PLATFORM = 'AWS'
-BIN_DIR = './bin'
 
+# This section is suppose to be available just from ocsci/config.py module from
+# ENV_DATA dictionary. Once we drop support of old runner we will delete this
+# data from here as well.
 ENV_DATA = {
     'platform': DEPLOYMENT_PLATFORM,
     'cluster_name': CLUSTER_NAME,

ocs/ocp.py

@@ -4,7 +4,7 @@
 import os
 import logging
 import yaml
-from ocs import defaults
+from ocsci.config import ENV_DATA
 from munch import munchify
 
 from ocs.exceptions import CommandFailed
@@ -256,7 +256,7 @@ def get_ceph_tools_pod():
     Returns:
         str: The Ceph tools pod name
     """
-    ocp_pod_obj = OCP(kind='pods', namespace=defaults.ROOK_CLUSTER_NAMESPACE)
+    ocp_pod_obj = OCP(kind='pods', namespace=ENV_DATA['cluster_namespace'])
     ct_pod = ocp_pod_obj.get(
         selector='app=rook-ceph-tools'
     ).toDict()['items'][0]['metadata']['name']
@@ -274,7 +274,7 @@ def exec_ceph_cmd(ceph_cmd):
     Returns:
         dict: Ceph command output
     """
-    ocp_pod_obj = OCP(kind='pods', namespace=defaults.ROOK_CLUSTER_NAMESPACE)
+    ocp_pod_obj = OCP(kind='pods', namespace=ENV_DATA['cluster_namespace'])
     ct_pod = get_ceph_tools_pod()
     ceph_cmd += " --format json-pretty"
     out = ocp_pod_obj.exec_cmd_on_pod(ct_pod, ceph_cmd)

ocs/rook.py

@@ -13,6 +13,7 @@
 from ocs.pod import Pod
 import ocs.defaults as default
 from utility.templating import generate_yaml_from_jinja2_template_with_data
+from ocsci.config import ENV_DATA
 
 logger = logging.getLogger(__name__)
 
@@ -62,7 +63,7 @@ def __init__(self, **config):
         Read namespace and cluster name from config.
         """
         self._namespace = config.get(
-            'namespace', default.ROOK_CLUSTER_NAMESPACE
+            'namespace', ENV_DATA['cluster_namespace']
         )
         # Keeping api_client_name for upcoming PR
         self._api_client_name = config.get('api_client_name', 'OCRESTClient')
@@ -182,7 +183,7 @@ def create_cephblockpool(
         )
         # overwrite the namespace with openshift-storage, since cephblockpool
         # is tied-up with openshift-storage
-        namespace = default.ROOK_CLUSTER_NAMESPACE
+        namespace = ENV_DATA['cluster_namespace']
 
         cephblockpool_data = {}
         cephblockpool_data['cephblockpool_name'] = cephblockpool_name