Testsuite requires OCI account to run

[glaubitz]

I'm currently packaging oci-cli for openSUSE and SUSE Linux Enterprise distributions.

While creating and building the RPM package is rather simple, it's not possible to enable the testsuite so it's run during package build as the testsuite apparently requires an OCI account:

[   22s] ==================================== ERRORS ====================================
[   22s] __________ ERROR collecting services/audit/tests/integ/test_audit.py ___________
[   22s] services/audit/tests/integ/test_audit.py:10: in <module>
[   22s]     from tests import util
[   22s] tests/util.py:38: in <module>
[   22s]     USER_ID = os.environ['OCI_CLI_USER_ID']
[   22s] /usr/lib64/python3.7/os.py:678: in __getitem__
[   22s]     raise KeyError(key) from None
[   22s] E   KeyError: 'OCI_CLI_USER_ID'
[   22s] ___ ERROR collecting services/budget/tests/unit/test_budget_cli_extended.py ____
[   22s] services/budget/tests/unit/test_budget_cli_extended.py:5: in <module>
[   22s]     from tests import util
[   22s] tests/util.py:38: in <module>
[   22s]     USER_ID = os.environ['OCI_CLI_USER_ID']
[   22s] /usr/lib64/python3.7/os.py:678: in __getitem__
[   22s]     raise KeyError(key) from None
[   22s] E   KeyError: 'OCI_CLI_USER_ID'
[   22s] _ ERROR collecting services/container_engine/tests/integ/test_containerengine.py _
[   22s] services/container_engine/tests/integ/test_containerengine.py:11: in <module>
[   22s]     from tests import util
[   22s] tests/util.py:38: in <module>
[   22s]     USER_ID = os.environ['OCI_CLI_USER_ID']
[   22s] /usr/lib64/python3.7/os.py:678: in __getitem__
[   22s]     raise KeyError(key) from None
[   22s] E   KeyError: 'OCI_CLI_USER_ID'
[   22s] _______ ERROR collecting services/core/tests/integ/test_blockstorage.py ________
[   22s] services/core/tests/integ/test_blockstorage.py:7: in <module>
[   22s]     from tests import util
[   22s] tests/util.py:38: in <module>
[   22s]     USER_ID = os.environ['OCI_CLI_USER_ID']
[   22s] /usr/lib64/python3.7/os.py:678: in __getitem__
[   22s]     raise KeyError(key) from None
[   22s] E   KeyError: 'OCI_CLI_USER_ID'

Would it be possible to provide a (simplified) offline version of the testsuite so that we can enable it for the oci-cli package in openSUSE/SLE?

[paul-hummel-oracle]

Hi,

Tests which are found under services/service/tests/integ/ are integration tests that require connectivity to live services. It would not make sense to try to execute the integration tests during rpm install.
Whereas tests found under “unit” are unit tests which can be run offline.

In either case, to execute the tests requires some setup. Externally, this is not documented. I think these are the environment variables used in the tests. Even for the offline/unit tests, it is probably neccessary to stub-in some dummy values here:

export OCI_CLI_CONFIG_FILE=
export OCI_CLI_COMPARTMENT_ID="{YOUR COMPARTMENT ID HERE}"
export OCI_CLI_COMPARTMENT_NAME="{NAME OF THE COMPARTMENT SPECIFIED BY OCI_CLI_COMPARTMENT_ID}"
export OCI_CLI_ADMIN_USER_ID="{YOUR ADMIN USER OCID}"
export OCI_CLI_USER_ID="{YOUR USER OCID}" # this CAN be the same as the ADMIN USER ID
export OCI_CLI_TENANT_ID="{YOUR TENANCY OCID}"
export OCI_CLI_NAMESPACE="{YOUR OBJECT STORAGE NAMESPACE}" # this is also your tenancy name

Internally, we use pytest to execute the test suite.

[glaubitz]

Whereas tests found under “unit” are unit tests which can be run offline.

Ah, that should be suitable then. The idea for running the testsuite for RPM builds is to be able to catch regressions early on if some of the dependencies of the oci-cli get updated.

For example, the regression with click 7.0 which I reported in #189 would have been caught in SUSE's Open Build System (OBS) after a new version 7.0 of the python-click RPM package was uploaded.

[paul-hummel-oracle]

I also forgot to mention that there are some unit tests found under tests/unit as well. Tests found here are related to the core product, as opposed to tests related to a specific service.

[glaubitz]

I have tried running solely the unit tests with py.test -s tests/unit but I'm still running into the same error:

[   43s] ============================= test session starts ==============================
[   43s] platform linux -- Python 3.7.3, pytest-5.0.1, py-1.8.0, pluggy-0.12.0
[   43s] rootdir: /home/abuild/rpmbuild/BUILD/oci-cli-2.6.1, inifile: tox.ini
[   43s] plugins: forked-1.0.2, cov-2.7.1, xdist-1.29.0
[   43s] collected 22 items
[   43s] 
[   44s] tests/unit/test_cli_util.py EEE
[   44s] tests/unit/test_verify_dependency_consistency.py E
[   44s] tests/unit/scripts/test_install.py EEEEEEEEE
[   45s] tests/unit/util/test_pymd5.py EEEEEEEEE
[   45s] 
[   45s] ==================================== ERRORS ====================================
[   45s] _ ERROR at setup of TestCliUtil.test_coalesce_param_with_explicit_default_value_for_file_type_param _
[   45s] 
[   45s]     @pytest.fixture(scope='session')
[   45s]     def config_file():
[   45s] >       return os.environ['OCI_CLI_CONFIG_FILE']
[   45s] 
[   45s] conftest.py:133: 
[   45s]

Is there anything else I need to do to be able to run unit tests only?

[paul-hummel-oracle]

Ahhh, this is probably getting more difficult to achieve.
The tests rely on CLI being setup and configured. That is, usually you will go through setup and have ~/.oci/config file.

For the test suite you need to specify the location (full path) of the config with this env variable:
export OCI_CLI_CONFIG_FILE=

Here are some sample contents:

[DEFAULT]
user = ocid1.user.oc1..dummy
fingerprint = xx:xx:40:d6:2e:e6:01:58:81:89:f7:76:03:d8:xx:xx
key_file =path_to_keyfile/cli_integration_test_user_key.pem
tenancy = ocid1.tenancy.oc1..dummy
region = us-phoenix-1

[glaubitz]

Is there a solution now? It would be nice to be able to run at least some unit tests.

[paul-hummel-oracle]

Hi @glaubitz,
Sorry I closed this b/c I didn't see a response since my last comment.
Were you able to try this?
Unit tests can be executed without an account. But having correct environment and setup is necessary.

[glaubitz]

I have played a bit with that, but were not able to succeed. I will give it another shot later this week once I find the time.

[vish1]

@glaubitz Were you able to get the unit tests working?

[vish1]

@glaubitz Please let us know if you were able to get the tests working. If there is no response, I'll close the ticket.

[glaubitz]

Testsuite currently fails with 2.9.1, see: https://build.opensuse.org/package/live_build_log/Cloud:Tools/oci-cli:test/Factory/x86_64

[smak-oracle]

Glaubitz, it has been a while since you tested. did you try with the latest CLI?We will close the issue if we didn't hear back from you.

[KartikShrikantHegde]

@glaubitz I will be closing this issue based on the comment above. Please feel free to reopen if required.

Thanks.