Unable to run unit tests locally - cannot investigate cpython regression

[vEpiphyte]

Code Sample, a copy-pastable example if possible

/tmp/azure-kusto-python$ python -m pytest --sw azure-kusto-data/tests/aio/
========================================================================================================================================================================================================== test session starts ==========================================================================================================================================================================================================
platform linux -- Python 3.11.4, pytest-7.4.0, pluggy-1.2.0
rootdir: /tmp/azure-kusto-python/azure-kusto-data
plugins: asyncio-0.21.1
asyncio: mode=Mode.STRICT
collected 21 items / 1 error                                                                                                                                                                                                                                                                                                                                                                                                            
stepwise: no previously failed tests, not skipping.

================================================================================================================================================================================================================ ERRORS =================================================================================================================================================================================================================
_______________________________________________________________________________________________________________________________________________________________________________________ ERROR collecting tests/aio/test_async_token_providers.py ________________________________________________________________________________________________________________________________________________________________________________________
/home/epiphyte/.pyenv/versions/tmp3114/lib/python3.11/site-packages/_pytest/runner.py:341: in from_call
    result: Optional[TResult] = func()
/home/epiphyte/.pyenv/versions/tmp3114/lib/python3.11/site-packages/_pytest/runner.py:372: in <lambda>
    call = CallInfo.from_call(lambda: list(collector.collect()), "collect")
/home/epiphyte/.pyenv/versions/tmp3114/lib/python3.11/site-packages/_pytest/python.py:531: in collect
    self._inject_setup_module_fixture()
/home/epiphyte/.pyenv/versions/tmp3114/lib/python3.11/site-packages/_pytest/python.py:545: in _inject_setup_module_fixture
    self.obj, ("setUpModule", "setup_module")
/home/epiphyte/.pyenv/versions/tmp3114/lib/python3.11/site-packages/_pytest/python.py:310: in obj
    self._obj = obj = self._getobj()
/home/epiphyte/.pyenv/versions/tmp3114/lib/python3.11/site-packages/_pytest/python.py:528: in _getobj
    return self._importtestmodule()
/home/epiphyte/.pyenv/versions/tmp3114/lib/python3.11/site-packages/_pytest/python.py:617: in _importtestmodule
    mod = import_path(self.path, mode=importmode, root=self.config.rootpath)
/home/epiphyte/.pyenv/versions/tmp3114/lib/python3.11/site-packages/_pytest/pathlib.py:565: in import_path
    importlib.import_module(module_name)
/home/epiphyte/.pyenv/versions/3.11.4/lib/python3.11/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1204: in _gcd_import
    ???
<frozen importlib._bootstrap>:1176: in _find_and_load
    ???
<frozen importlib._bootstrap>:1147: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:690: in _load_unlocked
    ???
/home/epiphyte/.pyenv/versions/tmp3114/lib/python3.11/site-packages/_pytest/assertion/rewrite.py:178: in exec_module
    exec(co, module.__dict__)
azure-kusto-data/tests/aio/test_async_token_providers.py:10: in <module>
    from ..test_token_providers import KUSTO_URI, TOKEN_VALUE, TEST_AZ_AUTH, TEST_MSI_AUTH, TEST_DEVICE_AUTH, TokenProviderTests, MockProvider
<frozen importlib._bootstrap>:1176: in _find_and_load
    ???
<frozen importlib._bootstrap>:1147: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:690: in _load_unlocked
    ???
/home/epiphyte/.pyenv/versions/tmp3114/lib/python3.11/site-packages/_pytest/assertion/rewrite.py:178: in exec_module
    exec(co, module.__dict__)
azure-kusto-data/tests/test_token_providers.py:49: in <module>
    KUSTO_URI = get_env("ENGINE_CONNECTION_STRING", None)
/home/epiphyte/.pyenv/versions/tmp3114/lib/python3.11/site-packages/azure/kusto/data/env_utils.py:7: in get_env
    if arg in os.environ:
<frozen _collections_abc>:780: in __contains__
    ???
<frozen os>:676: in __getitem__
    ???
<frozen os>:758: in encode
    ???
E   TypeError: str expected, not NoneType
======================================================================================================================================================================================================== short test summary info ========================================================================================================================================================================================================
ERROR azure-kusto-data/tests/aio/test_async_token_providers.py - TypeError: str expected, not NoneType
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=========================================================================================================================================================================================================== 1 error in 0.64s ============================================================================================================================================================================================================

Problem description

When following the directions in CONTRIBUTING.md to run the unit tests, they fail to execute.

The env_utils.get_env() function has def get_env(*args, optional=False, default=None):

but it is being called with KUSTO_URI = get_env("ENGINE_CONNECTION_STRING", None). This is including None as a arg value which gets checked in os.environ and throws an exception. I do not have any of these environment values set since they are not documented in CONTIBUTING.md.

This is preventing me from investigating a cpython regression encountered with the azure-kusto-data package in Python 3.11.5, which I have been able to bisect to python/cpython#106684 .

If query related, does it happen on other platforms (Kusto Web UI, Kusto Explorer)?

This is a python issue.

Output of pip freeze

$ python -m pip freeze
aiohttp==3.8.5
aioresponses==0.7.4
aiosignal==1.3.1
asgiref==3.7.2
async-timeout==4.0.3
attrs==23.1.0
azure-core==1.29.3
azure-identity==1.14.0
azure-kusto-data @ file:///tmp/azure-kusto-python/azure-kusto-data
azure-kusto-ingest @ file:///tmp/azure-kusto-python/azure-kusto-ingest
azure-storage-blob==12.17.0
azure-storage-queue==12.6.0
black==23.7.0
certifi==2023.7.22
cffi==1.15.1
charset-normalizer==3.2.0
click==8.1.7
cryptography==41.0.3
frozenlist==1.4.0
idna==3.4
ijson==3.2.3
iniconfig==2.0.0
isodate==0.6.1
mock==5.1.0
msal==1.23.0
msal-extensions==1.0.0
multidict==6.0.4
mypy-extensions==1.0.0
numpy==1.25.2
packaging==23.1
pandas==2.0.3
pathspec==0.11.2
platformdirs==3.10.0
pluggy==1.2.0
portalocker==2.7.0
pycparser==2.21
PyJWT==2.8.0
pytest==7.4.0
pytest-asyncio==0.21.1
python-dateutil==2.8.2
pytz==2023.3
PyYAML==6.0.1
requests==2.31.0
responses==0.23.3
six==1.16.0
tenacity==8.2.3
types-PyYAML==6.0.12.11
typing_extensions==4.7.1
tzdata==2023.3
urllib3==2.0.4
yarl==1.9.2

[yogilad]

Sorry about that.

You need a Kusto database you have admin rights on.
A Kusto free cluster is the easiest way to acquire one.
You can cerate one here: https://dataexplorer.azure.com/home

You should set then following environment vars.
ENGINE_CONNECTION_STRING=Your cluster URI
DM_CONNECTION_STRING=Your ingest cluster URI
TEST_DATABASE=The name of the database

Any other env should not be mandatory for running test locally.

[vEpiphyte]

I have been able to identify and fix the cpython regression ( which was in some private code which used azure-kusto-data, not in the library itself ).