Unable to mock Variable in python unittest

[ivanwakeup]

What do you see as an issue?

The airflow Best Practices documentation page suggests using unittest.mock.patch along with setting os.environ values to mock Variables and Connections. This doesn't seem to work in practice (testing against airflow 2.8.1)--airflow attempts to read Variable values from variable table in DB instead of using os.environ overrides.

Solving the problem

example non-working code:

from unittest.mock import patch
from airflow.models import Variable

def test_variable_fetch():
    with patch.dict('os.environ', AIRFLOW_VAR_my_test_var="my_test_value"):
        print(Variable.get("my_test_var"))

error stack:

/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pycharm/_jb_pytest_runner.py:8: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  from pkg_resources import iter_entry_points
Launching pytest with arguments test_nlx_bqs_tasks.py::test_variable_fetch --no-header --no-summary -q in /Users/ivanwakeup/projects/nl-airflow-dags/tests

============================= test session starts ==============================
collecting ... collected 1 item

test_nlx_bqs_tasks.py::test_variable_fetch FAILED                        [100%][2024-04-11T11:56:44.440-0700] {variable.py:283} ERROR - Unable to retrieve variable from secrets backend (MetastoreBackend). Checking subsequent secrets backend.
Traceback (most recent call last):
  File "/Users/ivanwakeup/Library/Caches/pypoetry/virtualenvs/nl-airflow-dags-VBs6ACil-py3.9/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context
    self.dialect.do_execute(
  File "/Users/ivanwakeup/Library/Caches/pypoetry/virtualenvs/nl-airflow-dags-VBs6ACil-py3.9/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)
sqlite3.OperationalError: no such table: variable

The above exception was the direct cause of the following exception:

Anything else

https://airflow.apache.org/docs/apache-airflow/2.8.4/best-practices.html#mocking-variables-and-connections - guide page in question

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[ivanwakeup]

I think i found the issue. lowercase my_test_var instead of MY_TEST_VAR in the variable definition causes airflow to not pickup the value. changing this fixes it.

[aritra24]

Closing this issue in that case