Some unit tests cannot be run independently

[fabioz]

I cannot run the tests on my local machine by downloading the code going into the RF dir and executing:

python utest/run.py -q running

The errors I have are below (after improving the output a bit with the PR: #4610).

The errors seem to happen because the import path isn't setup (so, I guess imports are failing, but I'm not sure how to print that additional info)...

i.e.: in test_imports.TestImports.test_create I can't see how the keyword My Test Keyword can be loaded from the robotframework\utest\resources\test_resource.txt.

Is some additional setup needed in order to run these tests?

(py311_tests) λ python utest/run.py -q running
======================================================================
FAIL: test_create (test_imports.TestImports.test_create)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\robocorpws\robotframework\utest\running\test_imports.py", line 33, in run_and_check_pass
    assert_suite(result, 'Suite', 'PASS')
  File "X:\robocorpws\robotframework\utest\running\test_imports.py", line 16, in assert_suite
    assert_equal(suite.status, status)
  File "X:\robocorpws\robotframework\utest\..\src\robot\utils\asserts.py", line 181, in assert_equal
    _report_inequality(first, second, '!=', msg, values, formatter)
  File "X:\robocorpws\robotframework\utest\..\src\robot\utils\asserts.py", line 230, in _report_inequality
    raise AssertionError(msg)
AssertionError: FAIL != PASS

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "X:\robocorpws\robotframework\utest\running\test_imports.py", line 52, in test_create
    self.run_and_check_pass(suite)
  File "X:\robocorpws\robotframework\utest\running\test_imports.py", line 41, in run_and_check_pass
    raise AssertionError('\n'.join(full_msg)) from e
AssertionError: No keyword with name 'My Test Keyword' found.

======================================================================
FAIL: test_resource (test_imports.TestImports.test_resource)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\robocorpws\robotframework\utest\running\test_imports.py", line 33, in run_and_check_pass
    assert_suite(result, 'Suite', 'PASS')
  File "X:\robocorpws\robotframework\utest\running\test_imports.py", line 16, in assert_suite
    assert_equal(suite.status, status)
  File "X:\robocorpws\robotframework\utest\..\src\robot\utils\asserts.py", line 181, in assert_equal
    _report_inequality(first, second, '!=', msg, values, formatter)
  File "X:\robocorpws\robotframework\utest\..\src\robot\utils\asserts.py", line 230, in _report_inequality
    raise AssertionError(msg)
AssertionError: FAIL != PASS

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "X:\robocorpws\robotframework\utest\running\test_imports.py", line 67, in test_resource
    self.run_and_check_pass(suite)
  File "X:\robocorpws\robotframework\utest\running\test_imports.py", line 41, in run_and_check_pass
    raise AssertionError('\n'.join(full_msg)) from e
AssertionError: No keyword with name 'My Test Keyword' found.

======================================================================
FAIL: test_variables (test_imports.TestImports.test_variables)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\robocorpws\robotframework\utest\running\test_imports.py", line 33, in run_and_check_pass
    assert_suite(result, 'Suite', 'PASS')
  File "X:\robocorpws\robotframework\utest\running\test_imports.py", line 16, in assert_suite
    assert_equal(suite.status, status)
  File "X:\robocorpws\robotframework\utest\..\src\robot\utils\asserts.py", line 181, in assert_equal
    _report_inequality(first, second, '!=', msg, values, formatter)
  File "X:\robocorpws\robotframework\utest\..\src\robot\utils\asserts.py", line 230, in _report_inequality
    raise AssertionError(msg)
AssertionError: FAIL != PASS

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "X:\robocorpws\robotframework\utest\running\test_imports.py", line 76, in test_variables
    self.run_and_check_pass(suite)
  File "X:\robocorpws\robotframework\utest\running\test_imports.py", line 41, in run_and_check_pass
    raise AssertionError('\n'.join(full_msg)) from e
AssertionError: Variable '${MY_VARIABLE}' not found.

----------------------------------------------------------------------
Ran 296 tests in 2.281s

FAILED (failures=3)

[pekkaklarck]

The failing tests import a resource file that exists in utest/resources from that should be in PYTHONPATH. utest/run.py should set that directory into PYTHONPATH but doesn't. Apparently it's set there by some other tests, because if you run all unit tests also these tests succeed. Anyway, easy to fix.