Several tests failing on Guix

[tlc28]

I'm working on packaging LibCST for Guix (https://guix.gnu.org), but several tests are failing (see full log at the bottom).

3 of them are failing with the same error message: "No data_provider tests were created for test_type_availability! Please double check your data.". Those are:

• libcst.metadata.tests.test_full_repo_manager
• libcst.metadata.tests.test_type_inference_provider
• libcst.tests.test_pyre_integration

The other is libcst.codemod.tests.test_codemod_cli.TestCodemodCLI, that fails because it cannot parse a piece of code.

Based on what I read in tox.ini, I run the tests using python -m unittest. I also had run python3 -m libcst.codegen.generate visitors; python3 -m libcst.codegen.generate return_types to fix 2 errors.

For the time being, I'll disable those tests, but any help in fixing those would be welcome!

======================================================================
ERROR: libcst.metadata.tests.test_full_repo_manager (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: libcst.metadata.tests.test_full_repo_manager
Traceback (most recent call last):
  File "/gnu/store/q9rm8h9imazsq2c4qiv2yjpvlvliywqb-python-3.8.2/lib/python3.8/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/gnu/store/q9rm8h9imazsq2c4qiv2yjpvlvliywqb-python-3.8.2/lib/python3.8/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/tmp/guix-build-python-libcst-0.3.7.drv-0/libcst-0.3.7/libcst/metadata/tests/test_full_repo_manager.py", line 11, in <module>
    from libcst.metadata.tests.test_type_inference_provider import _test_simple_class_helper
  File "/tmp/guix-build-python-libcst-0.3.7.drv-0/libcst-0.3.7/libcst/metadata/tests/test_type_inference_provider.py", line 14, in <module>
    from libcst.tests.test_pyre_integration import TEST_SUITE_PATH
  File "/tmp/guix-build-python-libcst-0.3.7.drv-0/libcst-0.3.7/libcst/tests/test_pyre_integration.py", line 95, in <module>
    class PyreIntegrationTest(UnitTest):
  File "/tmp/guix-build-python-libcst-0.3.7.drv-0/libcst-0.3.7/libcst/testing/utils.py", line 170, in __new__
    populate_data_provider_tests(dct)
  File "/tmp/guix-build-python-libcst-0.3.7.drv-0/libcst-0.3.7/libcst/testing/utils.py", line 96, in populate_data_provider_tests
    raise ValueError(
ValueError: No data_provider tests were created for test_type_availability! Please double check your data.


======================================================================
ERROR: libcst.metadata.tests.test_type_inference_provider (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: libcst.metadata.tests.test_type_inference_provider
Traceback (most recent call last):
  File "/gnu/store/q9rm8h9imazsq2c4qiv2yjpvlvliywqb-python-3.8.2/lib/python3.8/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/gnu/store/q9rm8h9imazsq2c4qiv2yjpvlvliywqb-python-3.8.2/lib/python3.8/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/tmp/guix-build-python-libcst-0.3.7.drv-0/libcst-0.3.7/libcst/metadata/tests/test_type_inference_provider.py", line 14, in <module>
    from libcst.tests.test_pyre_integration import TEST_SUITE_PATH
  File "/tmp/guix-build-python-libcst-0.3.7.drv-0/libcst-0.3.7/libcst/tests/test_pyre_integration.py", line 95, in <module>
    class PyreIntegrationTest(UnitTest):
  File "/tmp/guix-build-python-libcst-0.3.7.drv-0/libcst-0.3.7/libcst/testing/utils.py", line 170, in __new__
    populate_data_provider_tests(dct)
  File "/tmp/guix-build-python-libcst-0.3.7.drv-0/libcst-0.3.7/libcst/testing/utils.py", line 96, in populate_data_provider_tests
    raise ValueError(
ValueError: No data_provider tests were created for test_type_availability! Please double check your data.


======================================================================
ERROR: libcst.tests.test_pyre_integration (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: libcst.tests.test_pyre_integration
Traceback (most recent call last):
  File "/gnu/store/q9rm8h9imazsq2c4qiv2yjpvlvliywqb-python-3.8.2/lib/python3.8/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/gnu/store/q9rm8h9imazsq2c4qiv2yjpvlvliywqb-python-3.8.2/lib/python3.8/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/tmp/guix-build-python-libcst-0.3.7.drv-0/libcst-0.3.7/libcst/tests/test_pyre_integration.py", line 95, in <module>
    class PyreIntegrationTest(UnitTest):
  File "/tmp/guix-build-python-libcst-0.3.7.drv-0/libcst-0.3.7/libcst/testing/utils.py", line 170, in __new__
    populate_data_provider_tests(dct)
  File "/tmp/guix-build-python-libcst-0.3.7.drv-0/libcst-0.3.7/libcst/testing/utils.py", line 96, in populate_data_provider_tests
    raise ValueError(
ValueError: No data_provider tests were created for test_type_availability! Please double check your data.

======================================================================
FAIL: test_codemod_formatter_error_input (libcst.codemod.tests.test_codemod_cli.TestCodemodCLI)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/guix-build-python-libcst-0.3.7.drv-0/libcst-0.3.7/libcst/codemod/tests/test_codemod_cli.py", line 34, in test_codemod_formatter_error_input
    self.assertIn(
AssertionError: 'error: cannot format -: Cannot parse: 13:10:     async with AsyncExitStack() as stack:' not found in 'Finished codemodding 0 files!\n - Transformed 0 files successfully.\n - Skipped 0 files.\n - Failed to codemod 0 files.\n - 0 warnings were generated.\n'

[jimmylai]

For some reason the data_provider cannot find test fixture files in directory https://github.com/Instagram/LibCST/tree/master/libcst/tests/pyre

You can try run this script to generate the fixture.
https://github.com/Instagram/LibCST/blob/master/libcst/tests/test_pyre_integration.py#L119-L120

The error of test_codemod_formatter_error_input was caused by the expected error wasn't raised, you can manually run the command to dig into.

LibCST/libcst/codemod/tests/test_codemod_cli.py

Lines 16 to 24 in c023fa7

	rlt = subprocess.run(
	[
	"python",
	"-m",
	"libcst.tool",
	"codemod",
	"remove_unused_imports.RemoveUnusedImportsCommand",
	"libcst/codemod/tests/codemod_formatter_error_input.py.txt",
	],

[tlc28]

For some reason the data_provider cannot find test fixture files in directory
https://github.com/Instagram/LibCST/tree/master/libcst/tests/pyre

You can try run this script to generate the fixture.
https://github.com/Instagram/LibCST/blob/master/libcst/tests/test_pyre_integration.py#L119-L120

It's not obvious from the error message, but… could it be that pyre-check is missing? I cannot install it because it depends on libcst that depends on pyre-check that depends on libcst… you get the point.

The error of test_codemod_formatter_error_input was caused by the expected error wasn't raised,
you can manually run the command to dig into.

Thanks! It actually work when I call the script directly. I guess it's a problem with the Guix build environment that I'll have to fix myself.

[thatch]

I think this will be fixed with #365

[tlecarrour-ee]

Hi @thatch! Thanks for letting me know, but… it does not solve the problem! :-(

I fetch the latest 0.3.10 from PyPI, but still have to run the following commands to make the test suite pass:

python -m libcst.codegen.generate visitors
python -m libcst.codegen.generate return_types
rm libcst/tests/test_pyre_integration.py
rm libcst/metadata/tests/test_full_repo_manager.py
rm libcst/metadata/tests/test_type_inference_provider.py
python -m unittest