New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collecting tests from Python modules not generated by Python files #2369

Closed
Kodiologist opened this Issue Apr 16, 2017 · 8 comments

Comments

Projects
None yet
3 participants
@Kodiologist
Contributor

Kodiologist commented Apr 16, 2017

I'm a core developer of the Hy programming language, which is a Lisp syntax for Python, and I'm trying to migrate its test suite from Nose to pytest. The repository has a lot of tests written in Python but also tests written in Hy. Hy itself is implemented in Python and provides a meta-importer so that a Python program can import a Hy module just like a Python module. So in theory, pytest should be able to collect tests from Hy files just as well as Python files, so long as Hy's meta-importer is active. Once a Hy file is loaded, there's basically no difference between Hy and Python; a Hy module is in fact a Python module.

I found a hacky way to run the Hy tests, using this conftest.py, which calls _pytest.python.pytest_pycollect_makemodule in a pytest_collect_file hook. The catch is that this hits assert name.endswith(".py") in _pytest.python, and Hy files end with .hy, not .py. If the assertion is commented out, everything works.

Is there a better way to do this? If not, perhaps you should remove the assertion.

I'm using pytest 3.0.7. Hy is intended to support, and is tested on, Python 2.7 and Python 3.3 and up.

@Kodiologist

This comment has been minimized.

Contributor

Kodiologist commented Apr 18, 2017

I'm now just spoofing the module name to look like it ends with .py, for lack of better ideas.

@RonnyPfannschmidt

This comment has been minimized.

Member

RonnyPfannschmidt commented Apr 18, 2017

py.test itself checks the filenames of such modules, in order to override it you will need to create a own collector that handles the import

since you left out the trackback i dont know off-hand whats needed

@Kodiologist

This comment has been minimized.

Contributor

Kodiologist commented Apr 18, 2017

Sorry, here's an example of the backtraces I get when running pytest on my repository without the module-name spoof:

_______________ ERROR collecting tests/native_tests/extra/reserved.hy ________________
../lib/python3.6/site-packages/_pytest/runner.py:163: in __init__
    self.result = func()
../lib/python3.6/site-packages/_pytest/main.py:476: in _memocollect
    return self._memoizedcall('_collected', lambda: list(self.collect()))
../lib/python3.6/site-packages/_pytest/main.py:347: in _memoizedcall
    res = function()
../lib/python3.6/site-packages/_pytest/main.py:476: in <lambda>
    return self._memoizedcall('_collected', lambda: list(self.collect()))
../lib/python3.6/site-packages/_pytest/python.py:412: in collect
    return super(Module, self).collect()
../lib/python3.6/site-packages/_pytest/python.py:332: in collect
    l.sort(key=lambda item: item.reportinfo()[:2])
../lib/python3.6/site-packages/_pytest/python.py:332: in <lambda>
    l.sort(key=lambda item: item.reportinfo()[:2])
../lib/python3.6/site-packages/_pytest/python.py:264: in reportinfo
    modpath = self.getmodpath()
../lib/python3.6/site-packages/_pytest/python.py:238: in getmodpath
    assert name.endswith(".py")
E   AssertionError
@RonnyPfannschmidt

This comment has been minimized.

Member

RonnyPfannschmidt commented Apr 18, 2017

this one is a bit tricky. the "python module" collector of pytest expects a python module for the filename

for now its a pretty good idea to just copy the method and replace its constants replacing .py with .py

@Kodiologist

This comment has been minimized.

Contributor

Kodiologist commented Apr 18, 2017

Okay, thanks.

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Apr 19, 2017

Taking a quick look at the code, it seems to me removing the assertion as @Kodiologist suggests is OK:

        for node in chain:
            if isinstance(node, Instance):
                continue
            name = node.name
            if isinstance(node, Module):
                assert name.endswith(".py")
                name = name[:-3]
                if stopatmodule:
                    if includemodule:
                        parts.append(name)
                    break
            parts.append(name)

I think the assertion is there just because of the name = name[:-3] below it, which could be replaced by a safer os.path.splitext(name)[0]. If all tests pass I don't think this will bring any problems.

Would you care to give it a shot @Kodiologist?

@Kodiologist

This comment has been minimized.

Contributor

Kodiologist commented Apr 19, 2017

@nicoddemus That seems to work fine for Hy's test suite.

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Apr 19, 2017

Would you like to open a PR? Then we can test it against pytest test suite. 😉

This was referenced Mar 6, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment