Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

junitxml: Incorrect (?) file reported for inherited tests #9388

Open
tpict opened this issue Dec 7, 2021 · 1 comment
Open

junitxml: Incorrect (?) file reported for inherited tests #9388

tpict opened this issue Dec 7, 2021 · 1 comment
Labels
plugin: junitxml related to the junitxml builtin plugin topic: collection related to the collection phase type: question general question, might be closed after 2 weeks of inactivity

Comments

@tpict
Copy link

tpict commented Dec 7, 2021

When a class A inherits tests from a class B in another file, the test cases are reported as belonging to the file that houses class B.

Example:

# example/base.py

class Base:
    def test_something(self):
        assert 1 == 1
# example/test_something.py
from unittest import TestCase

from .base import Base

class MyTest(Base, TestCase):
    pass
> pytest -o junit_family=xunit1 --junit-xml=test.xml example/test_something.py
> cat test.xml
<?xml version="1.0" encoding="utf-8"?>
<testsuites>
  <testsuite name="pytest" errors="0" failures="0" skipped="0" tests="1" time="0.039" timestamp="2021-12-07T09:54:49.284569" hostname="hello">
    <testcase classname="example.test_something.MyTest" name="test_something" file="example/base.py" line="1" time="0.001" />
  </testsuite>
</testsuites>

I'm not sure if this is correct or not. In CircleCI, it prevents the "split tests by timing" feature from working correctly because it cannot find e.g. example/test_something.py in the XML doc. Is this intended? Is there a better method for inheriting tests?

> pip freeze
astroid==2.9.0
attrs==21.2.0
iniconfig==1.1.1
isort==5.10.1
lazy-object-proxy==1.6.0
mccabe==0.6.1
packaging==21.3
platformdirs==2.4.0
pluggy==1.0.0
py==1.11.0
pyparsing==3.0.6
pytest==6.2.5
toml==0.10.2
typing-extensions==4.0.1
wrapt==1.13.3
@Zac-HD Zac-HD added plugin: junitxml related to the junitxml builtin plugin topic: collection related to the collection phase type: question general question, might be closed after 2 weeks of inactivity labels Dec 9, 2021
@tpict
Copy link
Author

tpict commented Dec 9, 2021

Not strictly related but I figure anyone with domain-specific knowledge can answer this too: functional tests have their classname set to the module containing them, not the function itself. This means that Circle cannot divide these tests any more granularly than by file, whereas class-based tests can be split by class (because the classname points to the class). Any idea why the discrepancy?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
plugin: junitxml related to the junitxml builtin plugin topic: collection related to the collection phase type: question general question, might be closed after 2 weeks of inactivity
Projects
None yet
Development

No branches or pull requests

2 participants