Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Xunit & KnownFailure #1395

Merged
merged 2 commits into from Feb 12, 2012
Merged

Xunit & KnownFailure #1395

merged 2 commits into from Feb 12, 2012

Conversation

takluyver
Copy link
Member

For the buildbots, Xunit currently counts KnownFailures as failures. This monkeypatches it to count them as skips, so we get a clean test run, like this:

https://jenkins.shiningpanda.com/ipython/job/ipython-xunit-test/16/

@minrk
Copy link
Member

minrk commented Feb 11, 2012

Looks fine to me. Are there any versioning issues with this? i.e. nose.plugins.xunit not being present?

If that's guaranteed to exist, then go ahead and merge.

@takluyver
Copy link
Member Author

It looks like it was added in nose 0.11, which was out by 2010 or earlier.
Are we happy to require that?

@minrk
Copy link
Member

minrk commented Feb 11, 2012

0.11 is in Ubuntu 10.04 LTS, so that seems plenty conservative to me. Package folks don't break up the plugins into subpackages, do they?

@takluyver
Copy link
Member Author

Debian/Ubuntu packagers don't. It doesn't look like Fedora do, though I'm
less familiar with their system:
http://koji.fedoraproject.org/koji/buildinfo?buildID=293192

@minrk
Copy link
Member

minrk commented Feb 12, 2012

Okay, then go ahead and merge. I think we've done due diligence here.

takluyver added a commit that referenced this pull request Feb 12, 2012
Make Xunit count KnownFailure tests as skipped, not failures.
@takluyver takluyver merged commit 8096bb8 into ipython:master Feb 12, 2012
@minrk
Copy link
Member

minrk commented Feb 13, 2012

@takluyver - this seems to have broken oneiric auto-builds, if you know why that would be. I can't seem to reproduce locally on oneiric with apt-installed nose 1.0.

Output:

https://launchpadlibrarian.net/92736954/buildlog_ubuntu-oneiric-i386.ipython_0.12%2B2321-0~7~oneiric1_FAILEDTOBUILD.txt.gz

@takluyver
Copy link
Member Author

I'm mostly baffled, but I think these lines in nose trunk (a few lines
above the failure) might be a clue:

        if issubclass(ec, SkipTest):
            if not hasattr(self.plugin, 'addSkip'):
                return

https://github.com/nose-devs/nose/blob/master/nose/plugins/manager.py#L324

Not sure how that could make it error, though. I didn't even think we
had any known failures in IPython.parallel. ShiningPanda (Debian 6)
doesn't reproduce the error either.

@takluyver
Copy link
Member Author

(Of course ShiningPanda wouldn't replicate it - that's using nose installed by pip in a virtualenv).

@minrk
Copy link
Member

minrk commented Feb 13, 2012

There's exactly one KnownFailure in IPython.parallel, and it's only raised on Python 3. But I don't think that really as anything to do with this. The exception is raised as soon as the first IPython.parallel test group is finished, and nothing is skipped or fails there. What if we made a small tweak here, and only applied the monkeypatch when xunit is requested?

@takluyver
Copy link
Member Author

Good idea. Check over PR #1405 and merge it if you're happy.

mattvonrocketstein pushed a commit to mattvonrocketstein/ipython that referenced this pull request Nov 3, 2014
Make Xunit count KnownFailure tests as skipped, not failures.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants