-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Not uploading coverage reports when there are tests failures/errors #77
Comments
I believe that it is a common practice not to upload the report upon failures. However, you could modify your travis/appveyor configuration such that the report is still uploaded even upon failures. |
Yes, but I do think it is trivial because the upload seems to be done after the https://github.com/SublimeText/UnitTesting/blob/master/sbin/run_tests.py python script finishes. It seems that Lines 48 to 54 in 9b938ee
I think I could trick it by forcing the |
Instead of putting the |
Thanks, it kinda worked. But I have a hard time trying to edit the env:
global:
- PACKAGE="PackagesManager" # Package name
- SUBLIME_TEXT_VERSION="3"
# use UNITTESTING_TAG to specific tag of UnitTesting
# - UNITTESTING_TAG="master"
# mutliple os matrix
# https://docs.travis-ci.com/user/multi-os/#Python-example-(unsupported-languages)
matrix:
include:
- os: linux
language: python
python: 3.3
- os: osx
language: generic
before_install:
- curl -OL https://raw.githubusercontent.com/SublimeText/UnitTesting/master/sbin/travis.sh
- if [ "$TRAVIS_OS_NAME" == "osx" ]; then
brew update;
brew install python3;
pip3 install python-coveralls;
pip3 install codecov;
fi
- if [ "$TRAVIS_OS_NAME" == "linux" ]; then
pip install python-coveralls;
pip install codecov;
fi
# enable gui, see https://docs.travis-ci.com/user/gui-and-headless-browsers
- if [ "$TRAVIS_OS_NAME" == "linux" ]; then
export DISPLAY=:99.0;
sh -e /etc/init.d/xvfb start;
fi
install:
# bootstrap the testing environment
- sh travis.sh bootstrap
# install Package Control and package denepdencies
# - sh travis.sh install_package_control
script:
# run tests with test coverage report
- sh travis.sh run_tests --coverage
# testing syntax_test files
# - sh travis.sh run_syntax_tests
- coveralls
- codecov
after_success:
# remove the following if `coveralls` is not needed
- echo
notifications:
email: false |
I tried the same for AppVeyor, but the script just halted:
https://ci.appveyor.com/project/evandrocoan/packagesmanager/build/1.0.11 This were the changes to environment:
# The package name
PACKAGE: "PackagesManager"
SUBLIME_TEXT_VERSION : "3"
install:
- ps: appveyor DownloadFile "https://raw.githubusercontent.com/SublimeText/UnitTesting/master/sbin/appveyor.ps1"
- ps: .\appveyor.ps1 "bootstrap" -verbose
# install Package Control
# - ps: .\appveyor.ps1 "install_package_control" -verbose
build: off
test_script:
# run tests with test coverage report
- ps: .\appveyor.ps1 "run_tests" -coverage -verbose
- "SET PYTHON=C:\Python33"
- "SET PATH=%PYTHON%;%PYTHON%\\Scripts;%PATH%"
- pip install codecov
- codecov
after_test:
- echo |
you may try putting the codecov under |
It was originally under there. It seems that Then I tried putting it under some other sections. Hence, it seems to be running when I put it under
https://ci.appveyor.com/project/evandrocoan/packagesmanager/build/1.0.14 When it runs under Travis the output is a little different (i.e., with no not find error):
https://travis-ci.org/evandrocoan/PackagesManager/jobs/326004718 |
issue: Not uploading coverage reports when there are tests failures/errors SublimeText/UnitTesting#77
Even with failure, the coverage is generated, but after that its results are not uploaded. The repository is: https://github.com/evandrocoan/PackagesManager
It must always load the results because there is nothing wrong with some tests failing. It is well expected that some tests are going to fail, for some time or even for ever, but just because they fail does not mean they should be moved from the application or block something else.
The text was updated successfully, but these errors were encountered: