Coveralls fails to upload with HTTP 422 "Unprocessable Entity" on retried builds

[nolar]

Hello. First of all, thank you for your service free of charge!

Now, to the problem:

Coveralls fails on sending the data with HTTP 422 Unprocessable Entity:

• https://travis-ci.org/github/zalando-incubator/kopf/jobs/680475472

$ coveralls
Submitting coverage to coveralls.io...
Could not submit coverage: 422 Client Error: Unprocessable Entity for url: https://coveralls.io/api/v1/jobs
Traceback (most recent call last):
  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/coveralls/api.py", line 194, in wear
    response.raise_for_status()
  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 422 Client Error: Unprocessable Entity for url: https://coveralls.io/api/v1/jobs
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/coveralls/cli.py", line 78, in main
    result = coverallz.wear()
  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/coveralls/api.py", line 197, in wear
    raise CoverallsException('Could not submit coverage: {}'.format(e))
coveralls.exception.CoverallsException: Could not submit coverage: 422 Client Error: Unprocessable Entity for url: https://coveralls.io/api/v1/jobs
The command "coveralls" exited with 1.

This happens always with one specific scenario:

• Run a build normally, allow the tests to fail, allow the coverage to upload whatever it has collected.
• Retrigger the build.
• The tests do pass now (it was a temporary timing error).
• The coverage upload now does not work with HTTP 422.

First of all, some better error message or status code would be nice. HTTP 409 "Conflict" is usually returned when trying to create something that already exists (if that is actually the cause).

And second: how to fix this issue? Previously, it worked even when the tests were failing and then retried.

Thanks.

[Ark-kun]

We have the same issue in many of our PRs. We use coveralls in Travis. Couple of days ago it stopped working in some cases. And the retries do not help.
I'll be removing coveralls for now to unblock the PRs.

[piyush0]

I had the same issue, make sure that travis has COVERALLS_REPO_TOKEN this environment variable correctly set.

[hannes-ucsc]

Yes, status code should be 409.

[duttonw]

un-related but still helpful for me.

If you fork a repo that has circleci and it uses coveralls, you need to setup coveralls yourself and add to the repo project environment settings the COVERALLS_REPO_TOKEN and the token for your forked repo.

I think this is the case if "Pass secrets to builds from forked pull requests" is set to Disabled.

[solancer]

I was facing the same issue on Github actions, adding --service=gihtub in front of the coveralls command fixed it for me

• name: Coveralls
  env:
  GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  run: coveralls --service=github

[jugmac00]

I was facing the same issue on Github actions, adding --service=gihtub in front of the coveralls command fixed it for me

• name: Coveralls
  env:
  GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  run: coveralls --service=github

Same issue - and this solution is not helping.

I mirgrated 20+ repositories from Travis CI to GitHub Actions, and of those ca. 17 just worked, and 3 do not work at all and error with above (422) error message, although all repositories have a uniform layout.

gh actions with errors
https://github.com/morepath/morepath_rest_dump_load/runs/1754287368?check_suite_focus=true
https://github.com/morepath/morepath_batching/actions/runs/497059413
https://github.com/morepath/more.mako/runs/1730894898?check_suite_focus=true

[gdubicki]

  run: coveralls --service=github

Thanks @solancer , this helped.

But I was really close to not even trying this. Because a moment before Coveralls outputted this in a previously failed run:

coveralls.exception.CoverallsException: Running on Github Actions but GITHUB_TOKEN is not set. Add "env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}" to your step config.

...so I was sure that it already knows that it is running on GitHub (Actions).

[solancer]

I was facing the same issue on Github actions, adding --service=gihtub in front of the coveralls command fixed it for me

• name: Coveralls
  env:
  GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  run: coveralls --service=github

Same issue - and this solution is not helping.

I mirgrated 20+ repositories from Travis CI to GitHub Actions, and of those ca. 17 just worked, and 3 do not work at all and error with above (422) error message, although all repositories have a uniform layout.

gh actions with errors
https://github.com/morepath/morepath_rest_dump_load/runs/1754287368?check_suite_focus=true
https://github.com/morepath/morepath_batching/actions/runs/497059413
https://github.com/morepath/more.mako/runs/1730894898?check_suite_focus=true

Try setting the env before the run

[jugmac00]

@solancer I am not sure what you mean with before the run.

Could you create a Pull Request to any of the three mentioned repositories?

That would be super awesome!

[afinetooth]

@jugmac00 can you please try running some builds in debug mode for those failing repos, per instructions here.

Please share the new build logs and we'll see if we can gain some more insight as to what is failing and why.

[solancer]

@jugmac00 I've raised a PR with a fix, seems to work fine now

morepath/morepath_rest_dump_load#12

[jugmac00]

@afinetooth As requested, I ran coveralls with the debug command, see https://github.com/morepath/morepath_rest_dump_load/pull/14/checks?check_run_id=1767517263

The relevant output is:

Run pip install coveralls
Collecting coveralls
  Downloading coveralls-3.0.0-py2.py3-none-any.whl (13 kB)
Collecting requests>=1.0.0
  Downloading requests-2.25.1-py2.py3-none-any.whl (61 kB)
Collecting docopt>=0.6.1
  Downloading docopt-0.6.2.tar.gz (25 kB)
Collecting coverage<6.0,>=4.1
  Downloading coverage-5.3.1-cp39-cp39-manylinux2010_x86_64.whl (244 kB)
Collecting certifi>=2017.4.17
  Downloading certifi-2020.12.5-py2.py3-none-any.whl (147 kB)
Collecting chardet<5,>=3.0.2
  Downloading chardet-4.0.0-py2.py3-none-any.whl (178 kB)
Collecting idna<3,>=2.5
  Downloading idna-2.10-py2.py3-none-any.whl (58 kB)
Collecting urllib3<1.27,>=1.21.1
  Downloading urllib3-1.26.2-py2.py3-none-any.whl (136 kB)
Building wheels for collected packages: docopt
  Building wheel for docopt (setup.py): started
  Building wheel for docopt (setup.py): finished with status 'done'
  Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13705 sha256=a49c0db692e5e04e9eb0f135e93357271954c5786cb44a1a89f3d8069c65a8e4
  Stored in directory: /home/runner/.cache/pip/wheels/70/4a/46/1309fc853b8d395e60bafaf1b6df7845bdd82c95fd59dd8d2b
Successfully built docopt
Installing collected packages: urllib3, idna, chardet, certifi, requests, docopt, coverage, coveralls
Successfully installed certifi-2020.12.5 chardet-4.0.0 coverage-5.3.1 coveralls-3.0.0 docopt-0.6.2 idna-2.10 requests-2.25.1 urllib3-1.26.2
Missing .coveralls.yml file. Using only env variables.
Testing coveralls-python...
{"source_files": [], "git": {"branch": "run-coveralls-in-debug-mode", "head": {"id": "746233dd8b1ad0854d83f88cff871f34ffd1f4ed", "author_name": "J\u00fcrgen Gmach", "author_email": "juergen.gmach@googlemail.com", "committer_name": "J\u00fcrgen Gmach", "committer_email": "juergen.gmach@googlemail.com", "message": "Run coveralls integration in debug mode"}, "remotes": [{"name": "origin", "url": "https://github.com/morepath/morepath_rest_dump_load"}]}, "service_job_id": null, "repo_token": "[secure]", "service_name": "github", "service_number": "511454775", "config_file": ".coveragerc"}
==
Reporting 0 files
==

The corresponding coveralls.io page is https://coveralls.io/github/morepath/morepath_rest_dump_load which still shows "Here are some quick instructions for this Python project..." and not showing any successful report.

@solancer Thanks so much for your effort. Unfortunately, you also seem to have activated the debug mode, which means, it is not tried to send data to coveralls, and thus the build succeeds, but without reporting :-/

[afinetooth]

@jugmac00 yes, the issue is that no source files are being sent.

Per the JSON in the debug output, the source_files hash is empty ([]):

{"source_files": [] [...] }

This means you'll need to configure your project to pass your coverage report to your python-based coveralls integration, so it can convert the report into JSON and POST it to the coveralls API.

As an example, here are instructions for coveralls-python:
https://coveralls-python.readthedocs.io/en/latest/usage/configuration.html#configuration

[jugmac00]

@afinetooth Thanks for you suggestion.

I have read the documentation but found no clue what I should change.

See, I have 20 repositories with the almost exact same configuration, 17 work, 3 fail.

I just copied the 100% exact same configuration (both gh action and tox config) from https://github.com/morepath/morepath_sqlalchemy and created a PR for morepath/morepath_rest_dump_load#15 - the first one works, the latter one fails.

I cannot imagine this is a configuration issue.

[afinetooth]

Hi @jugmac00 I still think the underlying issue here is that the project is not sending any source files to the coveralls API (according to that debug output).

{"source_files": [], "git": {"branch": "run-coveralls-in-debug-mode", [...]

"Configuration" was probably not the clearest description of what needs to be done. What the empty source_files array means, most likely, is that for some reason the coveralls-python integration is not finding the coverage report your code is generating. (Thus, not populating source_files in the JSON.)

coveralls-python uses a default location where it looks for your coverage report. If it's not finding it, it means your project might be failing to generate one, or that it's saving it someplace besides the standard location.

This documentation gives the config flag to tell coveralls-python where your coverage report is:

coveralls --rcfile=/path/to/coveragerc

You'll want to find out:

• If your project is indeed generating a coverage report
• Where it's saving that report
• If it's in a format that coveralls-python can handle

Once you do, you can (hopefully) use the flag above to pass the right location.

Here are two things to try in the meantime to help get you there:

1. Run one of your successful projects in debug mode. This will validate we have the right issue by showing a full source_files array in the debug output.
2. If that works, find out where the successful project is storing its coverage report; then see if you can determine any difference in the non-working project as to where it's storing its report.

[pbelskiy]

So, any update here? I've got this error on PR, but on master not.

pbelskiy/aiojenkins#23

[afinetooth]

@pbelskiy can I pls ask you to create a new issue with your specific issue, coveralls repo URL, and build URL(s) that are affected? I assume you're getting a 422 error, but I'm thrown off by the history here and the number of users / different possible scenarios. Feel free to refer to this issue or any relevant comment here. Thanks.

[hartwork]

Ran into this myself today (in context of asyncee/django-easy-select2#96); --service=github proved to work around the issue. To me that means that likely auto-detection of GitHub Actions has issues.

I have also seen coveralls crash when there is no GITHUB_REF in the environment, which happens easily when coveralls is invoked by tox. I have created a dedicated ticket #1607 for that now.

[stephane-caron]

Had this problem again today. The following workaround fixed it for me, sharing it for future travelers:

• Sign into Coveralls via GitHub
• Go to Add Repos
• Un-tick then re-tick the failing project
• Re-run the failing CI jobs

Here is the corresponding workflow: main.yml.