CodeCov Upload Error Message

[PlagueHO]

An error has recently started appearing during upload of the code coverage results to CodeCov.io. This error appears to not actually prevent upload or fail the build, so is not critical.

The error occurs when calling the codecov Python script from this line of the CodeCovIo module.

The output of the codecov python can be seen in the CodeCovUpload.log artifact stored in the AppVeyor build artifacts. This shows no errors:

      _____          _
     / ____|        | |
    | |     ___   __| | ___  ___ _____   __
    | |    / _ \ / _  |/ _ \/ __/ _ \ \ / /
    | |___| (_) | (_| |  __/ (_| (_) \ V /
     \_____\___/ \____|\___|\___\___/ \_/
                                    v2.0.10

�[90m==>�[0m Detecting CI provider
    AppVeyor Detected
�[90m==>�[0m Preparing upload
    Error running `cd 'C:\projects\xnetworking' && git ls-files`: None
�[31mXX>�[0m Skip processing gcov
�[90m==>�[0m Collecting reports
�[31mXX>�[0m Searching for reports disabled
    Targeting specific files
    �[32m+�[0m C:\Users\appveyor\AppData\Local\Temp\1\codeCov.json bytes=1015201
�[90m==>�[0m Uploading
    .url https://codecov.io
    .query service=appveyor&commit=287671eeb0117d7374e8102aaf6c329c9d9a7607&package=py2.0.10&branch=dev&build=bsjueksk050yjjm6&job=PowerShell%2Fxnetworking%2F3.1.786.0&pr=291&slug=PowerShell%2FxNetworking
    Pinging Codecov...
    Uploading to S3...
�[92m    https://codecov.io/github/PowerShell/xNetworking/commit/287671eeb0117d7374e8102aaf6c329c9d9a7607�[0m

It also shows the artifact being uploaded. The result can also be seen in CodeCov.io.

The error seems to occur or be outputted into the error stream somewhere in this Python code:
https://github.com/codecov/codecov-python/blob/master/codecov/__init__.py

The last changes to this code was made 5 days ago. looking back through AppVeyor logs, this error started appearing at least 4 days ago but not more than 6 days ago - so this would track with the recent changes made to the CodeCov Python.

This is occurring in all repos that process and upload code coverage.

[johlju]

Submitted an issue codecov/codecov-python#129.

Submitted to the wrong repo at first, deleted that comment.

[PlagueHO]

Great job @johlju - sorry I haven't had time to look into this (have had to do actual "work work" over the break - lots of work on creating security layers around CosmosDb and API Management). This month might be a bit slow on the DSC side from me unfortunately due to too many other commitments 😢

[johlju]

@PlagueHO No worries. Sorry to hear you had a lot of work over the holidays :/ This month will be a bit slower for me as well, after this week at least. Still on vacation, back at work on the 8th and have a deadline at end of January.

[ramlongcat]

i have the same issue.
im using AppVeyor and the CodeCov uploader via Powershell, and a build that used to work in December is now failing with that exact error message because of this upload issue :(
FYI in my experience the error started to appear around December 15th.
Let me know if i can provide anything else to help you guys.

[ramlongcat]

first time i look at your code but IMHO the only meaningful change i see around those dates is that one.
i'm not sure -- but i think the issue could be the content type on line 730 hasn't been updated.
i'd try to change 'Content-Type': 'text/plain' to 'Content-Type': 'text/plain; charset=utf-8' since
the default for this content type is us-ascii according to the W3C spec.
@stevepeak do you mind taking a look at this when you can ? that'd be awesome.

[johlju]

So a colleague visually debugged this for me. It seems that it could be this row that is problem.

https://github.com/codecov/codecov-python/blob/bc70732bc1e7fef2167afaa25f84a5d25af947f3/codecov/__init__.py#L567

It is not allowed to run && in PowerShell, and could it be that the python code is run in a PowerShell session instead of a Command Prompt (we do kick the .exe from PowerShell). Do we get this error if we run this from a Command Prompt?

[ramlongcat]

interesting ! now that you're saying this i compared my old/working codecov output:

==> Detecting CI provider
    AppVeyor Detected
==> Preparing upload
XX> Skip processing gcov
==> Collecting reports

with the current/non-working one:

==> Detecting CI provider
    AppVeyor Detected
==> Preparing upload
    Error running `cd 'C:\projects\parkserver' && git ls-files`: None
XX> Skip processing gcov
==> Collecting reports

...and it seems you're right (or close at least).

[ramlongcat]

but that change occurred more than 2 years ago ?

[ramlongcat]

maybe found something:
that same line is using root and it has been changed here using root = quote(root) around those dates.

[johlju]

It was just a possible issue. Our thought was that something changed in Window (how Python is calling this &&), because the error in the issue description says

Error running `cd 'C:\projects\xnetworking' && git ls-files`: None

But the 'root' row you found could be it too.

[PlagueHO]

That is all some interesting info. I did confirm that this issue also occurs when executing locally (I had to trick the machine into pretending to be AppVeyor by defining some environment variables that the CodeCov.io looks for). But as this issue seems to be mostly harmless I didn't get much further. But it would be great to resolve. I think @johlju's suggestion that this could be caused by running from PowerShell instead of CMD could be a big possibility.

But what might have changed was the version of Python and the behavior of the "try_to_run()" function. I'm thinking that this error may have always occurred, it is just the error never got pushed into the error stream so we never saw it. But I am just guessing - because I can't see any other change that could have caused this.

[ramlongcat]

FYI i manually patched the line i was talking about right after doing pip install codecov to be a comment and my coverage uploads now work, no more error message. thanks for your support ! hope this helps

[johlju]

@PlagueHO What do think about switching to this .exe report uploader for Codecov. It seems that the prerequisites 'Chocolatey' is installed on the build worker.

choco install codecov
.\codecov.exe -f <path to coverage report> -t <Codecov upload token>

I just saw it, so I haven't done anything more than read the README.md.

[johlju]

The problem seems to escalate, now more errors are shown. The errors still does not affect functionality (yet).

https://ci.appveyor.com/project/PowerShell/xpowershellexecutionpolicy/build/1.0.66.0#L279

[PlagueHO]

Good idea, but it looks like the CodeCov.exe uses the same Python code to upload the report. So we're seeing the same issue. Such a pain!

[johlju]

Oh, I hoped it did not do that :/

[johlju]

We have the Ruby one, and the Pearl one as well. Not can those use the Pyhton script? 🤔

[PlagueHO]

The Ruby one looks like it might work - Choco can be used to install Ruby pretty easily. Great ideas!

[PlagueHO]

Oh wait - I missed the message from @ramlongcat - great idea there. Might be worth looking into more. Thanks @ramlongcat and sorry I didn't see your post. What do you think @johlju

[johlju]

I’m all for the idea, if it solves the problem. :) Fixing the problem in the script at source I think would be the best option. Parsing the script at our end could maybe be problematic if changes occurs at source?

[johlju]

Looks like codecov.exe actually solves this. Sent in a PR #223