Capture coverage failing when using docker #35
Comments
|
To be completely clear. Once SonarCube is upgraded to within my CI/CD platform. SonarQube is run on a VM. Jenkins provides the |
|
Just in case it is helpful or relevant, the development container is defined as follows: FROM golang:1.9
RUN apt-get update && apt-get install -y --no-install-recommends \
gettext-base \
zip \
&& rm -rf /var/lib/apt/lists/*
ENV YAML_BIN_VERSION 1.13.1
ENV JQ_BIN_VERSION 1.5
RUN wget -O /usr/local/bin/jq "https://github.com/stedolan/jq/releases/download/jq-${JQ_BIN_VERSION}/jq-linux64" \
&& chmod 755 /usr/local/bin/jq \
&& wget -O /usr/local/bin/yaml "https://github.com/mikefarah/yaml/releases/download/${YAML_BIN_VERSION}/yaml_linux_amd64" \
&& chmod 755 /usr/local/bin/yaml
ARG DEVELOPER
RUN git config --global user.name "${DEVELOPER}" \
&& git config --global user.email "${DEVELOPER}@cisco.com" \
&& git config --list
ENV GLIDE_VERSION v0.12.3
RUN go get github.com/alecthomas/gometalinter \
&& gometalinter --install --vendor \
&& go get golang.org/x/tools/cmd/goimports \
&& go get github.com/mitchellh/gox \
&& go get github.com/davecheney/godoc2md \
&& go get github.com/axw/gocov/gocov \
&& go get github.com/t-yuki/gocov-xml \
&& go get github.com/tebeka/go2xunit \
&& go get github.com/go-swagger/go-swagger/cmd/swagger \
&& curl -Lo /tmp/glide.tgz https://github.com/Masterminds/glide/releases/download/$GLIDE_VERSION/glide-$GLIDE_VERSION-linux-amd64.tar.gz \
&& tar -C /usr/bin -xzf /tmp/glide.tgz --strip=1 linux-amd64/glide \
&& rm /tmp/glide.tgz
ENV CGO_ENABLED 0
ENV GOPATH /go:/cicdtest |
@thibaultfalque we need to retrieve the |
|
We just added the code to prevent files in the exclusion path to be analyzed. The next RC will allow to check that it fixes your problem. |
|
Please check release 1.2.11-RC8. |
|
@kenjones-cisco could you check with the last release and close the issue if the problem is resolved ? |
|
Issues 1 and 2 still exist Even though I have the following defined: And when the coverage file is generated the filename inside the coverage.xml shows up as |
|
@thibaultfalque we need to check if sonar.exclusions is properly honored in each sensor. Regarding the second issue, I have really no idea how to fix it since I am not a regular docker user. Any idea would be welcome. |
|
@danielleberre I wondering if there is an assumption about what file paths look like for a golang code base. Most have their source control within github which results in a base import package of As such if your code base has multiple packages you will have to provide the full import path. Therefore when you build, test, etc. The code needs to appear as if it is in Therefore when mounting the code base into a docker container, it will be mounted on a path that looks like As such I have tried running sonar-scanner inside a container with my source code mounted in a similar way with the sonar-project.properties file at the root of my project, but even then it seems to be unable to match the filename path to the actual files. It seems as if sonar-scanner expects to be run from |
|
You are right, we crawl the top level directory to find the resources. Maybe we do not do it properly because we have to take into account sonar.exclusion manually. This is our first SonarQube plugin, we looked around how the other plugin proceed, and did not notice any specific way to do that crawling. Let us sleep on it, we will try to find a way to solve your problem. Docker is getting so popular that it is probably an issue for other users too. |
|
Hi @kenjones-cisco. Try putting "." on the "Sources location" in build step (we use TeamCity). On the sonar-scanner side, I put this in my sonar-project.properties: sonar.sources=. This works for me (from a Docker container). Updating the plugin to try again via TC runner. |
|
Thanks @introllo but my configuration is below and that is what I use as well. |
|
Hmmm. Pretty much the same as mine: Let me dbl-check my setup. |
|
Apologies @kenjones-cisco . I checked my notes (its been awhile) and see that we use vagrant provisioned VMs for this as we can't run our test code inside of a Docker container. So, in my case, I'm running sonar-scanner in an Ubuntu VM. Thx. |
|
Ok, I asked for advices to take into account sonar.exclusion properly: |
|
I have fix the problem of sonar.exclusion in this commit d5c2d02 |
|
@kenjones-cisco could you test with the last release ? |
|
Hello, I also have this issue with the latest release (v1.2.11-rc13). This is what I have: And then no coverage in Sonarqube. And this is the beginning of the generated |
|
Could you post your coverage report? |
danielleberre
changed the title
|
Yes of course Thank you! |
|
/go/src/github.enterprise.com/enterprise/AuthenticationService-Go/handlers/handler.go exists ? Do you use docker ? What is your hierarchy of files ? What is the path when sonar-scanner is run ? |
|
This is the Dockerfile of the image where I create the sonar reports files: Then I use the following commands to get the reports: And then I execute directly on the jenkins agent: Containing the So in Sonarqube I have the lint reports but not the coverage informations. Thank you for your help!! |
|
I am not a daily user of docker. Your docker container have access on /go/... ? |
|
AFAI understand what's going on, the test and coverage reports are run on docker while the sonar-scanner is run directly. As such, the paths in the coverage report do not match the paths in the current filesystem. If we can rebuild the correct file path knowing the prefix of the path in the docker image, we could probably add a specific option for docker users to fix this issue. But we need some help from regular users of docker to do that properly. |
|
Yes that's also what I suspected. The project path is generated since I'm building a Jenkins Pipeline. What did you mean by "add a specific option"? Add a parameter when executing |
|
There is an opened pull request on When using instead of: So the I will investigate this on monday and I will give you some feedback. |
|
I could not wait until monday, I just tested it and it works, I now have the coverage informations in sonarqube. But I guess I will have to use the Thanks for your help! |
|
That's a good news! Maybe we should comment in gocov's PR that it would be helpful in our case? |
Provide the workaround found for #35 until the main gocov-xml supports the `-pwd` option.
|
Dear all, I updated the README.md page to mention the workaround for docker users. @kenjones-cisco could you check that the workaround solves your problem as well? Thanks again @denouche for providing us that workaround. |
|
Have only recently been able to return to this issue. Have deployed updated version and now see my Go coverage metrics in my Sonar server. I tested this combination: TeamCity 10.0.4 & 2017.1.2 running as Docker containers (both build server and build agent) Thanks for all the hard work! |
Description
When attempting to publish
coverage.xml,report.xml, andtest.xmlto SonarQube after generating the individual xml files, I'm seeing the following happen.sonar.exclusionsentry, as all the files in my.glide/**andvendor/**are being processed.unable to create InputFile object: gitscm.cisco.com/ccdev/cicdtest/version/info.gojava.lang.StringIndexOutOfBoundsException: String index out of range: -1Steps to Reproduce
docker build -t cicdtest_sonar -f Dockerfile.cd .Content of your sonar-project.properties
https://gist.github.com/kenjones-cisco/8b5f55f4f5197d3ddfba66142aeefbc1#file-sonar-project-properties
It will help us to check how you configured the plugin.
Log of sonar-scanner related to the plugin
https://gist.github.com/kenjones-cisco/8b5f55f4f5197d3ddfba66142aeefbc1#file-scanner-log
The plugin logs major events, including which report files have been found and analyzed.
Versions
SonarQube Server
sonar-scanner
go version
go version go1.9 linux/amd64gometalinter
c039b9f1fbff9a38595e19e9d97474ed57270cb8Last commit as of yesterdayOS
All
go,gometalinter, etc. are run inside of Docker containersVersions of your sonar installation (server, sonar-scanner), gometalinter, go and OS.
Additional Information
All *.xml are in the same gist as the other files:
https://gist.github.com/kenjones-cisco/8b5f55f4f5197d3ddfba66142aeefbc1
Seems similar to #14
Sample reports may be necessary to reproduce the issue. Please attach if possible reports files (report.xml, coverage.xml, test.xml) corresponding to the feature which is not working as expected.
The text was updated successfully, but these errors were encountered: