Cannot lint charts in pipeline

[g0blin79]

This probably is not a bug, but only a problem I have and I don't understand why. I'm asking an help.

I start using ct for our company chart repository.
Our charts are into a "stable" directory at the root of the repo.

If I try to lint my chart in local environment I do it with success.
But when I use the same command in my CI Environment (Gitlab Pipeline) I had this error:

Linting charts...
Using config file:  .bmeme/config.yaml
------------------------------------------------------------------------------------------------------------------------
 Configuration
------------------------------------------------------------------------------------------------------------------------
Remote: origin
TargetBranch: master
BuildId: 
LintConf: /etc/ct/lintconf.yaml
ChartYamlSchema: /etc/ct/chart_schema.yaml
ValidateMaintainers: false
ValidateChartSchema: true
ValidateYaml: true
CheckVersionIncrement: true
ProcessAllCharts: false
Charts: []
ChartRepos: []
ChartDirs: [stable]
ExcludedCharts: []
HelmExtraArgs: --timeout 600
HelmRepoExtraArgs: []
Debug: false
Upgrade: false
SkipMissingValues: false
Namespace: 
ReleaseLabel: 
------------------------------------------------------------------------------------------------------------------------
Error linting charts: Error identifying charts to process: Error running process: exit status 128
------------------------------------------------------------------------------------------------------------------------
No chart changes detected.
------------------------------------------------------------------------------------------------------------------------

Some infos:

• I'm using this yaml configuration:

target-branch: master
chart-dirs:
  - stable
helm-extra-args: --timeout 600
validate-maintainers: false

• In local development environment I run lint command using docker image, in this way:
  `docker run -v $(pwd):/workdir gcr.io/kubernetes-charts-ci/test-image:v3.4.0 sh -c "ct lint --config .bmeme/config.yaml"

• In pipeline, I tried running this command both native and with docker-in-docker: in both case I had the same error.

If I try to run same command with --debug option, I noticed the following differences about git commands:

local execution:

------------------------------------------------------------------------------------------------------------------------
>>> git rev-parse --is-inside-work-tree
>>> git merge-base origin/master HEAD
>>> git diff --find-renames --name-only f34b8dae2977eb310860a145f8b429567128ebc7 -- stable

------------------------------------------------------------------------------------------------------------------------

pipeline execution

------------------------------------------------------------------------------------------------------------------------
>>> git rev-parse --is-inside-work-tree
>>> git merge-base origin/master HEAD
Error linting charts: Error identifying charts to process: Error running process: exit status 128
------------------------------------------------------------------------------------------------------------------------

Can you help me to debug this problem?

[lenardchristopher]

Hi @g0blin79. I started experiencing the same issue today. It is not specific to my computer/user and abruptly started happening about 6 hours ago.

Did you ever find a fix?

[coreywagehoft]

Having a similar problem when running inside of a GitHub action. Here is the output.

/usr/bin/docker run --name <redacted> --label f6cdeb --workdir /github/workspace --rm -e INPUT_ARGS -e HOME -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e RUNNER_OS -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e GITHUB_ACTIONS=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/helm-charts/helm-charts":"/github/workspace" <redacted> lint --target-branch develop --chart-dirs . --chart-yaml-schema .github/actions/chart-testing-action/etc/chart_schema.yaml --lint-conf .github/actions/chart-testing-action/etc/lintconf.yaml --validate-maintainers=false --debug
Linting charts...
------------------------------------------------------------------------------------------------------------------------
 Configuration
------------------------------------------------------------------------------------------------------------------------
Remote: origin
TargetBranch: develop
BuildId:
LintConf: .github/actions/chart-testing-action/etc/lintconf.yaml
ChartYamlSchema: .github/actions/chart-testing-action/etc/chart_schema.yaml
ValidateMaintainers: false
ValidateChartSchema: true
ValidateYaml: true
CheckVersionIncrement: true
ProcessAllCharts: false
Charts: []
ChartRepos: []
ChartDirs: [.]
ExcludedCharts: []
HelmExtraArgs: 
HelmRepoExtraArgs: []
Debug: true
Upgrade: false
SkipMissingValues: false
Namespace: 
ReleaseLabel: 
------------------------------------------------------------------------------------------------------------------------
>>> git rev-parse --is-inside-work-tree
>>> git merge-base origin/develop HEAD
Error linting charts: Error identifying charts to process: Error running process: exit status 128
------------------------------------------------------------------------------------------------------------------------
No chart changes detected.
------------------------------------------------------------------------------------------------------------------------

[unguiculus]

git merge-base failed in these cases. It probably complained because the given remote branch doesn't exist. Please verify the remote is there and fetched.

[coreywagehoft]

git merge-base failed in these cases. It probably complained because the given remote branch doesn't exist. Please verify the remote is there and fetched.

Thanks for the insight on this @unguiculus. This turned my attention to the actions/checkout action that gets our helm charts to lint. Turns out yesterday a breaking change release came out and we were previously targeting the master branch.

https://github.com/actions/checkout/releases/tag/v2-beta

If you are using chart-testing with GitHub actions make sure to only use actions/checkout:v1.2.0 for use with helm chart testing & linting.

[evannorstrand]

I can confirm this issue happens in gitlab as well.

ct lint --debug --remote origin --target-branch master --chart-dirs stable --chart-repos xxx=yyy

[cprovencher]

@unguiculus I added a "git fetch" per your recommendation. Now I no longer get a 128 exit code, but instead an exit code 1 with no logs at all. Any idea how to debug this? When would git merge-base return 1 with no logs? GIT_TRACE=1 didn't help either.

------------------------------------------------------------------------------------------------------------------------
>>> git rev-parse --is-inside-work-tree
>>> git merge-base origin/master HEAD
Error linting charts: Error identifying charts to process: Error running process: exit status 1
------------------------------------------------------------------------------------------------------------------------
No chart changes detected.
------------------------------------------------------------------------------------------------------------------------

[joejulian]

This issue is caused by your SCM creating a shallow clone. This leads to two git errors.

error 128

This is because there is no master branch with the shallow clone that most scm's create.

$ git clone git@github.com:example/project --branch pullrequest --depth 1 work
$ cd work
$ git merge-base origin/master HEAD ; echo $?
fatal: Not a valid object name origin/master
128

As it says, origin/master doesn't exist.

$ git branch -a                                                                 
* pullrequest
  remotes/origin/pullrequest

As has been discovered, you can overcome that by pulling in master:

git fetch origin '+refs/heads/master:refs/remotes/origin/master'

error 1

merge-base will still return errno 1 because this branch of your shallow clone is what's known as "grafted" meaning as far as git's concerned, it has no parents

$ git reflog show --decorate
cf5a1df (grafted, HEAD -> pullrequest, origin/pullrequest) HEAD@{0}: clone: from git@github.com:example/project

To overcome that obsticle, you'll need to "unshallow" the branch checked out by CI

git fetch --unshallow

This will now allow merge-base to work

$ git merge-base origin/master HEAD
2f7422522c60f2bc4325362d7e4b5996c4b7720f

[unguiculus]

@coreywagehoft actions/checkout@v2 does a shallow checkout by default. This can be reconfigured as follows

uses: actions/checkout@v2
with:
  fetch-depth: "0"

[lsowen]

For someone else running into the exit 128 error in Jenkins, you have to disable shallow cloning:

For some reason, to actually disable shallow cloning you must set Shallow clone depth to 0 (just having Shallow clone unchecked doesn't actually fix anything.)

[dminca]

@coreywagehoft actions/checkout@v2 does a shallow checkout by default. This can be reconfigured as follows

uses: actions/checkout@v2
with:
  fetch-depth: "0"

This worked for me in GitLab CI as well, by setting:

  variables:
    GIT_DEPTH: "0"

[pathob]

Took me some time to find the solution so I want to add it here:

I'm also using Jenkins and in my case I had to add the following ref spec for the Git(Lab) branch source (so the main branch as the target branch is also available locally):

+refs/heads/main:refs/remotes/origin/main

Also related to helm/chart-testing-action#25