Cannot build a project imported with the --nested-repo option

[svilenvul]

Hello,

We need your help with building a project using the the maven-quarkus-buildpack. The project is imported with the --nested-repo option (adding support for nested repositories in GitLab).

Currently our build is failing in one of the last steps promote-helm-release.

We tried to debug this today and found following conflicts:

1. In the build step step-promote-changelog the pipeline was not able to find our Helm chart. We realized this by the fact that there was a log message no charts which we tracked to this script:

   jx/.github/workflows/jenkins-x/changelog.sh

   Line 13 in 19684a1

   echo no charts

2. This was resulting in the fact that JX was not updating the version of the Helm Chart located in the charts folder with the name my-organisation-my-suborganisation-my-service-name (concatenated name with dashes). This chart folder is created during the project import with the --nested-repo option
3. Because of that the step-promote-helm-release step was failing with the error

error: failed to create chart release in dir charts/my-organisation-my-suborganisation-my-service-name: failed to publish: failed to run 'curl --fail -sS -u ....:..... --data-binary @my-organisation-my-suborganisation-my-service-name-1.0.3.tgz http://jenkins-x-chartmuseum.jx.svc.cluster.local:8080/api/charts' command in directory 'charts/my-organisation-my-suborganisation-my-service-name', output: 'curl: (22) The requested URL returned error: 500'     

4. Looking on the .jx/variables.sh generated during the build we saw that the REPO_NAME variable was not matching the directory where the chart is located. The variable had the value my-organisation/my-suborganisation/my-service-name (concatenated with forward slash)

We are providing here the output of the build logs and the .jx/variables.sh

• .jx/variables.sh

# generated by: jx gitops variables
export APP_NAME='my-organisation/my-suborganisation/my-service-name'
export BRANCH_NAME='main'
export BUILD_NUMBER='3'
export DOCKERFILE_PATH='Dockerfile'
export DOCKER_REGISTRY='registry.domain.subdomain.com'
export DOCKER_REGISTRY_ORG='company'
export DOMAIN='company-name.cloud'
export GIT_BRANCH='main'
export JENKINS_X_URL='http://dashboard.jx.company-name.cloud/smart-ix/my-organisation/my-suborganisation/my-service-name/main/3'
export JX_CHART_REPOSITORY='http://jenkins-x-chartmuseum.jx.svc.cluster.local:8080/'
export KANIKO_FLAGS='--skip-unused-stages --snapshotMode=redo'
export MINK_AS='tekton-bot'
export MINK_IMAGE='registry.domain.subdomain.com/smart-ix/my-organisation/my-suborganisation/my-service-name:1.0.3'
export NAMESPACE_SUB_DOMAIN='.jx.'
export PIPELINE_KIND='release'
export PUSH_CONTAINER_REGISTRY='registry.domain.subdomain.com'
export REPO_NAME='my-organisation/my-suborganisation/my-service-name'
export REPO_OWNER='company'
export VERSION='1.0.3'

• build logs

step-promote-changelog
no charts
step-promote-changelog
Using batch mode as inside a pipeline
step-promote-changelog
about to run: git for-each-ref --sort=-creatordate --format=%(objectname)%00%(refname:short) --count=2 refs/tags in dir .
step-promote-changelog
3397d754ea7b95dad5e88192eb1d7ef92dbeb3e1v1.0.3
step-promote-changelog
4e19b98f5bb8c957cfc2866a86e05171fa2a85dev1.0.2
step-promote-changelog
about to run: git rev-list -n 1 4e19b98f5bb8c957cfc2866a86e05171fa2a85de in dir .
step-promote-changelog
49432b62efd436b8b7763209a34c1e12f62afdda
step-promote-changelog
about to run: git for-each-ref --sort=-creatordate --format=%(objectname)%00%(refname:short) --count=1 refs/tags in dir .
step-promote-changelog
3397d754ea7b95dad5e88192eb1d7ef92dbeb3e1v1.0.3
step-promote-changelog
about to run: git rev-list -n 1 3397d754ea7b95dad5e88192eb1d7ef92dbeb3e1 in dir .
step-promote-changelog
062df9666eb7660f2634631db62c99aa0ce6f19f
step-promote-changelog
Generating change log from git ref 49432b62efd436b8b7763209a34c1e12f62afdda => 062df9666eb7660f2634631db62c99aa0ce6f19f
step-promote-changelog
WARNING: no $GIT_SECRET_MOUNT_PATH environment variable set
step-promote-changelog
about to run: git clone https://domain.subdomain.com/company-name-cicd-platform/environment-production.git /tmp/jx-git-769918337 in dir /tmp
step-promote-changelog
Cloning into '/tmp/jx-git-769918337'...
step-promote-changelog
Can't find any issue tracker setting; defaulting to git provider: gitlab
step-promote-changelog
Finding issues in commit messages using git format
step-promote-changelog
about to run: git tag --list v1.0.3 in dir .
step-promote-changelog
v1.0.3
step-promote-changelog
about to run: git tag --list vv1.0.3 in dir .
step-promote-changelog
WARNING: Failed to create the release for company-name/my-service-name: Not Found
step-promote-changelog
The following paths are ignored by one of your .gitignore files:
step-promote-changelog
target
step-promote-changelog
hint: Use -f if you really want to add them.
step-promote-changelog
hint: Turn this message off by running
step-promote-changelog
hint: "git config advice.addIgnoredFile false"
step-promote-changelog
[main 7f7889a] chore: release 1.0.3
step-promote-changelog
2 files changed, 53 insertions(+), 1 deletion(-)
step-promote-changelog
create mode 100644 pom.xml.versionsBackup
step-promote-changelog
Updated tag 'v1.0.3' (was 3397d75)
step-promote-changelog
To https://domain.subdomain.com/company-name/my-organisation/my-suborganisation/my-service-name.git
step-promote-changelog
+ 3397d75...8e3eb35 v1.0.3 -> v1.0.3 (forced update)
step-promote-helm-release
WARNING: no $GIT_SECRET_MOUNT_PATH environment variable set
step-promote-helm-release
releasing chart my-organisation-my-suborganisation-my-service-name
step-promote-helm-release
Adding repository for dependency mongodb
step-promote-helm-release
Adding repository for dependency company-name-template-company-name-charts
step-promote-helm-release
error: failed to create chart release in dir charts/my-organisation-my-suborganisation-my-service-name: failed to publish: failed to run 'curl --fail -sS -u admin:0eBw.1kOFdwgMcWFWyUCh9RkyYJdGbM+UqyyEQiP2 --data-binary @my-organisation-my-suborganisation-my-service-name-1.0.3.tgz http://jenkins-x-chartmuseum.jx.svc.cluster.local:8080/api/charts' command in directory 'charts/my-organisation-my-suborganisation-my-service-name', output: 'curl: (22) The requested URL returned error: 500'

• output form jx version command

jx version
version: 3.2.253

cc @ankitm123 and @marziman

[ankitm123]

/assign

[marziman]

@ankitm123 @rawlingsj @jstrachan
we are blocked at the last step by this helm promote release step, which is connected to nested projects. Can you please help here, cause it will allow us to use JXv3 fully. Currently on hold because of this issue.

Many thanks!

[ankitm123]

Replace:

- name: promote-changelog
  resources: {}

with (basically replace REPO_NAME with subgroup-repo)

- image: ghcr.io/jenkins-x/jx-changelog:0.2.0
          name: promote-changelog
          resources: {}
          script: |
            #!/usr/bin/env sh
            source .jx/variables.sh
            cat .jx/variables.sh
            if [ -d "charts/$REPO_NAME" ]; then
            jx gitops yset -p version -v "$VERSION" -f ./charts/$REPO_NAME/Chart.yaml
            jx gitops yset -p appVersion -v "$VERSION" -f ./charts/$REPO_NAME/Chart.yaml
            jx gitops yset -p 'image.repository' -v $DOCKER_REGISTRY/$DOCKER_REGISTRY_ORG/$APP_NAME -f ./charts/$REPO_NAME/values.yaml
            jx gitops yset -p 'image.tag' -v "$VERSION" -f ./charts/$REPO_NAME/values.yaml;
            else echo no charts; fi
            git add * || true
            git commit -a -m "chore: release $VERSION" --allow-empty
            git tag -fa v$VERSION -m "Release version $VERSION"
            git push --force origin v$VERSION
            jx changelog create --version v${VERSION}

For example, for my test I have export REPO_NAME=cluster-print-go

The latest jx version has the fixes.

[spaily]

@ankitm123 Can you please check if #8453 is a related issue?