Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker references in gitlab-ci.yml split on multiple lines is not updated correctly #11595

Open
martin-sucha opened this issue Sep 6, 2021 · 5 comments
Labels
manager:gitlabci GitLab CI config files priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others type:bug Bug fix of existing functionality

Comments

@martin-sucha
Copy link

How are you running Renovate?

Self-hosted

Please select which platform you are using if self-hosting.

GitLab self-hosted

If you're self-hosting Renovate, tell us what version of Renovate you run.

26.19.1

Describe the bug

When the Docker image reference in gitlab-ci.yml file is split into multiple lines in yaml file, renovate does
not update the whole reference, only the part on the first line.

Screenshot from 2021-09-06 09-56-23

I would expect renovate to either fail with an error because it can't update multiline references or update the whole reference (does not need to stay split on multiple lines).

Relevant debug logs

Logs
INFO: Repository started (repository=search-team/gonuts/conveyance-store)
       "renovateVersion": "26.19.1"
 INFO: Dependency extraction complete (repository=search-team/gonuts/conveyance-store)
       "baseBranch": "master",
       "stats": {
         "managers": {
           "docker-compose": {"fileCount": 6, "depCount": 7},
           "dockerfile": {"fileCount": 18, "depCount": 20},
           "gitlabci": {"fileCount": 7, "depCount": 20},
           "gomod": {"fileCount": 2, "depCount": 55},
           "pre-commit": {"fileCount": 1, "depCount": 3}
         },
         "total": {"fileCount": 34, "depCount": 105}
       }
 INFO: Ignoring upgrade collision (repository=search-team/gonuts/conveyance-store, packageFile=.gitlab/ci/test.yml, branch=renovate/all)
       "manager": "gitlabci",
       "depName": "scylladb/scylla-enterprise",
       "currentValue": "2021.1.2",
       "previousNewValue": "2021.1.5",
       "thisNewValue": "2021.1.2"
 INFO: Ignoring upgrade collision (repository=search-team/gonuts/conveyance-store, packageFile=.gitlab/ci/test.yml, branch=renovate/all)
       "manager": "gitlabci",
       "depName": "scylladb/scylla-enterprise",
       "currentValue": "2021.1.2",
       "previousNewValue": "2021.1.5",
       "thisNewValue": "2021.1.2"
 INFO: Ignoring upgrade collision (repository=search-team/gonuts/conveyance-store, packageFile=.gitlab/ci/lint.yml, branch=renovate/all)
       "manager": "gitlabci",
       "depName": "renovate/renovate",
       "currentValue": "25.64-slim",
       "previousNewValue": "26.19-slim",
       "thisNewValue": "25.64-slim"
 INFO: Repository finished (repository=search-team/gonuts/conveyance-store)
       "durationMs": 143437

Have you created a minimal reproduction repository?

No reproduction repository

@martin-sucha martin-sucha added priority-5-triage status:requirements Full requirements are not yet known, so implementation should not be started type:bug Bug fix of existing functionality labels Sep 6, 2021
@viceice viceice added manager:gitlabci GitLab CI config files auto:reproduction A minimal reproduction is necessary to proceed labels Sep 6, 2021
@github-actions
Copy link
Contributor

github-actions bot commented Sep 6, 2021

Hi there,

Help us by making a minimal reproduction repository.

Before we can start work on your issue we first need to know exactly what's causing the current behavior. A minimal reproduction helps us with this.

To get started, please read our guide on creating a minimal reproduction to understand what is needed.

We may close the issue if you (or someone else) have not provided a minimal reproduction within two weeks. If you need more time, or are stuck, please ask for help or more time in a comment.

Good luck,

The Renovate team

@martin-sucha
Copy link
Author

I've created a reproducer repo: https://github.com/kiwicom/renovate-11595

@viceice
Copy link
Member

viceice commented Sep 7, 2021

This looks pretty weird, is this supported by gitlab / yaml ?

@martin-sucha
Copy link
Author

Gitlab just parses yaml.

Splitting a string to multiple lines is a standard feature of yaml:

In a multi-line double-quoted scalar, line breaks are are subject to flow line folding, which discards any trailing white space characters. It is also possible to escape the line break character. In this case, the line break is excluded from the content, and the trailing white space characters are preserved. Combined with the ability to escape white space characters, this allows double-quoted lines to be broken at arbitrary positions.

Some examples are also at https://yaml-multiline.info/

So in

  services:
    - name: "alpine:3.12.8@\
              sha256:a296b4c6f6ee2b88f095b61e95c7ef4f51ba25598835b4978c9256d8c8ace48a"

the value for name key parses as alpine:3.12.8@sha256:a296b4c6f6ee2b88f095b61e95c7ef4f51ba25598835b4978c9256d8c8ace48a

We had the image reference split to multiple lines because of the line length. Obviously we worked around the issue by putting the image reference entirely on a single line. However, I wonder how renovate updates the yaml file as it had not replaced whole value for image key in the yaml, but has replaced the \ at the end of the line and I would have expected renovate to replace the entire yaml string.

@viceice viceice added reproduction:provided priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others and removed auto:reproduction A minimal reproduction is necessary to proceed priority-5-triage labels Sep 8, 2021
@viceice
Copy link
Member

viceice commented Sep 8, 2021

Ok, this will be very tricky to fix.

  • If we parse the file as yaml, we get right value, but string replace fails.
  • Parse with regex is difficult too, as the line break can be at arbitrary position.

I've an idea and will experiment a little.

@viceice viceice self-assigned this Sep 8, 2021
@viceice viceice removed their assignment Dec 30, 2022
@rarkins rarkins added status:ready and removed status:requirements Full requirements are not yet known, so implementation should not be started labels Apr 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
manager:gitlabci GitLab CI config files priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others type:bug Bug fix of existing functionality
Projects
None yet
Development

No branches or pull requests

3 participants