Renovate attempts to post 'Artifact update problem' with invalid content

[nfm]

Not sure if this is worth opening as an issue, but flagging here for discussion, and visibility in case it's useful context for anyone else.

We had an issue with our renovate setup. We run self-hosted renovate via the NPM package installed on top of our Docker image, and use GitLab. We were accidentally running renovate against an old version of our Docker image, which contained versions of both Ruby and Bundler that predated those used to generate our Gemfile.lock.

As a result, when renovate attempted to open a merge request to bump a gem version, it was hitting an artifact update problem trying to modify the Gemfile.lock.

Here's a (lightly redacted) snippet from the debug logs:

DEBUG: Creating PR (repository=repo/app, baseBranch=master, branch=renovate/pg-1.4.x)
       "prTitle": "Update dependency pg to \"~>1.4.5\""
DEBUG: Creating Merge Request: Update dependency pg to "~>1.4.5" (repository=repo/app, baseBranch=master, branch=renovate/pg-1.4.x)
 INFO: PR created (repository=repo/app, baseBranch=master, branch=renovate/pg-1.4.x)
       "pr": 8736,
       "prTitle": "Update dependency pg to \"~>1.4.5\""
DEBUG: addParticipants(pr=8736) (repository=repo/app, baseBranch=master, branch=renovate/pg-1.4.x)
DEBUG: Adding assignees 'assignee' to #8736 (repository=repo/app, baseBranch=master, branch=renovate/pg-1.4.x)
DEBUG: Added assignees (repository=repo/app, baseBranch=master, branch=renovate/pg-1.4.x)
       "assignees": ["assignee"]
DEBUG: Created Merge Request #8736 (repository=repo/app, baseBranch=master, branch=renovate/pg-1.4.x)
 WARN: artifactErrors (repository=repo/app, baseBranch=master, branch=renovate/pg-1.4.x)
       "artifactErrors": [
         {
           "lockFile": "Gemfile.lock",
           "stderr": "Fetching source index from https://gems.contribsys.com/\nFetching source index from https://gems.contribsys.com/\nFetching gem metadata from https://rubygems.org/.........\nFetching source index from https://rubygems.org/\n\nWarning: the running version of Bundler (2.1.4) is older than the version that created the lockfile (2.3.3). We suggest you to upgrade to the version that created the lockfile by running `gem install bundler:2.3.3`.\nCould not find gem 'Ruby\u0000 (~> 3.1.0.0)' in the local ruby installation.\nThe source contains 'Ruby\u0000' at: 2.7.6.219\n"
         }
       ]
DEBUG: Getting comments for #8736 (repository=repo/app, baseBranch=master, branch=renovate/pg-1.4.x)
DEBUG: Found 1 comments (repository=repo/app, baseBranch=master, branch=renovate/pg-1.4.x)
DEBUG: Ensuring comment "⚠ Artifact update problem" in #8736 (repository=repo/app, baseBranch=master, branch=renovate/pg-1.4.x)
DEBUG: POST https://gitlab.com/api/v4/projects/repo%2Fapp/merge_requests/8736/notes = (code=ERR_NON_2XX_3XX_RESPONSE, statusCode=400 retryCount=0, duration=85) (repository=repo/app, baseBranch=master, branch=renovate/pg-1.4.x)

When renovate was attempting to post the "⚠ Artifact update problem" comment to the merge request it created, this failed with a 400 error. As a result, the setup problem was masked for us and hard to track down - we didn't realize what was happening until digging into the debug logs.

It turns out the issue is that the error message emitted by bundler includes characters that are not allowed in GitLab comments. Note this part of the stderr output in the debug logs: The source contains 'Ruby\u0000' at: 2.7.6.219. Copy and pasting just that content into a GitLab comment in their web interface and hitting the 'Preview' button fails in the same way. I tested GitHub too, and it looks like it allows this character in comments without issue.

I appreciate this is a wild edge-case and the bad output appears to be generated by bundler. I guess the only fix would be to sanitize the comment body before attempting to post it from ensureComment. It looks like there's an existing sanitize utility function that strips out secrets. Perhaps that could also remove invalid characters.

Anyway, mostly posting here in case anyone else runs into the issue and is Googling it ❤️

[rarkins]

Thanks for reporting this.

Each platform (e.g. gitlab) has a massageMarkdown() function, e.g.

renovate/lib/modules/platform/gitlab/index.ts

Lines 696 to 714 in 6db995d

	export function massageMarkdown(input: string): string {
	let desc = input
	.replace(regEx(/Pull Request/g), 'Merge Request')
	.replace(regEx(/PR/g), 'MR')
	.replace(regEx(/\]\(\.\.\/pull\//g), '](!');
	if (semver.lt(defaults.version, '13.4.0')) {
	logger.debug(
	{ version: defaults.version },
	'GitLab versions earlier than 13.4 have issues with long descriptions, truncating to 25K characters'
	);
	desc = smartTruncate(desc, 25000);
	} else {
	desc = smartTruncate(desc, 1000000);
	}
	return desc;
	}

So that would be the place to add such massaging if it is platform-specific (however cleaning out \u000 maybe benefits all platforms, even if it's allowed by some.

[nfm]

Thanks for the input @rarkins, appreciate it! I'll try to put a PR together when I have time :)

[nfm]

PR: #19664

I ended up scoping the changes to just platform/gitlab as it looks like this is unnecessary for at least platform/github. I haven't checked if this bug exhibits on other platforms as I don't use them.