[BUG] gh actions-importer audit gitlab corrupts generated config.json/source.yml/workflow output with interleaved REDACTED text

[kalaiguna]

gh actions-importer audit gitlab inserts REDACTED between characters in generated artifacts

gh actions-importer audit gitlab corrupts generated config.json/source.yml/workflow output with interleaved "REDACTED" text

Description

Running gh actions-importer audit gitlab against certain GitLab pipelines
produces severely corrupted per-project output files. The generated
config.json, the copied source.yml, and the generated
.github/workflows/*.yml contain the literal string REDACTED interleaved
between almost every character of the file, making the JSON/YAML invalid and
unreadable. Even non-secret syntax (e.g. {, "id": 157,) is affected, so
this does not look like an intentional (if overzealous) secret-redaction
feature — it looks like a genuine corruption bug.

Environment

• gh actions-importer extension: v1.3.6
• Underlying Docker image: ghcr.io/actions-importer/cli:latest, reported
  version 1.3.22671 (confirmed latest via gh actions-importer update)
• gh CLI: 2.100.0
• Host OS: Windows 11, via Docker Desktop
• Command: gh actions-importer audit gitlab --output-dir <dir> --github-instance-url <url> --gitlab-instance-url <url> --gitlab-access-token *** --namespace <namespace>

Steps to Reproduce

1. Run gh actions-importer audit gitlab against a GitLab group containing
   pipelines that convert successfully or partially.
2. Inspect the generated <output-dir>/<namespace>/<project-path>/config.json,
   source.yml, and .github/workflows/*.yml.
3. For a subset of projects, the files are present but corrupted as described.

Observed (sanitized example)

A config.json that should begin:

{
  "id": 157,
  ...

instead begins (byte-for-byte, confirmed via hex dump — not a display or
encoding artifact):

REDACTED{REDACTED
REDACTED REDACTED REDACTED"REDACTEDiREDACTEDd"REDACTED:REDACTED REDACTED1REDACTED5REDACTED7REDACTED,REDACTED

i.e. the literal text REDACTED is inserted between almost every character.

The same pattern appears in source.yml (a verbatim copy of the original
.gitlab-ci.yml) and in the generated GitHub Actions workflow YAML for the
same project — including for pipelines that otherwise convert successfully
(the tool's own audit_summary.md, produced separately, is unaffected and
accurate).

Expected behavior

config.json, source.yml, and generated workflow files should contain
clean, valid JSON/YAML matching the source pipeline.

Additional notes

• Reproducible and deterministic — re-running the audit into a fresh output
  directory reproduces byte-identical corruption.
• Ruled out our own tooling as the cause: we wrap this command in a
  PowerShell script that applies its own secret-redaction regex as a
  post-processing step; the corruption is present even with that step fully
  disabled, directly on the tool's raw output.
• Searched the public github/gh-actions-importer source for a literal
  "REDACTED" string or related redaction/masking logic and found no match,
  suggesting the fault may be inside the actions-importer/cli Docker
  image's implementation rather than this repository's C# source, or in a
  dependency.
• Happy to provide the exact (non-sensitive) invocation and file layout if
  useful; the GitLab project content itself is internal/private and can't be
  shared directly.

[tomm1990]

that byte pattern is almost certainly empty-string secret redaction, not random corruption and not normal GitHub Actions masking (which uses ***).

if anything in the redaction list is "", a naive replace inserts the marker between every character:

{  "id": 157
→ REDACTED{REDACTED
REDACTED REDACTED..."REDACTEDiREDACTEDd"...

that matches your hex dump and why { / "id" get hit too. audit_summary.md staying clean also fits: only the per-project write path is going through that scrubber.

do this first (Windows + Docker Desktop)

1. right before gh actions-importer audit gitlab, dump env and look for empty values the extension/container might treat as secrets (token args, GH_*, GITLAB_*, ACTIONS_IMPORTER_*, anything you pass with -e / $env:):

Get-ChildItem Env: | Where-Object { $_.Value -eq '' } | Format-Table Name

2. unset / do not pass those empties into the Docker run. common footgun: optional flags set to "" in a wrapper script, or an empty secret in the shell profile.

3. re-run into a fresh --output-dir with only the required non-empty token env vars set explicitly.

4. if it still reproduces, try the same audit from WSL2/Linux (same image tag) with a clean env - rules out Docker Desktop env plumbing vs the CLI image itself.

why only a subset of projects

often those projects pull extra empty CI/variable values into the conversion context (or your wrapper injects empties only on some namespaces). if you can say whether the bad projects share empty GitLab CI variables / empty optional importer flags, that narrows it fast.

I also found no REDACTED string in the public github/gh-actions-importer repo - so this is likely inside ghcr.io/actions-importer/cli (or a dependency). worth a bug against that image with: Windows 11 + Docker Desktop, image 1.3.22671, and that the corruption matches empty-string replace.

[kalaiguna]

Confirmed root cause from the ghcr.io/actions-importer/cli:1.3.22671 image:

The image contains actions_importer-1.3.22671. Its SecretScanner writes redactions with:

content.gsub(leaked_secret["Raw"], "REDACTED")

There is no guard for an empty Raw value. In Ruby, replacing an empty string inserts REDACTED between every character, which exactly reproduces the observed output. For example:

"{\"id\":157}".gsub("", "REDACTED")
# REDACTED{"REDACTEDidREDACTED"...}

The cloned .NET CLI already skips empty host environment variables before passing them to Docker, and the corruption still reproduced after removing the only empty host variable and rerunning with a fresh output directory. This points to the container image's secret-scanner path (or its dependency), not the CLI launcher.

We are proceeding with two tracks:

1. Build a temporary derived image from the official image with an empty-value guard in SecretScanner, and use it locally for validation/migration work.
2. Request the upstream patch and switch back to the official image as soon as a fixed release is available.

The temporary image will not be published as an official replacement, and no source code, credentials, private URLs, or private pipeline content are included in this report.

[kalaiguna]

Further update: the proposed guard was applied to a temporary derived image based on ghcr.io/actions-importer/cli:1.3.22671 and the audit was rerun into a fresh output directory.

The audit produced the expected results, but the same interleaved REDACTED corruption remained: 62 affected per-project artifacts. The guard works in isolation, but it is not yet a validated workaround for the full audit.

This means either the launcher did not select the derived image, or another redaction path is responsible. We are not treating the derived image as a production workaround until that is isolated. The confirmed evidence and exact failing image version remain unchanged.

[tomm1990]

the guard can be right and still never execute. the public gh-actions-importer launcher has no image override.

DockerService.ExecuteCommandAsync always starts one ref: {CONTAINER_REGISTRY}/actions-importer/cli:{tag}. App.cs sets the image to actions-importer/cli, the tag to latest (or pre only when you pass --prerelease), and CONTAINER_REGISTRY to ghcr.io unless that env var is set. a derived image under any other name or tag is not started. gh actions-importer update runs docker pull on that same ref and replaces a local retag.

check the id the audit will actually run:

docker image inspect ghcr.io/actions-importer/cli:latest --format "{{.Id}} {{.Created}}"

that id has to be your derived build, not the official 1.3.22671 pull. to prove the process inside the container is yours, change the image version command so it prints a unique marker, then:

docker tag <derived-image> ghcr.io/actions-importer/cli:latest
gh actions-importer version

do not run update between that tag and the audit. version is docker run --rm on that same ref (App.GetVersionAsync). if it still prints stock 1.3.22671 and no marker, the audit never entered your image, which matches 62 files still interleaved.

if version shows the marker and a fresh --output-dir audit is still interleaved, the writer is not the SecretScanner copy you edited (another load path inside the image). paste that version line.

if the retag makes the audit clean, please mark the earlier comment on empty Raw + gsub as the answer: #357 (comment)

[kalaiguna]

Confirmed. The earlier local-registry test did not prove that the derived image was selected.

We rebuilt the derived image with the empty-Raw guard and a unique version marker, then tagged it as ghcr.io/actions-importer/cli:latest. gh actions-importer version displayed the marker, confirming the launcher used the derived image.

A fresh isolated GitLab audit completed with two successful pipeline extractions. It produced valid config.json, source YAML, and generated workflow YAML artifacts, with no interleaved REDACTED corruption detected.

This confirms the guard works when the launcher runs the derived image. The remaining extraction failures in separate tests were unrelated authorization errors while reading protected CI/CD variables.