CodeQL codesign error message

[rnychyporuk-rbi]

We`re workin on CodeQL Static Application Security Testing (SAST) implementation for a Swift language using GitHub Actions. During this process, I encountered an issue with the CodeQL initialization step when integrating with a Fastlane-based build process.

When including the CodeQL initialization step (github/codeql-action/init@v2) in the GitHub Actions workflow before Fastlane-based build commands, the build job fails with a CodeSign error. The specific error message indicates that the CodeSign command has failed.

Interesting Behavior: if I remove the CodeQL initialization step from the GitHub Actions workflow, the build application process works perfectly without any issues - application builds well.

Error message from console output:

▸ ** ARCHIVE FAILED **
▸ The following build commands failed:
▸ 	CodeSign /Users/runner/Library/Developer/Xcode/DerivedData/APP-amdyapuwajsvoeelnrjmqpntrblu/Build/Intermediates.noindex/ArchiveIntermediates/APP_straging/IntermediateBuildFilesPath/UninstalledProducts/iphoneos/SaleforceNotificationService_staging.appex (in target 'SaleforceNotificationService_staging' from project 'APP')
▸ (1 failure)

From debug logs we got below entries:

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: warning: changes being made to the file will invalidate the code signature in: /Users/runner/actions-runner/_work/_temp/codeql_databases/working/copy-root/000001F6/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang-stat-cache.semmle.00008433.slice.x86_64
/Users/runner/actions-runner/_work/_temp/codeql_databases/working/copy-root/000001F6/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang-stat-cache.semmle.00008433.slice.x86_64: replacing existing signature
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: warning: changes being made to the file will invalidate the code signature in: /Users/runner/actions-runner/_work/_temp/codeql_databases/working/copy-root/000001F6/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang-stat-cache.semmle.00008433.slice.arm64
/Users/runner/actions-runner/_work/_temp/codeql_databases/working/copy-root/000001F6/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang-stat-cache.semmle.00008433.slice.arm64: replacing existing signature
[T 17:01:12 33906] Attempting to switch stdout/stderr to 7...
[T 17:01:12 33908] Attempting to switch stdout/stderr to 7...
[T 17:01:12 33909] Attempting to switch stdout/stderr to 7...
[T 17:01:12 33913] Attempting to switch stdout/stderr to 7...
/Users/runner/actions-runner/_work/_temp/codeql_databases/working/copy-root/000001F6/usr/bin/codesign.semmle.00008433.145DE1E8.slice.x86_64: replacing existing signature
[T 17:01:12 33914] Attempting to switch stdout/stderr to 7...
[T 17:01:12 33912] Initializing tracer.
[T 17:01:12 33912] Initialising tags...
[T 17:01:12 33912] ID set to 0000000000008478_0000000000000001 (parent 0000000000008433_0000000000000001)
[T 17:01:12 33912] ==== Candidate to intercept: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang-stat-cache (canonical: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang-stat-cache) ====
[T 17:01:12 33912] Executing the following tracer actions:
[T 17:01:12 33912] Tracer actions:
[T 17:01:12 33912] pre_invocations(0)
[T 17:01:12 33912] post_invocations(0)
[T 17:01:12 33912] trace_languages(1): [swift]
/Users/runner/actions-runner/_work/_temp/codeql_databases/working/copy-root/000001F6/usr/bin/codesign.semmle.00008433.slice.x86_64: replacing existing signature
/Users/runner/actions-runner/_work/_temp/codeql_databases/working/copy-root/000001F6/usr/bin/codesign.semmle.00008433.slice.x86_64: replacing existing signature
/Users/runner/actions-runner/_work/_temp/codeql_databases/working/copy-root/000001F6/usr/bin/codesign.semmle.00008433.slice.x86_64: replacing existing signature
/Users/runner/actions-runner/_work/_temp/codeql_databases/working/copy-root/000001F6/usr/bin/codesign.semmle.00008433.slice.x86_64: replacing existing signature

Why codeql replace signature and is it the case for failing our scan ?

[aibaars]

CodeQL traces the build process to figure out how source files are compiled and against which libraries/header files etc. To do this it uses the DYLD_INSERT_LIBRARIES environment variable to inject the CodeQL tracer into the running build process. On macOS the use of DYLD_INSERT_LIBRARIES is restricted when System Integrity Protection is enabled (it is on by default). System Integrity Protection removes the DYLD_INSERT_LIBRARIES variable from the environment for any binaries it considers to be part of the "system" (for example binaries installed in /usr/bin). To side-step these limitations, the CodeQL tracer copies binaries to a different folder (codeql_databases/working/copy-root) before running them. This copying may invalidate the original signature, so CodeQL provides them with new signatures.

The lines from the build-tracer.log above show that the CodeQL tracer is re-signing /usr/bin/clang-stat-cache which is normal. However, also see /usr/bin/codesign being re-signed, and perhaps that is causing problems. I'll ask the team to have a look.

In the mean time, could you try disabling CodeSigning for your build? CodeQL analysis only needs to intercept compiler processes, steps like packaging and signing are not important for the CodeQL scan.

[aibaars]

@rnychyporuk-rbi Hopefully github/codeql#15134 will fix the problem.

[adityasharad]

This will be fixed in CodeQL 2.15.5.

[rnychyporuk-rbi]

Thank you @adityasharad .
Sorry for the question but how to specify 2.15.15 version for CodeQL Init Action (github/codeql-action/init@v2). I tried to force a couple options but none of them works. CodeQL Init action initialise 2.15.0 version - so I still get the same codesign error

  Found CodeQL tools version 2.15.0 in the toolcache.
  /Users/runner/actions-runner/_work/_tool/CodeQL/2.15.0/x64/codeql/codeql version --format=json
  {
    "productName" : "CodeQL",
    "vendor" : "GitHub",
    "version" : "2.15.0",
    "sha" : "d0c03afb046bf4964cf4c",
    "branches" : [
      "codeql-cli-2.15.0"
    ],
    "copyright" : "Copyright (C) 2019-2023 GitHub, Inc.",
    "unpackedLocation" : "/Users/runner/actions-runner/_work/_tool/CodeQL/2.15.0/x64/codeql",
    "configFileLocation" : "/Users/runner/.config/codeql/config",
    "configFileFound" : false,
    "features" : {
      "featuresInVersionResult" : true,
      "supportsPython312" : true
    }
  }

[aibaars]

Have you tried the tools: option? You can set it to the URL of a codeql bundle asset from the releases page of the github/codeql-action repository.

[jakobholmgrenhiq]

I am running into a similar issue when building with fastlane where my action gets stuck on the codesigning step only when CodeQL is initialized prior to building.

The failing command is the following:
set -o pipefail && xcodebuild -workspace ./REDACTED.xcodeproj/project.xcworkspace -scheme REDACTED -configuration QA-Release -destination 'generic/platform=iOS' -archivePath ./build.xcarchive archive | tee /Users/runner/Library/Logs/gym/REDACTED\ QA.log | xcbeautify

In my case, the action gets stuck indefinitely with the last readable output being the following:
[13:03:49]: ▸ Signing REDACTED.framework (in target 'REDACTED' from project 'REDACTED')

This step usually completes in seconds but will get stuck until the action times out or is cancelled. This does not happen for the exact same pipeline without CodeQL.

The initialization, build, and analysis steps looks as follows:

      - name: Initialize CodeQL
        uses: github/codeql-action/init@v3.22.12
        with:
          languages: swift
          queries: security-and-quality
          tools: https://github.com/github/codeql-action/releases/download/codeql-bundle-v2.15.5/codeql-bundle-osx64.tar.gz

      - name: Build QA
         REDACTED fastlane step

      - name: Perform CodeQL Analysis
        uses: github/codeql-action/analyze@v3.22.12
        with:
          category: "/language:swift"

Runner: macos-13
Xcode-version: 15.0.1

Let me know if there is any additional information I should provide!

[aibaars]

@jakobholmgrenhiq Thanks for the detailed report! Could you try re-running the workflow with debug logging enabled? In that case the codeql-action uploads a debug artifact. The debug artifact contains a file log/build-tracer.log . Could you have a look at that file and post any lines that have the word codesign in them?

@AlexDenisov FYI

[jakobholmgrenhiq]

@jakobholmgrenhiq Thanks for the detailed report! Could you try re-running the workflow with debug logging enabled? In that case the codeql-action uploads a debug artifact. The debug artifact contains a file log/build-tracer.log . Could you have a look at that file and post any lines that have the word codesign in them?

@AlexDenisov FYI

First match is unique:

[T 13:21:58 9563] Attempting to switch stdout/stderr to 6...
/Users/runner/work/_temp/codeql_databases/working/copy-root/000001F5/usr/bin/codesign.semmle.000023CF.0A82CBC0.slice.x86_64: replacing existing signature
/Users/runner/work/_temp/codeql_databases/working/copy-root/000001F5/Applications/Xcode_15.0.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang-stat-cache.semmle.000023CF.09925910.slice.x86_64: replacing existing signature
/Users/runner/work/_temp/codeql_databases/working/copy-root/000001F5/usr/bin/codesign.semmle.000023CF.0B6BD7E8.slice.x86_64: replacing existing signature
/Users/runner/work/_temp/codeql_databases/working/copy-root/000001F5/usr/bin/codesign.semmle.000023CF.0BE22128.slice.x86_64: replacing existing signature

Rest looks as follows with slight variations:

[T 13:21:58 9570] Initializing tracer.
[T 13:21:58 9570] Initialising tags...
[T 13:21:58 9570] ID set to 0000000000002562_0000000000000001 (parent 00000000000023CF_0000000000000001)
[T 13:21:58 9570] ==== Candidate to intercept: /usr/bin/codesign (canonical: /usr/bin/codesign) ====
[T 13:21:58 9570] Lua: === Intercepted call to /usr/bin/codesign ===
[T 13:21:58 9570] Lua: Disabling tracing for language swift.
[T 13:21:58 9570] Executing the following tracer actions:
[T 13:21:58 9570] Tracer actions:
[T 13:21:58 9570] pre_invocations(0)
[T 13:21:58 9570] post_invocations(0)
[T 13:21:58 9570] trace_languages(0): []
[T 13:21:58 9570] Disabling tracing for this command.
/Applications/Xcode_15.0.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: warning: changes being made to the file will invalidate the code signature in: /Users/runner/work/_temp/codeql_databases/working/copy-root/000001F5/Applications/Xcode_15.0.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang-stat-cache.semmle.000023CF.09925910.slice.arm64

[T 13:21:58 9573] Initializing tracer.
[T 13:21:58 9573] Initialising tags...
[T 13:21:58 9573] ID set to 0000000000002565_0000000000000001 (parent 00000000000023CF_0000000000000001)
[T 13:21:58 9573] ==== Candidate to intercept: /usr/bin/codesign (canonical: /usr/bin/codesign) ====
[T 13:21:58 9573] Lua: === Intercepted call to /usr/bin/codesign ===
[T 13:21:58 9573] Lua: Disabling tracing for language swift.
[T 13:21:58 9573] Executing the following tracer actions:
[T 13:21:58 9573] Tracer actions:
[T 13:21:58 9573] pre_invocations(0)
[T 13:21:58 9573] post_invocations(0)
[T 13:21:58 9573] trace_languages(0): []
[T 13:21:58 9573] Disabling tracing for this command.
/Users/runner/work/_temp/codeql_databases/working/copy-root/000001F5/Applications/Xcode_15.0.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang-stat-cache.semmle.000023CF.09925910.slice.arm64: replacing existing signature

Let me know if you want me to attach more complete logs.

[aibaars]

Thanks!

@AlexDenisov I see that tracing is disabled after intercepting /usr/bin/codesign, however, I still see re-signed versions of codesign in the copy-root folder so I guess github/codeql@6cbf8ab did not entirely have the desired effect. I had expected that the tracer would skip all the SIP avoidance once it was turned "off". I suppose this is because it is only turned off for swift, and there could in theory still be another language that requires tracing of the process.

[angelapwen]

Is there any downside to adding the fix from github/codeql@6cbf8ab to all traced languages? If not we can try that out and see how it goes.

[jakobholmgrenhiq]

Let me know if I can assist by testing some release candidate, if at all possible, in our pipelines to see if it resolves it.

[aibaars]

Is there any downside to adding the fix from github/codeql@6cbf8ab to all traced languages? If not we can try that out and see how it goes.

I'm afraid that the above fix doesn't actually work. Even with trace: false the tracer will still relocate binaries, and I believe codesign does not "like" being relocated.

[aeisenberg]

Any ideas on other ways we can solve this? Perhaps we can special-case codesign so that it's not even relocated?

[angelapwen]

Following up — yes, Alex had a PR for exactly that in https://github.com/github/semmle-code/pull/48593 but it had to be reverted because it was failing integration tests on SIP-disabled ARM runners. I believe Alex is now working on the (separate) fix behind that issue https://github.com/github/codeql-c-team/issues/2160 and then https://github.com/github/semmle-code/pull/48593 can go in again. So, this issue is blocked on https://github.com/github/codeql-c-team/issues/2160 (correct me if wrong!)

[jakobholmgrenhiq]

Sorry to bump this if it is still blocked on another issue (which I sadly cant check before posting as it is internal). Do you have any estimates on when we can see this resolved?