Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Codeql create golang database error on macos #863

Open
p1keman opened this issue Jan 2, 2022 · 5 comments
Open

Codeql create golang database error on macos #863

p1keman opened this issue Jan 2, 2022 · 5 comments

Comments

@p1keman
Copy link

p1keman commented Jan 2, 2022

codeql-cli v2.7.3
golang v1.17.5
M1 macOS Monterey v12.0.1

I have installed go and added it to the environment variable. I tried adding go environment variables to .zshrc and /etc/paths, but none of them worked.

% go version
go version go1.17.5 darwin/arm64

% codeql database create --language=go test --source-root=../codeql-study --overwrite
Initializing database at /codeql-study/test.
Running build command: [/codeql/cli/codeql/go/tools/autobuild.sh]
[2022-01-02 19:00:46] [build-stderr] 2022/01/02 19:00:46 Unable to run the go command, is it installed?
[2022-01-02 19:00:46] [build-stderr] Error: signal: abort trap
[2022-01-02 19:00:46] [ERROR] Spawned process exited abnormally (code 1; tried to run: [/codeql/cli/codeql/tools/osx64/preload_tracer, /codeql/cli/codeql/go/tools/autobuild.sh])
A fatal error occurred: Exit status 1 from command: [/codeql/cli/codeql/go/tools/autobuild.sh]
@aeisenberg
Copy link
Contributor

Thanks for raising your issue. Mac M1 support is still in beta. Please make sure to read about all of the requirements for running on an M1.

If you have fulfilled all of these requirements and are still seeing the error, please run the command with maximum verbosity and attach the logs created to this issue.

@p1keman
Copy link
Author

p1keman commented Jan 3, 2022 •
edited by aeisenberg
Loading

I have met the requirements mentioned in the document. Codeql can create databases in python and javascript, but errors will occur in go and Java. This is the log of my go database creation.

build-tracer.log

[T 15:28:48 1445] Reading configuration file /Desktop/codeql-study/test/working/tracing/compiler-tracing5901691412012282673.spec...
[T 15:28:48 1445] Compilers file as of 2022-01-03 15:28:48
[T 15:28:48 1445] Compiler pattern: **/go-autobuilder
[T 15:28:48 1445] Replace intercepted call: 0
[T 15:28:48 1445] Extractor: <null>
[T 15:28:48 1445] Compilers file as of 2022-01-03 15:28:48
[T 15:28:48 1445] Compiler pattern: **/go
[T 15:28:48 1445] Replace intercepted call: 0
[T 15:28:48 1445] Extractor: /codeql/cli/codeql/go/tools/osx64/go-extractor
[T 15:28:48 1445] Prepend[0] "--mimic"
[T 15:28:48 1445] Prepend[1] ""${compiler}""
[T 15:28:48 1445] Initialising tags...
[T 15:28:48 1445] ID set to 00000000000005A5_0000000000000001 (parent root)
[T 15:28:48 1445] Reading configuration file /Desktop/codeql-study/test/working/tracing/compiler-tracing5901691412012282673.spec...
[T 15:28:48 1445] Initialising tags...
[T 15:28:48 1445] ID set to 00000000000005A5_0000000000000002 (parent root)
[T 15:28:48 1445] Warning: SEMMLE_EXEC and SEMMLE_EXECP not set. Falling back to path lookup on argv[0].
[T 15:28:48 1445] ==== Candidate to intercept: /codeql/cli/codeql/tools/osx64/runner-osx (canonical: /codeql/cli/codeql/tools/osx64/runner-osx) ====
[T 15:28:48 1446] Attempting to switch stdout/stderr to 3...
/Desktop/codeql-study/test/working/copy-root/000001F5/bin/sh.semmle.000005A5.slice.x86_64: replacing existing signature
[T 15:28:49 1445] Reading configuration file /Desktop/codeql-study/test/working/tracing/compiler-tracing5901691412012282673.spec...
[T 15:28:49 1445] Initialising tags...
[T 15:28:49 1445] ID set to 00000000000005A5_0000000000000003 (parent 00000000000005A5_0000000000000002)
[T 15:28:49 1445] ==== Candidate to intercept: /codeql/cli/codeql/go/tools/autobuild.sh (canonical: /codeql/cli/codeql/go/tools/autobuild.sh) ====
[T 15:28:49 1455] Attempting to switch stdout/stderr to 3...
/Desktop/codeql-study/test/working/copy-root/000001F5/bin/bash.semmle.000005A5.slice.x86_64: replacing existing signature
[T 15:28:49 1445] Reading configuration file /Desktop/codeql-study/test/working/tracing/compiler-tracing5901691412012282673.spec...
[T 15:28:49 1445] Initialising tags...
[T 15:28:49 1445] ID set to 00000000000005A5_0000000000000004 (parent 00000000000005A5_0000000000000003)
[T 15:28:49 1445] ==== Candidate to intercept: /bin/bash (canonical: /bin/bash) ====
[T 15:28:49 1461] Attempting to switch stdout/stderr to 3...
[T 15:28:50 1460] Reading configuration file /Desktop/codeql-study/test/working/tracing/compiler-tracing5901691412012282673.spec...
[T 15:28:50 1460] Initialising tags...
[T 15:28:50 1460] ID set to 00000000000005B4_0000000000000001 (parent 00000000000005A5_0000000000000004)
[T 15:28:50 1460] ==== Candidate to intercept: /codeql/cli/codeql/go/tools/osx64/go-autobuilder (canonical: /codeql/cli/codeql/go/tools/osx64/go-autobuilder) ====
[T 15:28:50 1460] === Intercepted call to /codeql/cli/codeql/go/tools/osx64/go-autobuilder === 
[T 15:28:50 1460] Disabling tracing for this command.
[T 15:28:50 1465] Error: Environment variable SEMMLE_PRELOAD_libtrace is not set (or empty), but it should be set.
[T 15:28:50 1465] Error: Environment variable SEMMLE_PRELOAD_libtrace is not set (or empty), but it should be set.

database-create-20220103.152847.727.log

[2022-01-03 15:28:47] This is codeql database create test --language=go --source-root=./go --overwrite
[2022-01-03 15:28:47] Log file was started late.
[2022-01-03 15:28:47] [PROGRESS] database create> Initializing database at /Desktop/codeql-study/test.
[2022-01-03 15:28:47] Running plumbing command: codeql database init --overwrite --language=go --source-root=/Desktop/codeql-study/go --allow-missing-source-root=false --allow-already-existing -- /Desktop/codeql-study/test
[2022-01-03 15:28:47] [PROGRESS] database init> Counting lines of code in /Desktop/codeql-study/go
[2022-01-03 15:28:47] [PROGRESS] database init> Resolving extractor go.
[2022-01-03 15:28:47] Calling plumbing command: codeql resolve languages --format=json
[2022-01-03 15:28:47] [DETAILS] resolve languages> Scanning for [codeql-extractor.yml] from /codeql/cli/codeql/.codeqlmanifest.json
[2022-01-03 15:28:47] [DETAILS] resolve languages> Parsing /codeql/cli/codeql/go/codeql-extractor.yml.
[2022-01-03 15:28:47] [DETAILS] resolve languages> Parsing /codeql/cli/codeql/python/codeql-extractor.yml.
[2022-01-03 15:28:47] [DETAILS] resolve languages> Parsing /codeql/cli/codeql/java/codeql-extractor.yml.
[2022-01-03 15:28:47] [DETAILS] resolve languages> Parsing /codeql/cli/codeql/html/codeql-extractor.yml.
[2022-01-03 15:28:47] [DETAILS] resolve languages> Parsing /codeql/cli/codeql/xml/codeql-extractor.yml.
[2022-01-03 15:28:47] [DETAILS] resolve languages> Parsing /codeql/cli/codeql/properties/codeql-extractor.yml.
[2022-01-03 15:28:47] [DETAILS] resolve languages> Parsing /codeql/cli/codeql/cpp/codeql-extractor.yml.
[2022-01-03 15:28:47] [DETAILS] resolve languages> Parsing /codeql/cli/codeql/csv/codeql-extractor.yml.
[2022-01-03 15:28:47] [DETAILS] resolve languages> Parsing /codeql/cli/codeql/csharp/codeql-extractor.yml.
[2022-01-03 15:28:47] [DETAILS] resolve languages> Parsing /codeql/cli/codeql/javascript/codeql-extractor.yml.
[2022-01-03 15:28:47] [DETAILS] resolve languages> Parsing /codeql/cli/codeql/ruby/codeql-extractor.yml.
[2022-01-03 15:28:48] Plumbing command codeql resolve languages completed:
                      {
                        "extractors" : {
                          "go" : [
                            {
                              "extractor_root" : "/codeql/cli/codeql/go"
                            }
                          ],
                          "python" : [
                            {
                              "extractor_root" : "/codeql/cli/codeql/python"
                            }
                          ],
                          "java" : [
                            {
                              "extractor_root" : "/codeql/cli/codeql/java"
                            }
                          ],
                          "html" : [
                            {
                              "extractor_root" : "/codeql/cli/codeql/html"
                            }
                          ],
                          "xml" : [
                            {
                              "extractor_root" : "/codeql/cli/codeql/xml"
                            }
                          ],
                          "properties" : [
                            {
                              "extractor_root" : "/codeql/cli/codeql/properties"
                            }
                          ],
                          "cpp" : [
                            {
                              "extractor_root" : "/codeql/cli/codeql/cpp"
                            }
                          ],
                          "csv" : [
                            {
                              "extractor_root" : "/codeql/cli/codeql/csv"
                            }
                          ],
                          "csharp" : [
                            {
                              "extractor_root" : "/codeql/cli/codeql/csharp"
                            }
                          ],
                          "javascript" : [
                            {
                              "extractor_root" : "/codeql/cli/codeql/javascript"
                            }
                          ],
                          "ruby" : [
                            {
                              "extractor_root" : "/codeql/cli/codeql/ruby"
                            }
                          ]
                        }
                      }
[2022-01-03 15:28:48] [DETAILS] database init> Found candidate extractor root for go: /codeql/cli/codeql/go.
[2022-01-03 15:28:48] [PROGRESS] database init> Successfully loaded extractor Go (go) from /codeql/cli/codeql/go.
[2022-01-03 15:28:48] [PROGRESS] database init> Created skeleton CodeQL database at /Desktop/codeql-study/test. This in-progress database is ready to be populated by an extractor.
[2022-01-03 15:28:48] Plumbing command codeql database init completed.
[2022-01-03 15:28:48] Using autobuild script /codeql/cli/codeql/go/tools/autobuild.sh.
[2022-01-03 15:28:48] [DETAILS] database create> Detected autobuild command for Go: /codeql/cli/codeql/go/tools/autobuild.sh.
[2022-01-03 15:28:48] [PROGRESS] database create> Running build command: [/codeql/cli/codeql/go/tools/autobuild.sh]
[2022-01-03 15:28:48] Running plumbing command: codeql database trace-command --working-dir=/Desktop/codeql-study/go --index-traceless-dbs --no-db-cluster -- /Desktop/codeql-study/test /codeql/cli/codeql/go/tools/autobuild.sh
[2022-01-03 15:28:48] Setting up tracing from specification at /codeql/cli/codeql/go/tools/osx64/compiler-tracing.spec.
[2022-01-03 15:28:48] [PROGRESS] database trace-command> Running command in /Desktop/codeql-study/go: [/codeql/cli/codeql/go/tools/autobuild.sh]
[2022-01-03 15:28:50] [build-stderr] 2022/01/03 15:28:50 Unable to run the go command, is it installed?
[2022-01-03 15:28:50] [build-stderr] Error: signal: abort trap
[2022-01-03 15:28:50] [ERROR] Spawned process exited abnormally (code 1; tried to run: [/codeql/cli/codeql/tools/osx64/preload_tracer, /codeql/cli/codeql/go/tools/autobuild.sh])
[2022-01-03 15:28:50] Exception caught at top level: Exit status 1 from command: [/codeql/cli/codeql/go/tools/autobuild.sh]
                      com.semmle.cli2.database.DatabaseProcessCommandCommon.executeSubcommand(DatabaseProcessCommandCommon.java:188)
                      com.semmle.cli2.database.TraceCommandCommand.executeSubcommand(TraceCommandCommand.java:86)
                      com.semmle.cli2.picocli.PlumbingRunner.run(PlumbingRunner.java:110)
                      com.semmle.cli2.picocli.SubcommandCommon.runPlumbingInProcess(SubcommandCommon.java:162)
                      com.semmle.cli2.database.CreateCommand.executeSubcommand(CreateCommand.java:151)
                      com.semmle.cli2.picocli.SubcommandCommon.call(SubcommandCommon.java:500)
                      com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:205)
                      com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:214)
                      com.semmle.cli2.CodeQL.main(CodeQL.java:98)

@edoardopirovano
Copy link
Contributor

edoardopirovano commented Jan 3, 2022 •
edited
Loading

Hmm, there is a known issue with tracing Java builds on M1 that will be fixed in the next release: github/codeql-cli-binaries#101

Looking at that log, this appears to be a separate issue however. @AlexDenisov do you agree?

@AlexDenisov
Copy link

@edoardopirovano it manifests in a different way, but the root cause is the same.

@p1keman at this point I can recommend either using CodeQL 2.7.1, or waiting for the next release (which is around the corner, but no exact ETA yet).

@p1keman
Copy link
Author

p1keman commented Jan 5, 2022

@edoardopirovano it manifests in a different way, but the root cause is the same.

@p1keman at this point I can recommend either using CodeQL 2.7.1, or waiting for the next release (which is around the corner, but no exact ETA yet).

Very good, I switched to codeql v2.7.1, can create a go database, thank you

This was referenced Apr 25, 2023
Open
Open
@snyk-bot snyk-bot mentioned this issue Apr 29, 2023
Open
@aliscco aliscco mentioned this issue Jun 2, 2023
Open
@snyk-bot snyk-bot mentioned this issue Jun 4, 2023
Open
This was referenced Oct 13, 2023
Open
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@aeisenberg @AlexDenisov @edoardopirovano @p1keman