segfault when running fetch #10

dydx · 2020-12-17T19:14:56Z

➜  ~ uname -a
Darwin MacBook-Pro.local 19.5.0 Darwin Kernel Version 19.5.0: Tue May 26 20:41:44 PDT 2020; root:xnu-6153.121.2~2/RELEASE_X86_64 x86_64

➜  ~ ./cloudquery version
Version: 0.4.6
Commit: 96e6541ee8c8a86e8bb4b580bf7bb526ce3ec68a
Date: 2020-12-17 14:03:28.112132 -0500 EST m=+0.011338626

➜  ~ ./cloudquery gen config gcp

➜  ~ ./cloudquery fetch
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x530fbcd]

goroutine 1 [running]:
github.com/cloudquery/cloudquery/providers/gcp/compute.(*Client).transformInstance(0xc000431c40, 0xc000147200, 0x9)
	/go/src/github.com/troian/golang-cross-example/providers/gcp/compute/instances.go:420 +0x1ad
github.com/cloudquery/cloudquery/providers/gcp/compute.(*Client).transformInstances(0xc000431c40, 0xc0005ee520, 0x2, 0x4, 0x0, 0x0, 0x0)
	/go/src/github.com/troian/golang-cross-example/providers/gcp/compute/instances.go:446 +0x7e
github.com/cloudquery/cloudquery/providers/gcp/compute.(*Client).instances(0xc000431c40, 0x5b57720, 0x0, 0x5e5a7e0, 0x0)
	/go/src/github.com/troian/golang-cross-example/providers/gcp/compute/instances.go:493 +0x20b
github.com/cloudquery/cloudquery/providers/gcp/compute.(*Client).CollectResource(0xc000431c40, 0xc00033c8e8, 0x9, 0x5b57720, 0x0, 0xc00043c990, 0xc)
	/go/src/github.com/troian/golang-cross-example/providers/gcp/compute/client.go:42 +0x187
github.com/cloudquery/cloudquery/providers/gcp.(*Provider).collectResource(0xc00031cc80, 0xc00033c8e0, 0x11, 0x5b57720, 0x0, 0x0, 0x0)
	/go/src/github.com/troian/golang-cross-example/providers/gcp/provider.go:99 +0x289
github.com/cloudquery/cloudquery/providers/gcp.(*Provider).Run(0xc00031cc80, 0x5b57720, 0xc00038f710, 0xc00031cc80, 0x0)
	/go/src/github.com/troian/golang-cross-example/providers/gcp/provider.go:64 +0xd2
github.com/cloudquery/cloudquery/cloudqueryclient.(*Client).Run(0xc00038ed50, 0x5fbdb3e, 0xc, 0xf, 0xc00063fd00)
	/go/src/github.com/troian/golang-cross-example/cloudqueryclient/client.go:137 +0x4b9
github.com/cloudquery/cloudquery/cmd.glob..func2(0x720b4c0, 0x72518c8, 0x0, 0x0, 0x0, 0x0)
	/go/src/github.com/troian/golang-cross-example/cmd/fetch.go:22 +0xae
github.com/spf13/cobra.(*Command).execute(0x720b4c0, 0x72518c8, 0x0, 0x0, 0x720b4c0, 0x72518c8)
	/go/pkg/mod/github.com/spf13/cobra@v1.1.0/command.go:850 +0x47c
github.com/spf13/cobra.(*Command).ExecuteC(0x720ba00, 0x4008965, 0xc00010e058, 0x0)
	/go/pkg/mod/github.com/spf13/cobra@v1.1.0/command.go:958 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
	/go/pkg/mod/github.com/spf13/cobra@v1.1.0/command.go:895
github.com/cloudquery/cloudquery/cmd.Execute()
	/go/src/github.com/troian/golang-cross-example/cmd/root.go:22 +0x31
main.main()
	/go/src/github.com/troian/golang-cross-example/main.go:8 +0x25

Using the provided config.yml example, i get the above error immediately. Iteratively commenting out components, I can get it to run:

providers:
  - name: gcp
    project_id: my-gcp-project
    resources:
      # - name: compute.instances
      - name: compute.autoscalers
      - name: compute.disk_types
      - name: compute.images
      - name: compute.interconnects
      # - name: compute.ssl_certificates
      # - name: compute.vpn_gateways
      # - name: iam.project_roles
      - name: iam.service_accounts
      - name: storage.buckets

➜  ~ ./cloudquery fetch
2020-12-17T14:10:51.616-0500	INFO	Fetched resources	{"provider": "gcp", "resource": "compute.addresses", "count": 0}
2020-12-17T14:10:52.215-0500	INFO	Fetched resources	{"provider": "gcp", "resource": "compute.disk_types", "count": 412}
2020-12-17T14:10:52.359-0500	INFO	Fetched resources	{"provider": "gcp", "resource": "compute.images", "count": 0}
2020-12-17T14:10:52.518-0500	INFO	Fetched resources	{"provider": "gcp", "resource": "compute.interconnects", "count": 0}
2020-12-17T14:10:52.968-0500	INFO	Fetched resources	{"provider": "gcp", "resource": "iam.service_accounts", "count": 17}
2020-12-17T14:10:58.918-0500	INFO	Fetched resources	{"provider": "gcp", "resource": "storage.buckets", "count": 23}

The text was updated successfully, but these errors were encountered:

#10

ghost · 2020-12-17T20:01:44Z

@dydx Thanks for opening this issue. https://github.com/cloudquery/cloudquery/releases/tag/v0.4.7 should solve. can you please confirm that it is solved.

dydx · 2020-12-17T20:13:03Z

I pulled this updated version down and ran it, with similar results:

➜  ~ ./cloudquery --version
cloudquery version 0.4.7

➜  ~ cat config.yml
providers:
  - name: gcp
    project_id: my-gcp-project
    resources:
      - name: compute.instances
      - name: compute.autoscalers
      - name: compute.disk_types
      - name: compute.images
      - name: compute.instances
      - name: compute.interconnects
      - name: compute.ssl_certificates
      - name: compute.vpn_gateways
      - name: iam.project_roles
      - name: iam.service_accounts
      - name: storage.buckets

➜  ~ ./cloudquery fetch
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x530ffb4]

goroutine 1 [running]:
github.com/cloudquery/cloudquery/providers/gcp/compute.(*Client).transformInstance(0xc000433c40, 0xc000146fc0, 0xe)
	/go/src/github.com/troian/golang-cross-example/providers/gcp/compute/instances.go:395 +0x2f4
github.com/cloudquery/cloudquery/providers/gcp/compute.(*Client).transformInstances(0xc000433c40, 0xc0005d0cd0, 0x7, 0x9, 0x0, 0x0, 0x0)
	/go/src/github.com/troian/golang-cross-example/providers/gcp/compute/instances.go:455 +0x7e
github.com/cloudquery/cloudquery/providers/gcp/compute.(*Client).instances(0xc000433c40, 0x5b57720, 0x0, 0x5e5a7e0, 0x0)
	/go/src/github.com/troian/golang-cross-example/providers/gcp/compute/instances.go:502 +0x20b
github.com/cloudquery/cloudquery/providers/gcp/compute.(*Client).CollectResource(0xc000433c40, 0xc0000385c8, 0x9, 0x5b57720, 0x0, 0xc0005b4830, 0xc)
	/go/src/github.com/troian/golang-cross-example/providers/gcp/compute/client.go:42 +0x187
github.com/cloudquery/cloudquery/providers/gcp.(*Provider).collectResource(0xc00009c8c0, 0xc0000385c0, 0x11, 0x5b57720, 0x0, 0x0, 0x0)
	/go/src/github.com/troian/golang-cross-example/providers/gcp/provider.go:99 +0x289
github.com/cloudquery/cloudquery/providers/gcp.(*Provider).Run(0xc00009c8c0, 0x5b57720, 0xc00009e8a0, 0xc00009c8c0, 0x0)
	/go/src/github.com/troian/golang-cross-example/providers/gcp/provider.go:64 +0xd2
github.com/cloudquery/cloudquery/cloudqueryclient.(*Client).Run(0xc00009e3f0, 0x5fbdb3e, 0xc, 0xf, 0xc00063fd00)
	/go/src/github.com/troian/golang-cross-example/cloudqueryclient/client.go:137 +0x4b9
github.com/cloudquery/cloudquery/cmd.glob..func2(0x720b4c0, 0x72518c8, 0x0, 0x0, 0x0, 0x0)
	/go/src/github.com/troian/golang-cross-example/cmd/fetch.go:22 +0xae
github.com/spf13/cobra.(*Command).execute(0x720b4c0, 0x72518c8, 0x0, 0x0, 0x720b4c0, 0x72518c8)
	/go/pkg/mod/github.com/spf13/cobra@v1.1.0/command.go:850 +0x47c
github.com/spf13/cobra.(*Command).ExecuteC(0x720ba00, 0x4008c05, 0xc00010e058, 0x0)
	/go/pkg/mod/github.com/spf13/cobra@v1.1.0/command.go:958 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
	/go/pkg/mod/github.com/spf13/cobra@v1.1.0/command.go:895
github.com/cloudquery/cloudquery/cmd.Execute()
	/go/src/github.com/troian/golang-cross-example/cmd/root.go:22 +0x31
main.main()
	/go/src/github.com/troian/golang-cross-example/main.go:8 +0x25

ghost · 2020-12-17T20:55:02Z

@dydx fixed this one and a few others. hopefully didn't miss any others https://github.com/cloudquery/cloudquery/releases

can you please check again?

…12.3 (#10) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [github.com/cloudquery/cq-provider-sdk](https://togithub.com/cloudquery/cq-provider-sdk) | require | patch | `v0.12.2` -> `v0.12.3` | --- ### Release Notes <details> <summary>cloudquery/cq-provider-sdk</summary> ### [`v0.12.3`](https://togithub.com/cloudquery/cq-provider-sdk/releases/tag/v0.12.3) [Compare Source](https://togithub.com/cloudquery/cq-provider-sdk/compare/v0.12.2...v0.12.3) ##### Bug Fixes - Sysctl freebsd ([#370](https://togithub.com/cloudquery/cq-provider-sdk/issues/370)) ([f52efe9](https://togithub.com/cloudquery/cq-provider-sdk/commit/f52efe93291f72637ed3236466b9b8c8713efd4a)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, click this checkbox. --- This PR has been generated by [Renovate Bot](https://togithub.com/renovatebot/renovate).

🤖 I have created a release *beep* *boop* --- ## [0.0.4](https://github.com/cloudquery/cq-provider-fuzz/compare/v0.0.3...v0.0.4) (2022-06-26) ### Bug Fixes * **deps:** Update module github.com/cloudquery/cq-provider-sdk to v0.12.3 ([#10](https://github.com/cloudquery/cq-provider-fuzz/issues/10)) ([fcdbd54](https://github.com/cloudquery/cq-provider-fuzz/commit/fcdbd545b76015f322119a81c41a1fdc952a2499)) * Embed correct version ([7ecbad0](https://github.com/cloudquery/cq-provider-fuzz/commit/7ecbad0f37376e9361712d001f0a2d95f2e7c814)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: cq-bot <null>

…14.7 (#10) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>

ghost pushed a commit that referenced this issue Dec 17, 2020

Bugfix segfault in gcp compute.instances

317791e

#10

ghost closed this as completed Dec 17, 2020

ghost reopened this Dec 17, 2020

ghost closed this as completed Dec 18, 2020

cq-bot mentioned this issue Aug 9, 2022

chore: release monorepo #1167

Merged

erezrokah pushed a commit that referenced this issue Aug 14, 2022

chore: Synced file(s) with cloudquery/.github (#10)

f997084

Co-authored-by: cq-bot <null>

erezrokah pushed a commit that referenced this issue Aug 14, 2022

fix(deps): Update module github.com/cloudquery/cq-provider-sdk to v0.…

06bd391

…14.7 (#10) Co-authored-by: Renovate Bot <renovate@whitesourcesoftware.com>

This issue was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

segfault when running fetch #10

segfault when running fetch #10

dydx commented Dec 17, 2020

ghost commented Dec 17, 2020

dydx commented Dec 17, 2020

ghost commented Dec 17, 2020

segfault when running fetch #10

segfault when running fetch #10

Comments

dydx commented Dec 17, 2020

ghost commented Dec 17, 2020

dydx commented Dec 17, 2020

ghost commented Dec 17, 2020