Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Steampipe fails to run when too many benchmarks use the same controls #528

Closed
cbruno10 opened this issue Jun 14, 2021 · 0 comments
Closed
Assignees
Labels
bug Something isn't working

Comments

@cbruno10
Copy link
Contributor

Describe the bug
I’m trying to run steampipe query on the hipaa branch (https://github.com/turbot/steampipe-mod-aws-compliance/tree/hipaa/hipaa):

Codys-MacBook-Pro:aws cbruno$ steampipe query
fatal error: pipe failed
​
goroutine 66 [running]:
runtime.throw(0x1a8950b, 0xb)
	/opt/hostedtoolcache/go/1.16.5/x64/src/runtime/panic.go:1117 +0x72 fp=0xc000059e00 sp=0xc000059dd0 pc=0x1035e12
runtime.sigNoteSetup(0x21c1560)
	/opt/hostedtoolcache/go/1.16.5/x64/src/runtime/os_darwin.go:98 +0xc5 fp=0xc000059e28 sp=0xc000059e00 pc=0x1032305
os/signal.signal_enable(0xbf0bb55300000002)
	/opt/hostedtoolcache/go/1.16.5/x64/src/runtime/sigqueue.go:228 +0xa5 fp=0xc000059e48 sp=0xc000059e28 pc=0x1068f45
os/signal.enableSignal(...)
	/opt/hostedtoolcache/go/1.16.5/x64/src/os/signal/signal_unix.go:49
os/signal.Notify.func1(0x2)
	/opt/hostedtoolcache/go/1.16.5/x64/src/os/signal/signal.go:145 +0x88 fp=0xc000059e68 sp=0xc000059e48 pc=0x15cd2c8
os/signal.Notify(0xc000a469c0, 0xc000059f08, 0x2, 0x2)
	/opt/hostedtoolcache/go/1.16.5/x64/src/os/signal/signal.go:165 +0x185 fp=0xc000059ee0 sp=0xc000059e68 pc=0x15ccce5
github.com/turbot/steampipe/db.(*InteractiveClient).startCancelHandler(0xc000a46960, 0xc000059fb8)
	/home/runner/work/steampipe/steampipe/db/interactive_client.go:89 +0x9f fp=0xc000059f38 sp=0xc000059ee0 pc=0x18856ff
github.com/turbot/steampipe/db.(*InteractiveClient).InteractiveQuery(0xc000a46960)
	/home/runner/work/steampipe/steampipe/db/interactive_client.go:66 +0x6c fp=0xc000059fd8 sp=0xc000059f38 pc=0x188546c
runtime.goexit()
	/opt/hostedtoolcache/go/1.16.5/x64/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000059fe0 sp=0xc000059fd8 pc=0x106c3e1
created by github.com/turbot/steampipe/db.RunInteractivePrompt
	/home/runner/work/steampipe/steampipe/db/query.go:49 +0x186
​
goroutine 1 [chan receive]:
github.com/turbot/steampipe/query/execute.RunInteractiveSession(0xc000314100, 0xc0008b4168)
	/home/runner/work/steampipe/steampipe/query/execute/execute.go:27 +0x105
github.com/turbot/steampipe/cmd.runQueryCmd(0xc00040c2c0, 0x21c0de0, 0x0, 0x0)
	/home/runner/work/steampipe/steampipe/cmd/query.go:101 +0x4ca
github.com/spf13/cobra.(*Command).execute(0xc00040c2c0, 0x21c0de0, 0x0, 0x0, 0xc00040c2c0, 0x21c0de0)
	/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:846 +0x2c2
github.com/spf13/cobra.(*Command).ExecuteC(0x216ba20, 0xc0003a2600, 0xc0003a2600, 0xc0003a2600)
	/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:950 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
	/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:887
github.com/turbot/steampipe/cmd.Execute()
	/home/runner/work/steampipe/steampipe/cmd/root.go:133 +0x2d
main.main()
	/home/runner/work/steampipe/steampipe/main.go:31 +0x6f
​
goroutine 65 [runnable]:
github.com/turbot/steampipe/utils.(*FileWatcher).start.func1(0xc000108420)
	/home/runner/work/steampipe/steampipe/utils/file_watcher.go:103
created by github.com/turbot/steampipe/utils.(*FileWatcher).start
	/home/runner/work/steampipe/steampipe/utils/file_watcher.go:103 +0x4d
​
goroutine 23 [select]:
database/sql.(*DB).connectionOpener(0xc000462a90, 0x1bc94c0, 0xc000d980c0)
	/opt/hostedtoolcache/go/1.16.5/x64/src/database/sql/sql.go:1133 +0xb5
created by database/sql.OpenDB
	/opt/hostedtoolcache/go/1.16.5/x64/src/database/sql/sql.go:740 +0x12a
​
goroutine 56 [syscall]:
syscall.syscall6(0x1109a80, 0x8, 0x0, 0x0, 0xc000494e88, 0xa, 0x21c10b0, 0x0, 0x0, 0x0)
	/opt/hostedtoolcache/go/1.16.5/x64/src/runtime/sys_darwin.go:41 +0x2e
golang.org/x/sys/unix.kevent(0x8, 0x0, 0x0, 0xc000494e88, 0xa, 0x21c10b0, 0x0, 0x0, 0x0)
	/home/runner/go/pkg/mod/golang.org/x/sys@v0.0.0-20201119102817-f84b799fce68/unix/zsyscall_darwin_amd64.go:292 +0xa5
golang.org/x/sys/unix.Kevent(0x8, 0x0, 0x0, 0x0, 0xc000494e88, 0xa, 0xa, 0x21c10b0, 0x0, 0x0, ...)
	/home/runner/go/pkg/mod/golang.org/x/sys@v0.0.0-20201119102817-f84b799fce68/unix/syscall_bsd.go:428 +0x71
github.com/fsnotify/fsnotify.read(0x8, 0xc000494e88, 0xa, 0xa, 0x21c10b0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/runner/go/pkg/mod/github.com/fsnotify/fsnotify@v1.4.7/kqueue.go:511 +0x6e
github.com/fsnotify/fsnotify.(*Watcher).readEvents(0xc000908060)
	/home/runner/go/pkg/mod/github.com/fsnotify/fsnotify@v1.4.7/kqueue.go:274 +0x81b
created by github.com/fsnotify/fsnotify.NewWatcher
	/home/runner/go/pkg/mod/github.com/fsnotify/fsnotify@v1.4.7/kqueue.go:62 +0x199
​
goroutine 57 [sleep]:
time.Sleep(0x5f5e100)
	/opt/hostedtoolcache/go/1.16.5/x64/src/runtime/time.go:193 +0xd2
github.com/turbot/steampipe/utils.(*FileWatcher).scheduleHandler.func1(0xc000108420)
	/home/runner/work/steampipe/steampipe/utils/file_watcher.go:298 +0x3b
created by github.com/turbot/steampipe/utils.(*FileWatcher).scheduleHandler
	/home/runner/work/steampipe/steampipe/utils/file_watcher.go:297 +0x10d

In the HIPAA benchmarks, I’m re-using controls from the conformance_pack dir, sometimes using a control in multiple benchmarks.

When I had 10-20 controls being re-used, it was able to run correctly and run queries.

If I have any errors in my mod, it throws the error correctly:

Codys-MacBook-Pro:aws cbruno$ steampipe query
Error: failed to load workspace: Failed to load all mod source files
Missing item separator: Expected a comma to mark the beginning of the next item.
(/Users/cbruno/steampipe/mods/aws/hipaa/164_308_a_1_ii_b.sp:11,14-63)

If I just have the controls in the conformance_pack dir, but not the hipaa dir, it loads ok with steampipe query

If I only have 2 files in hipaa, e.g.,

Codys-MacBook-Pro:aws cbruno$ ls hipaa/
164_308_a_1_ii_b.sp	hipaa.sp

It loads ok.

Steampipe version (steampipe -v)
v0.5.2

To reproduce
Run steampipe query on the hipaa branch (https://github.com/turbot/steampipe-mod-aws-compliance/tree/hipaa/hipaa).

Expected behavior
Steampipe should compile/run correctly.

Additional context
Add any other context about the problem here.

@cbruno10 cbruno10 added the bug Something isn't working label Jun 14, 2021
kaidaguerre added a commit that referenced this issue Jun 14, 2021
…when too many benchmarks use the same controls. Closes #528
kaidaguerre added a commit that referenced this issue Jun 14, 2021
…when too many benchmarks use the same controls. Closes #528

v0.5.3. Closes #530
kaidaguerre added a commit that referenced this issue Jun 14, 2021
…when too many benchmarks use the same controls. v0.5.3. Closes #528. Closes #530
kaidaguerre added a commit that referenced this issue Jun 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants