Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flaky integration tests because of unexpected goroutines #4044

Closed
AlonZivony opened this issue May 15, 2024 · 1 comment · Fixed by #4047
Closed

Flaky integration tests because of unexpected goroutines #4044

AlonZivony opened this issue May 15, 2024 · 1 comment · Fixed by #4047

Comments

@AlonZivony
Copy link
Collaborator

Description

We seem to get a lot of found unexpected goroutines errors in the integration tests lately.
For example:

=== NAME  Test_EventsDependencies
    dependencies_test.go:158: found unexpected goroutines:
        [Goroutine 13 in state chan send, 1 minutes, with github.com/aquasecurity/tracee/tests/testutils.ExecCmdBgWithSudoAndCtx.func1.1 on top of the stack:
        github.com/aquasecurity/tracee/tests/testutils.ExecCmdBgWithSudoAndCtx.func1.1()
        	/home/runner/work/tracee/tracee/tests/testutils/exec.go:141 +0xcd
        created by github.com/aquasecurity/tracee/tests/testutils.ExecCmdBgWithSudoAndCtx.func1 in goroutine 12
        	/home/runner/work/tracee/tracee/tests/testutils/exec.go:136 +0x2c6
         Goroutine 127 in state chan send, 1 minutes, with github.com/aquasecurity/tracee/tests/testutils.ExecCmdBgWithSudoAndCtx.func1.1 on top of the stack:
        github.com/aquasecurity/tracee/tests/testutils.ExecCmdBgWithSudoAndCtx.func1.1()
        	/home/runner/work/tracee/tracee/tests/testutils/exec.go:141 +0xcd
        created by github.com/aquasecurity/tracee/tests/testutils.ExecCmdBgWithSudoAndCtx.func1 in goroutine 126
        	/home/runner/work/tracee/tracee/tests/testutils/exec.go:136 +0x2c6
         Goroutine 243 in state chan send, with github.com/aquasecurity/tracee/tests/testutils.ExecCmdBgWithSudoAndCtx.func1.1 on top of the stack:
        github.com/aquasecurity/tracee/tests/testutils.ExecCmdBgWithSudoAndCtx.func1.1()
        	/home/runner/work/tracee/tracee/tests/testutils/exec.go:141 +0xcd
        created by github.com/aquasecurity/tracee/tests/testutils.ExecCmdBgWithSudoAndCtx.func1 in goroutine 242
        	/home/runner/work/tracee/tracee/tests/testutils/exec.go:136 +0x2c6
        ]

It sabotages the efforts to merge PRs, so we should find its source.

@AlonZivony
Copy link
Collaborator Author

After looking into it, I think it is caused either by the capture tests or by the metrics tests which are using the ExecCmdBgWithSudoAndCtx function.
The function creates a goroutine that wait for the returned channel read.
However, if the channel won't be read, it will stay alive and be found afterwards by the tests that check for go leaks.
For first step we should probably add a check in these tests too for the leakage, and afterwards make sure that no goroutine is leaking.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants