Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

e2e.GoWorkflow utility function to always couple goroutine & workflow execution #927

Closed
wants to merge 1 commit into from

Conversation

serprex
Copy link
Member

@serprex serprex commented Dec 29, 2023

Also tune timeouts on mock workflow environment
Also create .editorconfig

…orkflow execution

Also tune timeouts on mock workflow environment
@@ -317,6 +318,18 @@ func RunXminFlowWorkflow(env *testsuite.TestWorkflowEnvironment, config *protos.
env.ExecuteWorkflow(peerflow.XminFlowWorkflow, config, state)
}

// Execute fn in parallel with ExecuteWorkflow, returns when both complete
func GoWorkflow(fn func(), env *testsuite.TestWorkflowEnvironment, workflowFn interface{}, args ...interface{}) {
wg := sync.WaitGroup{}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tempted to make this a chan struct{} & then put a 1 minute timeout on channel being closed after ExecuteWorkflow finishes

@serprex
Copy link
Member Author

serprex commented Dec 29, 2023

Looks like this is uncovering hanging problems that were getting by in tests:

2023-12-29T03:12:02.6119569Z goroutine 761130 [sleep]:
2023-12-29T03:12:02.6119962Z time.Sleep(0x3b9aca00)
2023-12-29T03:12:02.6121034Z 	/home/runner/actions-runner/_work/_tool/go/1.21.5/arm64/src/runtime/time.go:195 +0x10c
2023-12-29T03:12:02.6122733Z github.com/PeerDB-io/peer-flow/e2e.NormalizeFlowCountQuery(0x4000fb7a00?, ***0x4012708080, 0x31***, 0x4012688060, 0x1bdc, 0x4003cb0370, ***0x0, 0x0***, 0x0***, 0x2)
2023-12-29T03:12:02.6123715Z 	/home/runner/actions-runner/_work/peerdb/peerdb/flow/e2e/test_utils.go:116 +0x60
2023-12-29T03:12:02.6124900Z github.com/PeerDB-io/peer-flow/e2e/snowflake.PeerFlowE2ETestSuiteSF.Test_Composite_PKey_Toast_2_SF.func1()
2023-12-29T03:12:02.6126418Z 	/home/runner/actions-runner/_work/peerdb/peerdb/flow/e2e/snowflake/peer_flow_sf_test.go:1115 +0x28c
2023-12-29T03:12:02.6127063Z github.com/PeerDB-io/peer-flow/e2e.GoWorkflow.func1()
2023-12-29T03:12:02.6127978Z 	/home/runner/actions-runner/_work/peerdb/peerdb/flow/e2e/test_utils.go:326 +0x30
2023-12-29T03:12:02.6128798Z created by github.com/PeerDB-io/peer-flow/e2e.GoWorkflow in goroutine 68
2023-12-29T03:12:02.6129706Z 	/home/runner/actions-runner/_work/peerdb/peerdb/flow/e2e/test_utils.go:325 +0x9c

@serprex
Copy link
Member Author

serprex commented Jan 6, 2024

With #980 I'm moving things towards not using WaitGroup at all, instead workflow never ends & goroutine cancels it when done

@serprex serprex closed this Jan 6, 2024
@serprex serprex deleted the fix-wg-go-e2e branch February 5, 2024 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants