Skip to content

Commit

Permalink
Stop spamming for wait channel acquirement in orderer integration test
Browse files Browse the repository at this point in the history
The code passes a function that returns a channel instead of the channel returned by that function.
As a result, the function is being polled thousands of times needlessly and thousands of channels
are being created, after which a goroutine per channel feeds a value into each of them.

Unfortunately, Gomega uses reflection based select on each of these channels, which is non-blocking,
meaning it just grabs a lock on the channel, and if the sender goroutine hasn't sent the item yet,
it returns false.

Since we pass in the Gomega.Eventually, a function (and not a channel), the reflection based receive is only attempted
once on each channel (out of the thousands created) and this creates a flake, because sometimes
we reach the timeout before one of the goroutines has a chance to obtain the lock on the channel
before the reflection based select obtains it, sees the channel buffer is empty, and the Gomega
attempt for that channel is then given up.

Change-Id: I4417703d17e48afec50f46f3ae6b77a8e61a9be7
Signed-off-by: Yacov Manevich <yacovm@il.ibm.com>
  • Loading branch information
yacovm authored and C0rWin committed Aug 28, 2021
1 parent f62e877 commit 6ec8d72
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions integration/raft/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ var _ = Describe("EndToEnd reconfiguration and onboarding", func() {
By("Starting orderer with malformed genesis block")
ordererRunner := network.OrdererGroupRunner()
process := ifrit.Invoke(ordererRunner)
Eventually(process.Wait, network.EventuallyTimeout).Should(Receive()) // orderer process should exit
Eventually(process.Wait(), network.EventuallyTimeout).Should(Receive()) // orderer process should exit
network.Cleanup()
os.RemoveAll(testDir)

Expand Down Expand Up @@ -208,7 +208,7 @@ var _ = Describe("EndToEnd reconfiguration and onboarding", func() {

exitCode := network.CreateChannelExitCode(channel, orderer, org1Peer0, org1Peer0, org2Peer0, orderer)
Expect(exitCode).NotTo(Equal(0))
Consistently(process.Wait).ShouldNot(Receive()) // malformed tx should not crash orderer
Consistently(process.Wait()).ShouldNot(Receive()) // malformed tx should not crash orderer
Expect(runner.Err()).To(gbytes.Say(`invalid new config metadata: ElectionTick \(10\) must be greater than HeartbeatTick \(10\)`))

By("Submitting channel config update with illegal value")
Expand Down

0 comments on commit 6ec8d72

Please sign in to comment.