[QUESTION] Ginkgo parallel specs using go routines.

[Kush-Saxena]

I am new to ginkgo and wanted some help. I have few specs which are interdependent hence I'm running them in a ordered execution strategy. Now I want to speed it up and run them in parallel but since ginkgo does not prefer interdependency I have to make a lot of tweaks making it parallel. So instead I am using go routines to achieve something similar. However It will not be returning correct execution time for each spec i.e logs are incorrect. I need help in finding some other alternative way or finding some workaround in the current way.

---

In Below Scenario:

Some test 2 is dependent on setup to complete however Some test can run meanwhile. I am able to achieve that here but this solution is not correct. Overall run time is correct i.e around 10 seconds. However individual subject nodes are giving wrong execution time (which is expected ) and I want to fix that.

Sample code:

var wg = &sync.WaitGroup{}
var mutex = &sync.Mutex{}
var setupDone = false

var _ = ginkgo.Describe("Some node", ginkgo.Ordered, func() {
	wg.Add(3)
	ginkgo.It("It is setup", func(ctx ginkgo.SpecContext) {
		go func() {
			defer wg.Done()
			defer ginkgo.GinkgoRecover()
			fmt.Println("Setting up. Others will wait for it")
			time.Sleep(10 * time.Second)
			mutex.Lock()
			setupDone = true
			mutex.Unlock()
			fmt.Println("setup completed, others will run now")
		}()
	})
	ginkgo.It("Some test", func() {
		go func() {
			defer wg.Done()
			defer ginkgo.GinkgoRecover()
			fmt.Println("I am independent from setup node")
			time.Sleep(10 * time.Second)
			fmt.Println("I am independent and i completed.")
		}()
	})
	ginkgo.It("Some test 2", func() {
		go func() {
			defer wg.Done()
			defer ginkgo.GinkgoRecover()
			fmt.Println("I am dependent on setup node to complete")
			gomega.Eventually(func() bool {
				fmt.Println("I am waiting for setup to complete...")
				mutex.Lock()
				defer mutex.Unlock()
				return setupDone
			}, time.Second*15, time.Second*5).Should(gomega.Equal(true), "setup condition did not matched within timeout")
			fmt.Println("setup condition matched. I will run")
			time.Sleep(10 * time.Second)
			fmt.Println("I am dependent. I completed")
		}()
	})
	ginkgo.It("It will wait", func() {
		wg.Wait()
	})
})

Thanks

[onsi]

hi there - in general i recommend against trying to orchestrate specs in parallel with goroutines the way you are trying to do. A lot of Ginkgo's assumptions are broken when you launch a goroutine that outlives the lifecycle of the It that spawned it. I recommend reading through Ginkgo's docs about spec independence and spec parallelization to understand Ginkgo's multi-process parallelization model.

I don't know if you're working on a Kubernetes-related test suite but I've been noticing more and more folks working in that context trying to implement the pattern you are asking for and I recently shared an overview of what Ginkgo expects and doesn't support very well right now.

In your particular case - it sounds like you have (a) some setup, (b) some tests that depend on that setup which must wait until that setup completes, (c) some tests that don't depend on that setup. Ordinarily I would run these tests like this:

Describe("my tests", func() {
    Describe("tests that require shared setup", func() {
         var client *MyClient
         BeforeEach(func() {
             client = PerformSetup()
         })
         
         It("tests something that relies on the setup", func() {
             //do stuff with client and make assertions
         })

         It("tests something else that relies on the setup", func() {
             //do other stuff with client and make assertions
         })
   })

   It("some other test that is independent", func() {
       //...
   })
})

When you run this suite with ginkgo -p all the test cases will run in parallel and in arbitrary order. The tests that are in the shared setup block will each run their own setup.

Sometimes repeating setup like this is too costly (though I strongly recommend actually running in parallel to see if that is the case for you!). For such usecases Ginkgo 2.0 released Ordered containers that have a notion of shared setup in the form of BeforeAll:

Describe("my tests", func() {
    Describe("tests that require shared setup", Ordered, func() {
         var client *MyClient
         BeforeAll(func() {
             client = PerformSetup()
         })
         
         It("tests something that relies on the setup", func() {
             //do stuff with client and make assertions
         })

         It("tests something else that relies on the setup", func() {
             //do other stuff with client and make assertions
         })
   })

   It("some other test that is independent", func() {
       //...
   })
})

Now when you run the suite with ginkgo -p the independent test will run in parallel to the others, but the tests in the shared setup block will run sequentially. Currently there is no way to ask Ginkgo to perform the shared setup once and then run the specs associated with that setup in parallel. I have ideas for how to implement such support - though I'm a bit concerned about the complexity it will introduce and the degree to which folks who ask for this feature have thought through how, precisely, they would use it to test a single shared external resource (e.g. how do you ensure the parallel tests don't interfere with each other? what should happen if the shared setup enters a bad state because of one test? should the others keep running?)

So, concretely, I would suggest:

(a) try repeating the setup with BeforeEach and running with ginkgo -p to evaluate the performance of that approach.
(b) stick with Ordered but move the actually independent test (your "Some test") into a separate non-ordered context.
(c) share some actual code with me so I can help think through what other options might be available to you.