Concurrent data generation (thread safety)

[ridv]

I'm using faker to generate data for a simulation that has multiple goroutines. It seems faker is currently unable to work with more than a single thread due to the fact that it uses rand.NewSource
here .

The docs say:

NewSource returns a new pseudo-random Source seeded with the given value. Unlike the default Source used by top-level functions, this source is not safe for concurrent use by multiple goroutines.

https://golang.org/pkg/math/rand/

It would be great to make it so we're able to use faker across multiple threads.

[agoalofalife]

@rdelval can write an example of how you use faker in multiple goroutines?

[agoalofalife]

on this topic I've found this discussion

[ridv]

Sure, the following should trigger

panic: runtime error: index out of range

goroutine 6 [running]:
math/rand.(*rngSource).Uint64(...)
        /Users/<username>/.gvm/gos/go1.9.4/src/math/rand/rng.go:246
math/rand.(*rngSource).Int63(0xc420083500, 0x4519710ecadc72e8)
        /Users/<username>.gvm/gos/go1.9.4/src/math/rand/rng.go:231 +0x8a
github.com/bxcodec/faker.randomString(0x19, 0x18, 0xc420083500)
        /Users/r<username>/go/src/github.com/bxcodec/faker/faker.go:299 +0x10c
github.com/bxcodec/faker.getValue(0x1178c40, 0x10c7780, 0x10c7780, 0xc420010360, 0x16, 0x98, 0x0)
        /Users/<username>/go/src/github.com/bxcodec/faker/faker.go:182 +0x1710
github.com/bxcodec/faker.FakeData(0x10c3f40, 0xc420010360, 0x0, 0x0)
        /Users/ <username>/go/src/github.com/bxcodec/faker/faker.go:131 +0x1d6
main.main.func1(0xc42001a2a0, 0xc420010360)
        /Users/<username>/testarea/test.go:21 +0x6a
created by main.main
        /Users/<username>/testarea/test.go:17 +0x9c
exit status 2

package main

import (
	"sync"

	"github.com/bxcodec/faker"
)

func main() {
	var wg sync.WaitGroup
	for i := 0; i < 40; i++ {
		wg.Add(1)
		go func() {
			defer wg.Done()

                         // Need a fixed number of strings in an array
			var name string 
			lst := make([]string, 0, 1000)

			for x := 0; x < 1000; x++ {
				faker.FakeData(&name)

				lst = append(lst, name)
			}
		}()
	}

	wg.Wait()

	return
}

If you run this code with the -race flag as well you'll see where the race conditions are.

Thanks for the link, hopefully Go 2 solves this issue. I was able to get this running with the default source without any race conditions. Only issue is the randomness is decreased due to the seed always being the same.

[agoalofalife]

We can add methods from native package and set seed
rand.Seed(time.Now().UnixNano()) as suggested in the discussion.

I put this entry in the general function(FakeData)

func FakeData(a interface{}) error {
	rand.Seed(time.Now().UnixNano())
// ...

// excerpt
func randomString(n int) string {
	b := make([]byte, n)
	for i, cache, remain := n-1, rand.Int63(), letterIdxMax; i >= 0; {

I did not notice that he did the same

 ZwbxVNpRjFBVyWfARJTCZRKqS WQJphBDMjQZmYkmrYVgHJWVkM ZrIWzJKnMQrxIUXAuvFaCZFHF zLTjnSfNjILIQfjsXnmakPEAa JExvPuGysQIoZnSlqlCdUGbat

@rdelval Do you have any suggestions?
You or I can send PR, just let us know)

[ridv]

In this case, to keep the pseudo-randomess it's best to keep a single seed set at the beginning of the initialization and use the default source throughout the rest of the program.

I'll raise a PR and we can continue the discussion there.

Thanks for looking into this and being receptive to feedback!

[agoalofalife]

PR #23