Non-deterministic Behavior when Run in Multiple Goroutines

[bwendling]

I'm seeing non-deterministic behavior when I run dmp in multiple goroutines. Basically, the "diffs" generated by DiffMain() should be identical no matter how many goroutines are run, but they differ. I'm going to try my best to see if I can find the cause, but you might have a more deeper understanding of what's going on. :-)

Here is the code (also attached as a file):

package main

import (
        "fmt"
        "os"
        "sync"
        "sync/atomic"

        "github.com/sergi/go-diff/diffmatchpatch"
)

const (
        expect = "[{1 licensed } {0 under the apache license, version 2.0 (the} {-1  #} {0 'license'); you may not use this file except in compliance } {-1 # } {0 with the license. you may obtain a copy of the license at } {-1 # # } {0 http://www.apache.org/licenses/license-2.0 } {-1 # # } {0 unless required by applicable law or agreed to in writing, } {-1 # } {0 software distributed under the license is distributed on an} {-1  #} {0 'as is'basis, without warranties or conditions of any} {-1  #} {0  kind, either express or implied. see the license for the } {-1 # } {0 specific language governing permissions and limitations} {-1  #} {0  under the license.}]"
        
        unknown = "under the apache license, version 2.0 (the #'license'); you may not use this file except in compliance # with the license. you may obtain a copy of the license at # # http://www.apache.org/licenses/license-2.0 # # unless required by applicable law or agreed to in writing, # software distributed under the license is distributed on an #'as is'basis, without warranties or conditions of any # kind, either express or implied. see the license for the # specific language governing permissions and limitations # under the license."
        
        known = "licensed under the apache license, version 2.0 (the'license'); you may not use this file except in compliance with the license. you may obtain a copy of the license at http://www.apache.org/licenses/license-2.0 unless required by applicable law or agreed to in writing, software distributed under the license is distributed on an'as is'basis, without warranties or conditions of any kind, either express or implied. see the license for the specific language governing permissions and limitations under the license."
)

var dmp = diffmatchpatch.New()

const num = 50

func main() {
        var matched, missed int32
        var wg sync.WaitGroup
        wg.Add(num)
        for i := 0; i < num; i++ {
                go func(i int) {
                        defer wg.Done()
                        diffs := dmp.DiffMain(unknown, known, false)
                        s := fmt.Sprintf("%v", diffs)
                        if s != expect {
                                fmt.Fprintf(os.Stderr, "MISMATCH(%d):\n%s\n", i, s)
                                atomic.AddInt32(&missed, 1)
                        } else {
                                atomic.AddInt32(&matched, 1)
                        }
                }(i)
        }
        wg.Wait()
        fmt.Fprintf(os.Stderr, "NUMBER MATCHING: %d\n", matched)
        fmt.Fprintf(os.Stderr, "NUMBER MISMATCHING: %d\n", missed)
}

d.go.txt

[bwendling]

Note: You may need to increase the number of goroutines to 1000 or so.

[zimmski]

Needed 10000 goroutines to get a missmatch (and needed to changed the expected data, are you running the latest version?) 👍 However, this should be expected under load because of the timeout argument that is set to one second by default. Please add the following to your example. If you are still getting mixed results we should have serious look at this.

func init() {
	dmp.DiffTimeout = 0    // Do not time out on diff matching
}

Maybe we should change the default timeout so that there is no timeout?

[bwendling]

Ah! A timeout might explain things. I updated to the latest version after filing this, but I still had the same issue (as you discovered, you have to set it to over 10000). Anyway, I'll try the timeout setting. Thanks!

[zimmski]

Please let me know how it went

[bwendling]

It didn't go well. Setting dmp.DiffTimeout to 0 made it run indefinitely. I'm not so sure it matters though, because it takes so many goroutines to have it show.