testscript: workaround quardratic behaviour of github.com/pkg/diff (#137

)

The current version of github.com/pkg/diff.Diff has quadratic behaviour.
This means that when we attempt a diff between relatively modest sized
files, it's easy to find yourself out of memory.

Therefore, when we see a diff between two large files (large defined in
terms of number of lines), tersely report that as if the two were binary
files, i.e. do not try to render the diff.

When github.com/pkg/diff or similar supports a linear algorithm:

golang/go#45200 (comment)

we can revert this change.

go.mod

@@ -1,6 +1,6 @@
 module github.com/rogpeppe/go-internal
 
-go 1.11
+go 1.15
 
 require (
 	github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e

testscript/cmd.go

@@ -141,6 +141,16 @@ func (ts *TestScript) doCmdCmp(args []string, env bool) {
 		// update the script.
 	}
 
+	// pkg/diff is quadratic at the moment.
+	// If the product of the number of lines in the inputs is too large,
+	// don't call pkg.Diff at all as it might take tons of memory or time.
+	// We found one million to be reasonable for an average laptop.
+	const maxLineDiff = 1_000_000
+	if strings.Count(text1, "\n")*strings.Count(text2, "\n") > maxLineDiff {
+		ts.Fatalf("large files %s and %s differ", name1, name2)
+		return
+	}
+
 	var sb strings.Builder
 	if err := diff.Text(name1, name2, text1, text2, &sb); err != nil {
 		ts.Check(err)