Some data races in tests

[bep]

Not reported in regular hugo. Looks like a Go stdlib issue:

go test -race ./...

WARNING: DATA RACE
Write by goroutine 27:
  runtime.mapassign1()
      /private/var/folders/q8/bf_4b1ts2zj0l7b0p1dv36lr0000gp/T/workdir/go/src/runtime/hashmap.go:411 +0x0
  text/template.(*Template).associate()
      /usr/local/go/src/text/template/template.go:229 +0x4c7
  text/template.(*Template).AddParseTree()
      /usr/local/go/src/text/template/template.go:130 +0x2b9
  text/template.(*Template).Parse()
      /usr/local/go/src/text/template/template.go:202 +0x589
  html/template.(*Template).Parse()
      /usr/local/go/src/html/template/template.go:158 +0x11b
  github.com/spf13/hugo/hugolib.pageRenderer()
      /Users/bep/go/src/github.com/spf13/hugo/hugolib/site.go:943 +0x229

Previous read by goroutine 26:
  runtime.mapaccess1_faststr()
      /private/var/folders/q8/bf_4b1ts2zj0l7b0p1dv36lr0000gp/T/workdir/go/src/runtime/hashmap_fast.go:179 +0x0
  html/template.(*Template).lookupAndEscapeTemplate()
      /usr/local/go/src/html/template/template.go:137 +0x75f
  html/template.(*Template).ExecuteTemplate()
      /usr/local/go/src/html/template/template.go:114 +0x5e
  github.com/spf13/hugo/tpl.(*GoHTMLTemplate).ExecuteTemplate()
      <autogenerated>:16 +0xa6
  github.com/spf13/hugo/hugolib.(*Site).renderThing()
      /Users/bep/go/src/github.com/spf13/hugo/hugolib/site.go:1521 +0x29b
  github.com/spf13/hugo/hugolib.(*Site).render()
      /Users/bep/go/src/github.com/spf13/hugo/hugolib/site.go:1496 +0x38b
  github.com/spf13/hugo/hugolib.(*Site).renderAndWritePage()
      /Users/bep/go/src/github.com/spf13/hugo/hugolib/site.go:1447 +0xee
  github.com/spf13/hugo/hugolib.pageRenderer()
      /Users/bep/go/src/github.com/spf13/hugo/hugolib/site.go:954 +0x532

[anthonyfok]

Seems a recent CircleCI build caught this data race too:

• https://circleci.com/gh/spf13/hugo/373

[tatsushid]

This seems to occur at touching Template.Tree member from text/template/template.go, html/template/template.go or hugolib/shortcode.go. The last case, go test -race ./hugolib detects https://github.com/spf13/hugo/blob/master/hugolib/shortcode.go#L175 in my environment too.

[bep]

@tatsushid that's not what I'm seeing, but it's template related, seems to me like it happens when execute and parse happens at the same time.

As this happens in tests only, I suggest @spf13 turn of circle ci until we find a proper fix. I don't want to throw "locks everywhere" until we know more.

[moorereason]

The problem is hugolib/site.go#L958, not the stdlib:

_, err := s.Tmpl.New(self).Parse(string(p.Content))

It looks to me like s.Tmpl is a single template hanging off of Site that we have multiple goroutines trying to write to (Parse on L958) and read from (Execute via L969) during page rendering.

The race detector has detected a race on this line since goroutines were added by @spf13 back in May with 3dfb475. That line went unchanged when goroutines were added.

One odd bit of trivia as an aside:
I can only get the race detector to complain when running tests. If I use go run -race main.go server -s docs or go build -race && ./hugo server -s docs, I can't get it to error out. I must be doing something wrong.

[bep]

I can only get the race detector to complain when running tests. If I use go run -race main.go server -s docs or go build -race && ./hugo server -s docs, I can't get it to error out. I must be doing something wrong.

The code path that fails is executed in tests with some synthetic test data not possible in real life (me thinks), with some Page.IsRenderable flag = false that not even the original committer remembers why he put there ...

No one have come up with a explanation of that code, so I'm gonna remove it ... or fix it ... like today if someone not complains.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.