Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data race in template test #5926

Open
bep opened this issue May 3, 2019 · 7 comments

Comments

@bep
Copy link
Member

commented May 3, 2019

Probably a bug in Go.

WARNING: DATA RACE
Read at 0x00c004fdf7d0 by goroutine 143:
  runtime.mapaccess1_faststr()
      /home/travis/.gimme/versions/go1.12.2.linux.amd64/src/runtime/map_faststr.go:12 +0x0
  text/template.(*state).walkTemplate()
      /home/travis/.gimme/versions/go1.12.2.linux.amd64/src/text/template/exec.go:398 +0x136
  text/template.(*state).walk()
      /home/travis/.gimme/versions/go1.12.2.linux.amd64/src/text/template/exec.go:267 +0x2a5
  text/template.(*state).walk()
      /home/travis/.gimme/versions/go1.12.2.linux.amd64/src/text/template/exec.go:262 +0x1a0
  text/template.(*Template).execute()
      /home/travis/.gimme/versions/go1.12.2.linux.amd64/src/text/template/exec.go:217 +0x2f8
  html/template.(*Template).Execute()
      /home/travis/.gimme/versions/go1.12.2.linux.amd64/src/text/template/exec.go:200 +0xd3
  github.com/gohugoio/hugo/tpl.(*TemplateAdapter).Execute()
      /home/travis/src/hugo/tpl/template.go:163 +0xf8
  github.com/gohugoio/hugo/hugolib.(*Site).renderForLayouts()
      /home/travis/src/hugo/hugolib/site.go:1801 +0xc8
  github.com/gohugoio/hugo/hugolib.(*Site).renderAndWritePage()
      /home/travis/src/hugo/hugolib/site.go:1717 +0x297
  github.com/gohugoio/hugo/hugolib.pageRenderer()
      /home/travis/src/hugo/hugolib/site_render.go:152 +0x8ed
Previous write at 0x00c004fdf7d0 by goroutine 121:
  runtime.mapassign_faststr()
      /home/travis/.gimme/versions/go1.12.2.linux.amd64/src/runtime/map_faststr.go:202 +0x0
  text/template.(*Template).associate()
      /home/travis/.gimme/versions/go1.12.2.linux.amd64/src/text/template/template.go:222 +0x19e
  text/template.(*Template).AddParseTree()
      /home/travis/.gimme/versions/go1.12.2.linux.amd64/src/text/template/template.go:128 +0x2ab
  html/template.(*escaper).commit()
      /home/travis/.gimme/versions/go1.12.2.linux.amd64/src/html/template/escape.go:810 +0x2b6
  html/template.escapeTemplate()
      /home/travis/.gimme/versions/go1.12.2.linux.amd64/src/html/template/escape.go:38 +0x318
  html/template.(*Template).escape()
      /home/travis/.gimme/versions/go1.12.2.linux.amd64/src/html/template/template.go:102 +0x319
  html/template.(*Template).Execute()
      /home/travis/.gimme/versions/go1.12.2.linux.amd64/src/html/template/template.go:119 +0x3c
  github.com/gohugoio/hugo/tpl.(*TemplateAdapter).Execute()
      /home/travis/src/hugo/tpl/template.go:163 +0xf8
  github.com/gohugoio/hugo/hugolib.(*Site).renderForLayouts()
      /home/travis/src/hugo/hugolib/site.go:1801 +0xc8
  github.com/gohugoio/hugo/hugolib.(*Site).renderAndWritePage()
      /home/travis/src/hugo/hugolib/site.go:1717 +0x297
  github.com/gohugoio/hugo/hugolib.pageRenderer()
      /home/travis/src/hugo/hugolib/site_render.go:152 +0x8ed
Goroutine 143 (running) created at:
  github.com/gohugoio/hugo/hugolib.(*Site).renderPages()
      /home/travis/src/hugo/hugolib/site_render.go:73 +0x199
  github.com/gohugoio/hugo/hugolib.(*Site).render()
      /home/travis/src/hugo/hugolib/site.go:1161 +0x8b
  github.com/gohugoio/hugo/hugolib.(*HugoSites).render()
      /home/travis/src/hugo/hugolib/hugo_sites_build.go:302 +0x9f0
  github.com/gohugoio/hugo/hugolib.(*HugoSites).Build.func4()
      /home/travis/src/hugo/hugolib/hugo_sites_build.go:121 +0x57
  runtime/trace.WithRegion()
      /home/travis/.gimme/versions/go1.12.2.linux.amd64/src/runtime/trace/annotation.go:137 +0x10b
  github.com/gohugoio/hugo/hugolib.(*HugoSites).Build()
      /home/travis/src/hugo/hugolib/hugo_sites_build.go:123 +0xa50
  github.com/gohugoio/hugo/hugolib.(*sitesBuilder).build()
      /home/travis/src/hugo/hugolib/testhelpers_test.go:440 +0x1a7
  github.com/gohugoio/hugo/hugolib.TestEmbeddedTemplates()
      /home/travis/src/hugo/hugolib/testhelpers_test.go:407 +0x2c8
  testing.tRunner()
      /home/travis/.gimme/versions/go1.12.2.linux.amd64/src/testing/testing.go:865 +0x163
Goroutine 121 (finished) created at:
  github.com/gohugoio/hugo/hugolib.(*Site).renderPages()
      /home/travis/src/hugo/hugolib/site_render.go:73 +0x199
  github.com/gohugoio/hugo/hugolib.(*Site).render()
      /home/travis/src/hugo/hugolib/site.go:1161 +0x8b
  github.com/gohugoio/hugo/hugolib.(*HugoSites).render()
      /home/travis/src/hugo/hugolib/hugo_sites_build.go:302 +0x9f0
  github.com/gohugoio/hugo/hugolib.(*HugoSites).Build.func4()
      /home/travis/src/hugo/hugolib/hugo_sites_build.go:121 +0x57
  runtime/trace.WithRegion()
      /home/travis/.gimme/versions/go1.12.2.linux.amd64/src/runtime/trace/annotation.go:137 +0x10b
  github.com/gohugoio/hugo/hugolib.(*HugoSites).Build()
      /home/travis/src/hugo/hugolib/hugo_sites_build.go:123 +0xa50
  github.com/gohugoio/hugo/hugolib.(*sitesBuilder).build()
      /home/travis/src/hugo/hugolib/testhelpers_test.go:440 +0x1a7
  github.com/gohugoio/hugo/hugolib.TestEmbeddedTemplates()
      /home/travis/src/hugo/hugolib/testhelpers_test.go:407 +0x2c8
  testing.tRunner()
      /home/travis/.gimme/versions/go1.12.2.linux.amd64/src/testing/testing.go:865 +0x163
==================
--- FAIL: TestEmbeddedTemplates (0.51s)
    testing.go:809: race detected during execution of test

@bep bep added the Bug label May 3, 2019

@bep bep added this to the v0.56 milestone May 3, 2019

bep added a commit that referenced this issue May 3, 2019

@larzza

This comment has been minimized.

Copy link

commented May 19, 2019

@bep I’m curious... why do you think this is a Go bug?

@bep

This comment has been minimized.

Copy link
Member Author

commented May 19, 2019

Because the data race (both read and write) is inside template.Execute:

https://godoc.org/html/template#Template.Execute

Which states that:

A template may be executed safely in parallel, although if parallel executions share a Writer the output may be interleaved.

@larzza

This comment has been minimized.

Copy link

commented May 19, 2019

Thanks, seems reasonable!

  • I guess that page rendering was performed in parallell even before HUGO_NUMWORKERMULTIPLIER was introduced...?
  • When did the test begin to fail?

I get the race for my theme both when using Go 1.11 and 1.12 (only tested with latest Go patches though).

@larzza

This comment has been minimized.

Copy link

commented May 19, 2019

Just noticed you’ve been down this road before...

@bep

This comment has been minimized.

Copy link
Member Author

commented May 19, 2019

It's a road with many side roads...

@larzza

This comment has been minimized.

Copy link

commented May 19, 2019

Indeed it is!

@bep

This comment has been minimized.

Copy link
Member Author

commented May 19, 2019

We have always rendered in parallel.

bep added a commit to bep/hugo that referenced this issue May 19, 2019

@bep bep modified the milestones: v0.56, v0.57 Jun 14, 2019

@bep bep modified the milestones: v0.57, v0.58 Jul 31, 2019

@bep bep modified the milestones: v0.58, v0.59 Aug 13, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.