Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

html/template: data race in indirect #30286

Closed
bep opened this issue Feb 17, 2019 · 18 comments
Closed

html/template: data race in indirect #30286

bep opened this issue Feb 17, 2019 · 18 comments
Labels
FrozenDueToAge WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.

Comments

@bep
Copy link
Contributor

bep commented Feb 17, 2019

I have a test that when run with test -race fails the stack below when run with go1.12rc1 and go1.12beta2. It does not fail with go1.11.5.

Write at 0x00c006544010 by goroutine 120:
 sync/atomic.CompareAndSwapInt32()
     /Users/bep/sdk/go1.12rc1/src/runtime/race_amd64.s:293 +0xb
 sync.(*Mutex).Lock()
     /Users/bep/sdk/go1.12rc1/src/sync/mutex.go:74 +0x4d
 sync.(*Once).Do()
     /Users/bep/sdk/go1.12rc1/src/sync/once.go:40 +0x60
     

Write at 0x00c006544018 by goroutine 120:
 sync/atomic.StoreInt32()
     /Users/bep/sdk/go1.12rc1/src/runtime/race_amd64.s:229 +0xb
 sync.(*Once).Do()
     /Users/bep/sdk/go1.12rc1/src/sync/once.go:46 +0x92
@davecheney
Copy link
Contributor

@bep can you please supply complete instructions on how to reproduce this issue. Thanks

@davecheney davecheney added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Feb 17, 2019
@bep
Copy link
Contributor Author

bep commented Feb 17, 2019

@bep can you please supply complete instructions on how to reproduce this issue. Thanks

Sorry, I don't have time to do that. The test in question is a little bit too involved and in-progress for me to post here. I may find time to create a standalone program that fails later in the week, but I thought you appreciate this bug report sooner rather than later if you plan to release Go 1.12 anytime soon.

@davecheney
Copy link
Contributor

Anything you can provide now would be of great assistance. The unedited terminal transcript would probably be enough so that someone else can try to reproduce the issue.

@bep bep changed the title sync: Data race reported in sync.Once (regression) sync: Data race reported in sync.Once Feb 17, 2019
@bep
Copy link
Contributor Author

bep commented Feb 17, 2019

I have removed the "regression label" in the title. It also fails on Go 1.11.5 -- it just doesn't fail every time, which lead me to believe that this was a Go 1.12 thing. I will look into this and try to understand this tomorrow.

@bep
Copy link
Contributor Author

bep commented Feb 18, 2019 •
edited by agnivade
Loading

I have looked into this, and I fail to understand it (and haven't been able to create an isolated test).

Here is the full output:

==================
WARNING: DATA RACE
Write at 0x00c00071f610 by goroutine 65:
  sync/atomic.CompareAndSwapInt32()
      /Users/bep/sdk/go1.12rc1/src/runtime/race_amd64.s:293 +0xb
  sync.(*Mutex).Lock()
      /Users/bep/sdk/go1.12rc1/src/sync/mutex.go:74 +0x4d
  sync.(*Once).Do()
      /Users/bep/sdk/go1.12rc1/src/sync/once.go:40 +0x60
  github.com/gohugoio/hugo/hugolib.(*pageState).Data()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/page_composite.go:486 +0x68
  runtime.call32()
      /Users/bep/sdk/go1.12rc1/src/runtime/asm_amd64.s:519 +0x3a
  reflect.Value.Call()
      /Users/bep/sdk/go1.12rc1/src/reflect/value.go:308 +0xc0
  text/template.safeCall()
      /Users/bep/sdk/go1.12rc1/src/text/template/funcs.go:293 +0xdd
  text/template.(*state).evalCall()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:696 +0x6e0
  text/template.(*state).evalField()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:586 +0xdb6
  text/template.(*state).evalFieldChain()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:551 +0x194
  text/template.(*state).evalFieldNode()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:518 +0x161
  text/template.(*state).evalArg()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:774 +0x1457
  text/template.(*state).evalCall()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:671 +0x325
  text/template.(*state).evalFunction()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:564 +0x1d7
  text/template.(*state).evalCommand()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:461 +0xb04
  text/template.(*state).evalPipeline()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:430 +0x1ff
  text/template.(*state).walk()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:254 +0x69c
  text/template.(*state).walk()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:262 +0x1a0
  text/template.(*Template).execute()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:217 +0x37e
  html/template.(*Template).Execute()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:200 +0xd3
  github.com/gohugoio/hugo/tpl.(*TemplateAdapter).Execute()
      /Users/bep/dev/go/gohugoio/hugo/tpl/template.go:135 +0xf8
  github.com/gohugoio/hugo/hugolib.(*Site).renderForLayouts()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1901 +0xcb
  github.com/gohugoio/hugo/hugolib.(*Site).renderAndWritePage()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1825 +0x15c
  github.com/gohugoio/hugo/hugolib.pageRenderer()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site_render.go:137 +0x8a9

Previous read at 0x00c00071f610 by goroutine 91:
  reflect.typedmemmove()
      /Users/bep/sdk/go1.12rc1/src/runtime/mbarrier.go:177 +0x0
  reflect.packEface()
      /Users/bep/sdk/go1.12rc1/src/reflect/value.go:119 +0x104
  reflect.valueInterface()
      /Users/bep/sdk/go1.12rc1/src/reflect/value.go:1009 +0x170
  html/template.indirect()
      /Users/bep/sdk/go1.12rc1/src/reflect/value.go:979 +0x154
  html/template.stringify()
      /Users/bep/sdk/go1.12rc1/src/html/template/content.go:153 +0x240
  html/template.htmlEscaper()
      /Users/bep/sdk/go1.12rc1/src/html/template/html.go:43 +0x53
  runtime.call64()
      /Users/bep/sdk/go1.12rc1/src/runtime/asm_amd64.s:520 +0x3a
  reflect.Value.Call()
      /Users/bep/sdk/go1.12rc1/src/reflect/value.go:308 +0xc0
  text/template.safeCall()
      /Users/bep/sdk/go1.12rc1/src/text/template/funcs.go:293 +0xdd
  text/template.(*state).evalCall()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:696 +0x6e0
  text/template.(*state).evalFunction()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:564 +0x1d7
  text/template.(*state).evalCommand()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:461 +0xb04
  text/template.(*state).evalPipeline()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:430 +0x1ff
  text/template.(*state).walk()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:254 +0x69c
  text/template.(*state).walk()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:262 +0x1a0
  text/template.(*Template).execute()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:217 +0x37e
  html/template.(*Template).Execute()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:200 +0xd3
  github.com/gohugoio/hugo/tpl.(*TemplateAdapter).Execute()
      /Users/bep/dev/go/gohugoio/hugo/tpl/template.go:135 +0xf8
  github.com/gohugoio/hugo/hugolib.(*Site).renderForLayouts()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1901 +0xcb
  github.com/gohugoio/hugo/hugolib.(*Site).renderAndWritePage()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1825 +0x15c
  github.com/gohugoio/hugo/hugolib.pageRenderer()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site_render.go:137 +0x8a9

Goroutine 65 (running) created at:
  github.com/gohugoio/hugo/hugolib.(*Site).renderPages()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site_render.go:47 +0x196
  github.com/gohugoio/hugo/hugolib.(*Site).render()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1258 +0x75
  github.com/gohugoio/hugo/hugolib.(*HugoSites).render()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_sites_build.go:293 +0x4b3
  github.com/gohugoio/hugo/hugolib.(*HugoSites).Build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_sites_build.go:97 +0x8e1
  github.com/gohugoio/hugo/hugolib.(*sitesBuilder).build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/testhelpers_test.go:382 +0x11f
  github.com/gohugoio/hugo/hugolib.TestSmoke()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/testhelpers_test.go:370 +0x106f
  testing.tRunner()
      /Users/bep/sdk/go1.12rc1/src/testing/testing.go:865 +0x163

Goroutine 91 (running) created at:
  github.com/gohugoio/hugo/hugolib.(*Site).renderPages()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site_render.go:47 +0x196
  github.com/gohugoio/hugo/hugolib.(*Site).render()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1258 +0x75
  github.com/gohugoio/hugo/hugolib.(*HugoSites).render()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_sites_build.go:293 +0x4b3
  github.com/gohugoio/hugo/hugolib.(*HugoSites).Build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_sites_build.go:97 +0x8e1
  github.com/gohugoio/hugo/hugolib.(*sitesBuilder).build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/testhelpers_test.go:382 +0x11f
  github.com/gohugoio/hugo/hugolib.TestSmoke()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/testhelpers_test.go:370 +0x106f
  testing.tRunner()
      /Users/bep/sdk/go1.12rc1/src/testing/testing.go:865 +0x163
==================
==================
WARNING: DATA RACE
Write at 0x00c00071f620 by goroutine 65:
  github.com/gohugoio/hugo/hugolib.(*pageState).Data.func1()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/page_composite.go:487 +0x91
  sync.(*Once).Do()
      /Users/bep/sdk/go1.12rc1/src/sync/once.go:44 +0xde
  github.com/gohugoio/hugo/hugolib.(*pageState).Data()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/page_composite.go:486 +0x68
  runtime.call32()
      /Users/bep/sdk/go1.12rc1/src/runtime/asm_amd64.s:519 +0x3a
  reflect.Value.Call()
      /Users/bep/sdk/go1.12rc1/src/reflect/value.go:308 +0xc0
  text/template.safeCall()
      /Users/bep/sdk/go1.12rc1/src/text/template/funcs.go:293 +0xdd
  text/template.(*state).evalCall()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:696 +0x6e0
  text/template.(*state).evalField()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:586 +0xdb6
  text/template.(*state).evalFieldChain()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:551 +0x194
  text/template.(*state).evalFieldNode()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:518 +0x161
  text/template.(*state).evalArg()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:774 +0x1457
  text/template.(*state).evalCall()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:671 +0x325
  text/template.(*state).evalFunction()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:564 +0x1d7
  text/template.(*state).evalCommand()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:461 +0xb04
  text/template.(*state).evalPipeline()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:430 +0x1ff
  text/template.(*state).walk()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:254 +0x69c
  text/template.(*state).walk()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:262 +0x1a0
  text/template.(*Template).execute()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:217 +0x37e
  html/template.(*Template).Execute()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:200 +0xd3
  github.com/gohugoio/hugo/tpl.(*TemplateAdapter).Execute()
      /Users/bep/dev/go/gohugoio/hugo/tpl/template.go:135 +0xf8
  github.com/gohugoio/hugo/hugolib.(*Site).renderForLayouts()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1901 +0xcb
  github.com/gohugoio/hugo/hugolib.(*Site).renderAndWritePage()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1825 +0x15c
  github.com/gohugoio/hugo/hugolib.pageRenderer()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site_render.go:137 +0x8a9

Previous read at 0x00c00071f620 by goroutine 91:
  reflect.typedmemmove()
      /Users/bep/sdk/go1.12rc1/src/runtime/mbarrier.go:177 +0x0
  reflect.packEface()
      /Users/bep/sdk/go1.12rc1/src/reflect/value.go:119 +0x104
  reflect.valueInterface()
      /Users/bep/sdk/go1.12rc1/src/reflect/value.go:1009 +0x170
  html/template.indirect()
      /Users/bep/sdk/go1.12rc1/src/reflect/value.go:979 +0x154
  html/template.stringify()
      /Users/bep/sdk/go1.12rc1/src/html/template/content.go:153 +0x240
  html/template.htmlEscaper()
      /Users/bep/sdk/go1.12rc1/src/html/template/html.go:43 +0x53
  runtime.call64()
      /Users/bep/sdk/go1.12rc1/src/runtime/asm_amd64.s:520 +0x3a
  reflect.Value.Call()
      /Users/bep/sdk/go1.12rc1/src/reflect/value.go:308 +0xc0
  text/template.safeCall()
      /Users/bep/sdk/go1.12rc1/src/text/template/funcs.go:293 +0xdd
  text/template.(*state).evalCall()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:696 +0x6e0
  text/template.(*state).evalFunction()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:564 +0x1d7
  text/template.(*state).evalCommand()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:461 +0xb04
  text/template.(*state).evalPipeline()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:430 +0x1ff
  text/template.(*state).walk()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:254 +0x69c
  text/template.(*state).walk()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:262 +0x1a0
  text/template.(*Template).execute()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:217 +0x37e
  html/template.(*Template).Execute()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:200 +0xd3
  github.com/gohugoio/hugo/tpl.(*TemplateAdapter).Execute()
      /Users/bep/dev/go/gohugoio/hugo/tpl/template.go:135 +0xf8
  github.com/gohugoio/hugo/hugolib.(*Site).renderForLayouts()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1901 +0xcb
  github.com/gohugoio/hugo/hugolib.(*Site).renderAndWritePage()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1825 +0x15c
  github.com/gohugoio/hugo/hugolib.pageRenderer()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site_render.go:137 +0x8a9

Goroutine 65 (running) created at:
  github.com/gohugoio/hugo/hugolib.(*Site).renderPages()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site_render.go:47 +0x196
  github.com/gohugoio/hugo/hugolib.(*Site).render()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1258 +0x75
  github.com/gohugoio/hugo/hugolib.(*HugoSites).render()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_sites_build.go:293 +0x4b3
  github.com/gohugoio/hugo/hugolib.(*HugoSites).Build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_sites_build.go:97 +0x8e1
  github.com/gohugoio/hugo/hugolib.(*sitesBuilder).build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/testhelpers_test.go:382 +0x11f
  github.com/gohugoio/hugo/hugolib.TestSmoke()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/testhelpers_test.go:370 +0x106f
  testing.tRunner()
      /Users/bep/sdk/go1.12rc1/src/testing/testing.go:865 +0x163

Goroutine 91 (running) created at:
  github.com/gohugoio/hugo/hugolib.(*Site).renderPages()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site_render.go:47 +0x196
  github.com/gohugoio/hugo/hugolib.(*Site).render()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1258 +0x75
  github.com/gohugoio/hugo/hugolib.(*HugoSites).render()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_sites_build.go:293 +0x4b3
  github.com/gohugoio/hugo/hugolib.(*HugoSites).Build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_sites_build.go:97 +0x8e1
  github.com/gohugoio/hugo/hugolib.(*sitesBuilder).build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/testhelpers_test.go:382 +0x11f
  github.com/gohugoio/hugo/hugolib.TestSmoke()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/testhelpers_test.go:370 +0x106f
  testing.tRunner()
      /Users/bep/sdk/go1.12rc1/src/testing/testing.go:865 +0x163
==================
==================
WARNING: DATA RACE
Write at 0x00c00071f618 by goroutine 65:
  sync/atomic.StoreInt32()
      /Users/bep/sdk/go1.12rc1/src/runtime/race_amd64.s:229 +0xb
  sync.(*Once).Do()
      /Users/bep/sdk/go1.12rc1/src/sync/once.go:46 +0x92
  github.com/gohugoio/hugo/hugolib.(*pageState).Data()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/page_composite.go:486 +0x68
  runtime.call32()
      /Users/bep/sdk/go1.12rc1/src/runtime/asm_amd64.s:519 +0x3a
  reflect.Value.Call()
      /Users/bep/sdk/go1.12rc1/src/reflect/value.go:308 +0xc0
  text/template.safeCall()
      /Users/bep/sdk/go1.12rc1/src/text/template/funcs.go:293 +0xdd
  text/template.(*state).evalCall()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:696 +0x6e0
  text/template.(*state).evalField()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:586 +0xdb6
  text/template.(*state).evalFieldChain()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:551 +0x194
  text/template.(*state).evalFieldNode()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:518 +0x161
  text/template.(*state).evalArg()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:774 +0x1457
  text/template.(*state).evalCall()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:671 +0x325
  text/template.(*state).evalFunction()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:564 +0x1d7
  text/template.(*state).evalCommand()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:461 +0xb04
  text/template.(*state).evalPipeline()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:430 +0x1ff
  text/template.(*state).walk()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:254 +0x69c
  text/template.(*state).walk()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:262 +0x1a0
  text/template.(*Template).execute()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:217 +0x37e
  html/template.(*Template).Execute()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:200 +0xd3
  github.com/gohugoio/hugo/tpl.(*TemplateAdapter).Execute()
      /Users/bep/dev/go/gohugoio/hugo/tpl/template.go:135 +0xf8
  github.com/gohugoio/hugo/hugolib.(*Site).renderForLayouts()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1901 +0xcb
  github.com/gohugoio/hugo/hugolib.(*Site).renderAndWritePage()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1825 +0x15c
  github.com/gohugoio/hugo/hugolib.pageRenderer()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site_render.go:137 +0x8a9

Previous read at 0x00c00071f618 by goroutine 91:
  reflect.typedmemmove()
      /Users/bep/sdk/go1.12rc1/src/runtime/mbarrier.go:177 +0x0
  reflect.packEface()
      /Users/bep/sdk/go1.12rc1/src/reflect/value.go:119 +0x104
  reflect.valueInterface()
      /Users/bep/sdk/go1.12rc1/src/reflect/value.go:1009 +0x170
  html/template.indirect()
      /Users/bep/sdk/go1.12rc1/src/reflect/value.go:979 +0x154
  html/template.stringify()
      /Users/bep/sdk/go1.12rc1/src/html/template/content.go:153 +0x240
  html/template.htmlEscaper()
      /Users/bep/sdk/go1.12rc1/src/html/template/html.go:43 +0x53
  runtime.call64()
      /Users/bep/sdk/go1.12rc1/src/runtime/asm_amd64.s:520 +0x3a
  reflect.Value.Call()
      /Users/bep/sdk/go1.12rc1/src/reflect/value.go:308 +0xc0
  text/template.safeCall()
      /Users/bep/sdk/go1.12rc1/src/text/template/funcs.go:293 +0xdd
  text/template.(*state).evalCall()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:696 +0x6e0
  text/template.(*state).evalFunction()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:564 +0x1d7
  text/template.(*state).evalCommand()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:461 +0xb04
  text/template.(*state).evalPipeline()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:430 +0x1ff
  text/template.(*state).walk()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:254 +0x69c
  text/template.(*state).walk()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:262 +0x1a0
  text/template.(*Template).execute()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:217 +0x37e
  html/template.(*Template).Execute()
      /Users/bep/sdk/go1.12rc1/src/text/template/exec.go:200 +0xd3
  github.com/gohugoio/hugo/tpl.(*TemplateAdapter).Execute()
      /Users/bep/dev/go/gohugoio/hugo/tpl/template.go:135 +0xf8
  github.com/gohugoio/hugo/hugolib.(*Site).renderForLayouts()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1901 +0xcb
  github.com/gohugoio/hugo/hugolib.(*Site).renderAndWritePage()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1825 +0x15c
  github.com/gohugoio/hugo/hugolib.pageRenderer()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site_render.go:137 +0x8a9

Goroutine 65 (running) created at:
  github.com/gohugoio/hugo/hugolib.(*Site).renderPages()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site_render.go:47 +0x196
  github.com/gohugoio/hugo/hugolib.(*Site).render()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1258 +0x75
  github.com/gohugoio/hugo/hugolib.(*HugoSites).render()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_sites_build.go:293 +0x4b3
  github.com/gohugoio/hugo/hugolib.(*HugoSites).Build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_sites_build.go:97 +0x8e1
  github.com/gohugoio/hugo/hugolib.(*sitesBuilder).build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/testhelpers_test.go:382 +0x11f
  github.com/gohugoio/hugo/hugolib.TestSmoke()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/testhelpers_test.go:370 +0x106f
  testing.tRunner()
      /Users/bep/sdk/go1.12rc1/src/testing/testing.go:865 +0x163

Goroutine 91 (running) created at:
  github.com/gohugoio/hugo/hugolib.(*Site).renderPages()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site_render.go:47 +0x196
  github.com/gohugoio/hugo/hugolib.(*Site).render()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1258 +0x75
  github.com/gohugoio/hugo/hugolib.(*HugoSites).render()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_sites_build.go:293 +0x4b3
  github.com/gohugoio/hugo/hugolib.(*HugoSites).Build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_sites_build.go:97 +0x8e1
  github.com/gohugoio/hugo/hugolib.(*sitesBuilder).build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/testhelpers_test.go:382 +0x11f
  github.com/gohugoio/hugo/hugolib.TestSmoke()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/testhelpers_test.go:370 +0x106f
  testing.tRunner()
      /Users/bep/sdk/go1.12rc1/src/testing/testing.go:865 +0x163
==================
--- FAIL: TestSmoke (0.19s)
    testing.go:809: race detected during execution of test
FAIL
exit status 1

The code itself should be fairly standard:

func (p *pageState) Data() interface{} {
       // dataInit is a sync.Once
	p.dataInit.Do(func() {
		p.data = make(page.Data)
                // add some data to the map
        }
       return p.data
}

I don't see how the above construct could ever become a data race, but the stack suggests this fails at a much lower level.

  • Could this be a false positive?
  • If so, is it possible to turn of race detection for single tests?

@davecheney
Copy link
Contributor

davecheney commented Feb 18, 2019 via email

@bep
Copy link
Contributor Author

bep commented Feb 18, 2019 •
edited
Loading

Does the race fire under go 1.11.x?

It fires under Go 1.11.5 (which is what I have installed by default) and above (but I have not tested it with < Go 1.11.5).

@davecheney
Copy link
Contributor

davecheney commented Feb 18, 2019 via email

@bep
Copy link
Contributor Author

bep commented Feb 18, 2019

The same race in Go 1.11.5:

Details 
==================
WARNING: DATA RACE
Read at 0x00c0007a6010 by goroutine 60:
  reflect.typedmemmove()
      /usr/local/go/src/runtime/mbarrier.go:177 +0x0
  reflect.packEface()
      /usr/local/go/src/reflect/value.go:119 +0x103
  reflect.valueInterface()
      /usr/local/go/src/reflect/value.go:1008 +0x16f
  reflect.Value.Interface()
      /usr/local/go/src/reflect/value.go:978 +0x51
  html/template.indirect()
      /usr/local/go/src/html/template/content.go:127 +0x145
  html/template.stringify()
      /usr/local/go/src/html/template/content.go:153 +0x23d
  html/template.htmlEscaper()
      /usr/local/go/src/html/template/html.go:43 +0x53
  runtime.call64()
      /usr/local/go/src/runtime/asm_amd64.s:523 +0x3a
  reflect.Value.Call()
      /usr/local/go/src/reflect/value.go:308 +0xc0
  text/template.(*state).evalCall()
      /usr/local/go/src/text/template/exec.go:695 +0x727
  text/template.(*state).evalFunction()
      /usr/local/go/src/text/template/exec.go:563 +0x1d9
  text/template.(*state).evalCommand()
      /usr/local/go/src/text/template/exec.go:460 +0x9de
  text/template.(*state).evalPipeline()
      /usr/local/go/src/text/template/exec.go:429 +0x200
  text/template.(*state).walk()
      /usr/local/go/src/text/template/exec.go:254 +0x69b
  text/template.(*state).walk()
      /usr/local/go/src/text/template/exec.go:262 +0x19f
  text/template.(*Template).execute()
      /usr/local/go/src/text/template/exec.go:217 +0x3a2
  text/template.(*Template).Execute()
      /usr/local/go/src/text/template/exec.go:200 +0x64
  html/template.(*Template).Execute()
      /usr/local/go/src/html/template/template.go:122 +0xd2
  github.com/gohugoio/hugo/tpl.(*TemplateAdapter).Execute()
      /Users/bep/dev/go/gohugoio/hugo/tpl/template.go:135 +0xf8
  github.com/gohugoio/hugo/hugolib.(*Site).renderForLayouts()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1901 +0xcb
  github.com/gohugoio/hugo/hugolib.(*Site).renderAndWritePage()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1825 +0x12f
  github.com/gohugoio/hugo/hugolib.pageRenderer()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site_render.go:137 +0x8a3

Previous write at 0x00c0007a6010 by goroutine 66:
  sync/atomic.AddInt32()
      /usr/local/go/src/runtime/race_amd64.s:269 +0xb
  sync.(*Mutex).Unlock()
      /usr/local/go/src/sync/mutex.go:182 +0x54
  sync.(*Once).Do()
      /usr/local/go/src/sync/once.go:46 +0x92
  github.com/gohugoio/hugo/hugolib.(*pageState).Data()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/page_composite.go:486 +0x68
  runtime.call32()
      /usr/local/go/src/runtime/asm_amd64.s:522 +0x3a
  reflect.Value.Call()
      /usr/local/go/src/reflect/value.go:308 +0xc0
  text/template.(*state).evalCall()
      /usr/local/go/src/text/template/exec.go:695 +0x727
  text/template.(*state).evalField()
      /usr/local/go/src/text/template/exec.go:585 +0xec4
  text/template.(*state).evalFieldChain()
      /usr/local/go/src/text/template/exec.go:550 +0x194
  text/template.(*state).evalFieldNode()
      /usr/local/go/src/text/template/exec.go:517 +0x160
  text/template.(*state).evalArg()
      /usr/local/go/src/text/template/exec.go:773 +0x1325
  text/template.(*state).evalCall()
      /usr/local/go/src/text/template/exec.go:670 +0x365
  text/template.(*state).evalFunction()
      /usr/local/go/src/text/template/exec.go:563 +0x1d9
  text/template.(*state).evalCommand()
      /usr/local/go/src/text/template/exec.go:460 +0x9de
  text/template.(*state).evalPipeline()
      /usr/local/go/src/text/template/exec.go:429 +0x200
  text/template.(*state).walk()
      /usr/local/go/src/text/template/exec.go:254 +0x69b
  text/template.(*state).walk()
      /usr/local/go/src/text/template/exec.go:262 +0x19f
  text/template.(*Template).execute()
      /usr/local/go/src/text/template/exec.go:217 +0x3a2
  text/template.(*Template).Execute()
      /usr/local/go/src/text/template/exec.go:200 +0x64
  html/template.(*Template).Execute()
      /usr/local/go/src/html/template/template.go:122 +0xd2
  github.com/gohugoio/hugo/tpl.(*TemplateAdapter).Execute()
      /Users/bep/dev/go/gohugoio/hugo/tpl/template.go:135 +0xf8
  github.com/gohugoio/hugo/hugolib.(*Site).renderForLayouts()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1901 +0xcb
  github.com/gohugoio/hugo/hugolib.(*Site).renderAndWritePage()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1825 +0x12f
  github.com/gohugoio/hugo/hugolib.pageRenderer()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site_render.go:137 +0x8a3

Goroutine 60 (running) created at:
  github.com/gohugoio/hugo/hugolib.(*Site).renderPages()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site_render.go:47 +0x196
  github.com/gohugoio/hugo/hugolib.(*Site).render()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1258 +0x6b
  github.com/gohugoio/hugo/hugolib.(*HugoSites).render()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_sites_build.go:293 +0x4be
  github.com/gohugoio/hugo/hugolib.(*HugoSites).Build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_sites_build.go:97 +0x8e5
  github.com/gohugoio/hugo/hugolib.(*sitesBuilder).build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/testhelpers_test.go:382 +0x11c
  github.com/gohugoio/hugo/hugolib.(*sitesBuilder).Build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/testhelpers_test.go:370 +0xb7
  github.com/gohugoio/hugo/hugolib.TestSmoke()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_smoke_test.go:118 +0xf00
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162

Goroutine 66 (running) created at:
  github.com/gohugoio/hugo/hugolib.(*Site).renderPages()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site_render.go:47 +0x196
  github.com/gohugoio/hugo/hugolib.(*Site).render()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1258 +0x6b
  github.com/gohugoio/hugo/hugolib.(*HugoSites).render()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_sites_build.go:293 +0x4be
  github.com/gohugoio/hugo/hugolib.(*HugoSites).Build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_sites_build.go:97 +0x8e5
  github.com/gohugoio/hugo/hugolib.(*sitesBuilder).build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/testhelpers_test.go:382 +0x11c
  github.com/gohugoio/hugo/hugolib.(*sitesBuilder).Build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/testhelpers_test.go:370 +0xb7
  github.com/gohugoio/hugo/hugolib.TestSmoke()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_smoke_test.go:118 +0xf00
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162
==================
==================
WARNING: DATA RACE
Read at 0x00c0007a6018 by goroutine 60:
  reflect.typedmemmove()
      /usr/local/go/src/runtime/mbarrier.go:177 +0x0
  reflect.packEface()
      /usr/local/go/src/reflect/value.go:119 +0x103
  reflect.valueInterface()
      /usr/local/go/src/reflect/value.go:1008 +0x16f
  reflect.Value.Interface()
      /usr/local/go/src/reflect/value.go:978 +0x51
  html/template.indirect()
      /usr/local/go/src/html/template/content.go:127 +0x145
  html/template.stringify()
      /usr/local/go/src/html/template/content.go:153 +0x23d
  html/template.htmlEscaper()
      /usr/local/go/src/html/template/html.go:43 +0x53
  runtime.call64()
      /usr/local/go/src/runtime/asm_amd64.s:523 +0x3a
  reflect.Value.Call()
      /usr/local/go/src/reflect/value.go:308 +0xc0
  text/template.(*state).evalCall()
      /usr/local/go/src/text/template/exec.go:695 +0x727
  text/template.(*state).evalFunction()
      /usr/local/go/src/text/template/exec.go:563 +0x1d9
  text/template.(*state).evalCommand()
      /usr/local/go/src/text/template/exec.go:460 +0x9de
  text/template.(*state).evalPipeline()
      /usr/local/go/src/text/template/exec.go:429 +0x200
  text/template.(*state).walk()
      /usr/local/go/src/text/template/exec.go:254 +0x69b
  text/template.(*state).walk()
      /usr/local/go/src/text/template/exec.go:262 +0x19f
  text/template.(*Template).execute()
      /usr/local/go/src/text/template/exec.go:217 +0x3a2
  text/template.(*Template).Execute()
      /usr/local/go/src/text/template/exec.go:200 +0x64
  html/template.(*Template).Execute()
      /usr/local/go/src/html/template/template.go:122 +0xd2
  github.com/gohugoio/hugo/tpl.(*TemplateAdapter).Execute()
      /Users/bep/dev/go/gohugoio/hugo/tpl/template.go:135 +0xf8
  github.com/gohugoio/hugo/hugolib.(*Site).renderForLayouts()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1901 +0xcb
  github.com/gohugoio/hugo/hugolib.(*Site).renderAndWritePage()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1825 +0x12f
  github.com/gohugoio/hugo/hugolib.pageRenderer()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site_render.go:137 +0x8a3

Previous write at 0x00c0007a6018 by goroutine 66:
  sync/atomic.StoreInt32()
      /usr/local/go/src/runtime/race_amd64.s:229 +0xb
  sync.(*Once).Do()
      /usr/local/go/src/sync/once.go:46 +0x92
  github.com/gohugoio/hugo/hugolib.(*pageState).Data()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/page_composite.go:486 +0x68
  runtime.call32()
      /usr/local/go/src/runtime/asm_amd64.s:522 +0x3a
  reflect.Value.Call()
      /usr/local/go/src/reflect/value.go:308 +0xc0
  text/template.(*state).evalCall()
      /usr/local/go/src/text/template/exec.go:695 +0x727
  text/template.(*state).evalField()
      /usr/local/go/src/text/template/exec.go:585 +0xec4
  text/template.(*state).evalFieldChain()
      /usr/local/go/src/text/template/exec.go:550 +0x194
  text/template.(*state).evalFieldNode()
      /usr/local/go/src/text/template/exec.go:517 +0x160
  text/template.(*state).evalArg()
      /usr/local/go/src/text/template/exec.go:773 +0x1325
  text/template.(*state).evalCall()
      /usr/local/go/src/text/template/exec.go:670 +0x365
  text/template.(*state).evalFunction()
      /usr/local/go/src/text/template/exec.go:563 +0x1d9
  text/template.(*state).evalCommand()
      /usr/local/go/src/text/template/exec.go:460 +0x9de
  text/template.(*state).evalPipeline()
      /usr/local/go/src/text/template/exec.go:429 +0x200
  text/template.(*state).walk()
      /usr/local/go/src/text/template/exec.go:254 +0x69b
  text/template.(*state).walk()
      /usr/local/go/src/text/template/exec.go:262 +0x19f
  text/template.(*Template).execute()
      /usr/local/go/src/text/template/exec.go:217 +0x3a2
  text/template.(*Template).Execute()
      /usr/local/go/src/text/template/exec.go:200 +0x64
  html/template.(*Template).Execute()
      /usr/local/go/src/html/template/template.go:122 +0xd2
  github.com/gohugoio/hugo/tpl.(*TemplateAdapter).Execute()
      /Users/bep/dev/go/gohugoio/hugo/tpl/template.go:135 +0xf8
  github.com/gohugoio/hugo/hugolib.(*Site).renderForLayouts()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1901 +0xcb
  github.com/gohugoio/hugo/hugolib.(*Site).renderAndWritePage()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1825 +0x12f
  github.com/gohugoio/hugo/hugolib.pageRenderer()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site_render.go:137 +0x8a3

Goroutine 60 (running) created at:
  github.com/gohugoio/hugo/hugolib.(*Site).renderPages()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site_render.go:47 +0x196
  github.com/gohugoio/hugo/hugolib.(*Site).render()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1258 +0x6b
  github.com/gohugoio/hugo/hugolib.(*HugoSites).render()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_sites_build.go:293 +0x4be
  github.com/gohugoio/hugo/hugolib.(*HugoSites).Build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_sites_build.go:97 +0x8e5
  github.com/gohugoio/hugo/hugolib.(*sitesBuilder).build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/testhelpers_test.go:382 +0x11c
  github.com/gohugoio/hugo/hugolib.(*sitesBuilder).Build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/testhelpers_test.go:370 +0xb7
  github.com/gohugoio/hugo/hugolib.TestSmoke()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_smoke_test.go:118 +0xf00
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162

Goroutine 66 (running) created at:
  github.com/gohugoio/hugo/hugolib.(*Site).renderPages()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site_render.go:47 +0x196
  github.com/gohugoio/hugo/hugolib.(*Site).render()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1258 +0x6b
  github.com/gohugoio/hugo/hugolib.(*HugoSites).render()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_sites_build.go:293 +0x4be
  github.com/gohugoio/hugo/hugolib.(*HugoSites).Build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_sites_build.go:97 +0x8e5
  github.com/gohugoio/hugo/hugolib.(*sitesBuilder).build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/testhelpers_test.go:382 +0x11c
  github.com/gohugoio/hugo/hugolib.(*sitesBuilder).Build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/testhelpers_test.go:370 +0xb7
  github.com/gohugoio/hugo/hugolib.TestSmoke()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_smoke_test.go:118 +0xf00
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162
==================
==================
WARNING: DATA RACE
Read at 0x00c0007a6020 by goroutine 60:
  reflect.typedmemmove()
      /usr/local/go/src/runtime/mbarrier.go:177 +0x0
  reflect.packEface()
      /usr/local/go/src/reflect/value.go:119 +0x103
  reflect.valueInterface()
      /usr/local/go/src/reflect/value.go:1008 +0x16f
  reflect.Value.Interface()
      /usr/local/go/src/reflect/value.go:978 +0x51
  html/template.indirect()
      /usr/local/go/src/html/template/content.go:127 +0x145
  html/template.stringify()
      /usr/local/go/src/html/template/content.go:153 +0x23d
  html/template.htmlEscaper()
      /usr/local/go/src/html/template/html.go:43 +0x53
  runtime.call64()
      /usr/local/go/src/runtime/asm_amd64.s:523 +0x3a
  reflect.Value.Call()
      /usr/local/go/src/reflect/value.go:308 +0xc0
  text/template.(*state).evalCall()
      /usr/local/go/src/text/template/exec.go:695 +0x727
  text/template.(*state).evalFunction()
      /usr/local/go/src/text/template/exec.go:563 +0x1d9
  text/template.(*state).evalCommand()
      /usr/local/go/src/text/template/exec.go:460 +0x9de
  text/template.(*state).evalPipeline()
      /usr/local/go/src/text/template/exec.go:429 +0x200
  text/template.(*state).walk()
      /usr/local/go/src/text/template/exec.go:254 +0x69b
  text/template.(*state).walk()
      /usr/local/go/src/text/template/exec.go:262 +0x19f
  text/template.(*Template).execute()
      /usr/local/go/src/text/template/exec.go:217 +0x3a2
  text/template.(*Template).Execute()
      /usr/local/go/src/text/template/exec.go:200 +0x64
  html/template.(*Template).Execute()
      /usr/local/go/src/html/template/template.go:122 +0xd2
  github.com/gohugoio/hugo/tpl.(*TemplateAdapter).Execute()
      /Users/bep/dev/go/gohugoio/hugo/tpl/template.go:135 +0xf8
  github.com/gohugoio/hugo/hugolib.(*Site).renderForLayouts()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1901 +0xcb
  github.com/gohugoio/hugo/hugolib.(*Site).renderAndWritePage()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1825 +0x12f
  github.com/gohugoio/hugo/hugolib.pageRenderer()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site_render.go:137 +0x8a3

Previous write at 0x00c0007a6020 by goroutine 66:
  github.com/gohugoio/hugo/hugolib.(*pageState).Data.func1()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/page_composite.go:487 +0x91
  sync.(*Once).Do()
      /usr/local/go/src/sync/once.go:44 +0xde
  github.com/gohugoio/hugo/hugolib.(*pageState).Data()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/page_composite.go:486 +0x68
  runtime.call32()
      /usr/local/go/src/runtime/asm_amd64.s:522 +0x3a
  reflect.Value.Call()
      /usr/local/go/src/reflect/value.go:308 +0xc0
  text/template.(*state).evalCall()
      /usr/local/go/src/text/template/exec.go:695 +0x727
  text/template.(*state).evalField()
      /usr/local/go/src/text/template/exec.go:585 +0xec4
  text/template.(*state).evalFieldChain()
      /usr/local/go/src/text/template/exec.go:550 +0x194
  text/template.(*state).evalFieldNode()
      /usr/local/go/src/text/template/exec.go:517 +0x160
  text/template.(*state).evalArg()
      /usr/local/go/src/text/template/exec.go:773 +0x1325
  text/template.(*state).evalCall()
      /usr/local/go/src/text/template/exec.go:670 +0x365
  text/template.(*state).evalFunction()
      /usr/local/go/src/text/template/exec.go:563 +0x1d9
  text/template.(*state).evalCommand()
      /usr/local/go/src/text/template/exec.go:460 +0x9de
  text/template.(*state).evalPipeline()
      /usr/local/go/src/text/template/exec.go:429 +0x200
  text/template.(*state).walk()
      /usr/local/go/src/text/template/exec.go:254 +0x69b
  text/template.(*state).walk()
      /usr/local/go/src/text/template/exec.go:262 +0x19f
  text/template.(*Template).execute()
      /usr/local/go/src/text/template/exec.go:217 +0x3a2
  text/template.(*Template).Execute()
      /usr/local/go/src/text/template/exec.go:200 +0x64
  html/template.(*Template).Execute()
      /usr/local/go/src/html/template/template.go:122 +0xd2
  github.com/gohugoio/hugo/tpl.(*TemplateAdapter).Execute()
      /Users/bep/dev/go/gohugoio/hugo/tpl/template.go:135 +0xf8
  github.com/gohugoio/hugo/hugolib.(*Site).renderForLayouts()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1901 +0xcb
  github.com/gohugoio/hugo/hugolib.(*Site).renderAndWritePage()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1825 +0x12f
  github.com/gohugoio/hugo/hugolib.pageRenderer()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site_render.go:137 +0x8a3

Goroutine 60 (running) created at:
  github.com/gohugoio/hugo/hugolib.(*Site).renderPages()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site_render.go:47 +0x196
  github.com/gohugoio/hugo/hugolib.(*Site).render()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1258 +0x6b
  github.com/gohugoio/hugo/hugolib.(*HugoSites).render()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_sites_build.go:293 +0x4be
  github.com/gohugoio/hugo/hugolib.(*HugoSites).Build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_sites_build.go:97 +0x8e5
  github.com/gohugoio/hugo/hugolib.(*sitesBuilder).build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/testhelpers_test.go:382 +0x11c
  github.com/gohugoio/hugo/hugolib.(*sitesBuilder).Build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/testhelpers_test.go:370 +0xb7
  github.com/gohugoio/hugo/hugolib.TestSmoke()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_smoke_test.go:118 +0xf00
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162

Goroutine 66 (running) created at:
  github.com/gohugoio/hugo/hugolib.(*Site).renderPages()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site_render.go:47 +0x196
  github.com/gohugoio/hugo/hugolib.(*Site).render()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/site.go:1258 +0x6b
  github.com/gohugoio/hugo/hugolib.(*HugoSites).render()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_sites_build.go:293 +0x4be
  github.com/gohugoio/hugo/hugolib.(*HugoSites).Build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_sites_build.go:97 +0x8e5
  github.com/gohugoio/hugo/hugolib.(*sitesBuilder).build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/testhelpers_test.go:382 +0x11c
  github.com/gohugoio/hugo/hugolib.(*sitesBuilder).Build()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/testhelpers_test.go:370 +0xb7
  github.com/gohugoio/hugo/hugolib.TestSmoke()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/hugo_smoke_test.go:118 +0xf00
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162
==================
--- FAIL: TestSmoke (0.19s)

@cuonglm
Copy link
Member

cuonglm commented Feb 18, 2019 •
edited
Loading

@bep I don't see page_composite.go in any branch of hugo github repo, can you please push your working code in temporary branch so we can look into the code.

If not, then is there anywhere in your code that pageState.Data() nested call?

@bep
Copy link
Contributor Author

bep commented Feb 18, 2019

@Gnouc it is in a repo on my GitHub account, and in heavy development/experimental state...

I just tested this by reducing this as much as possible.

func (p *pageState) Data() interface{} {
	p.dataInit.Do(func() {
		p.data = make(map[string]interface{})
		p.data["Testing"] = []string{}
	})

	return p.data
}

And then in a (HTML) template:

{{ len .Data.Testing }}

The above also crashes fairly consistently when running go test -race.

But the above construct looks way too common to be something I experience first, so I thought a little harder. The Data() interface{} is provided by an interface that is embedded in pageState. The default implementation is, for convenience mostly, implemented by pageState itself. If I move that to its own struct:

bep/hugo@82feba9

I can run this:

for i in {1..100}; do go test -count=1 -run=Smoke -race ./hugolib; done

With no failures. If you do the same for the previous commit, it fails pretty fast.

Note that even if I have a workaround for the above concrete test case, I still think this is a bug somewhere in the Go source code.

@crvv
Copy link
Contributor

crvv commented Feb 18, 2019 

Read at 0x00c0007a6010 by goroutine 60:
  reflect.typedmemmove()
      /usr/local/go/src/runtime/mbarrier.go:177 +0x0
  reflect.packEface()
      /usr/local/go/src/reflect/value.go:119 +0x103
  reflect.valueInterface()
      /usr/local/go/src/reflect/value.go:1008 +0x16f
  reflect.Value.Interface()
      /usr/local/go/src/reflect/value.go:978 +0x51
  html/template.indirect()
      /usr/local/go/src/html/template/content.go:127 +0x145

Previous write at 0x00c0007a6010 by goroutine 66:
  sync/atomic.AddInt32()
      /usr/local/go/src/runtime/race_amd64.s:269 +0xb
  sync.(*Mutex).Unlock()
      /usr/local/go/src/sync/mutex.go:182 +0x54
  sync.(*Once).Do()
      /usr/local/go/src/sync/once.go:46 +0x92
  github.com/gohugoio/hugo/hugolib.(*pageState).Data()
      /Users/bep/dev/go/gohugoio/hugo/hugolib/page_composite.go:486 +0x68

html/template.indirect() dereferences the pointer passed to it.
The pointer is a *hugolib.pageState, which contains a sync.Once.
The sync.Once is copied by html/template code but sync.Once must not be copied.
I guess this is the data race.

@bep bep changed the title sync: Data race reported in sync.Once html/template: data race in indirect Feb 20, 2019
@cuonglm
Copy link
Member

cuonglm commented Feb 23, 2019 •
edited
Loading

@bep From stack trace, it seems to me that github.com/gohugoio/hugo/hugolib.(*Site).renderForLayouts() is called in 2 goroutines, causing the data race, not the html/template.indirect().

@bep
Copy link
Contributor Author

bep commented Feb 23, 2019

it seems to me that github.com/gohugoio/hugo/hugolib.(*Site).renderForLayouts() is called in 2 goroutines, causing the data race, not the html/template.indirect().

renderForLayouts is called in several Go routines, yes. Which is by design.

https://godoc.org/html/template#Template.Execute states clearly that this should be safe:

A template may be executed safely in parallel, although if parallel executions share a Writer the output may be interleaved.

I understand that the above statement does not extend to the methods called from the templates, but in this case Exectute => indirect() is doing some unsafe pointer dereferencing it cannot do and still keep the promise in the GoDoc.

@mvdan
Copy link
Member

mvdan commented Jun 15, 2021

Was #30286 (comment) the source of the data race, then? It's unclear to me if we can close this issue.

@bep
Copy link
Contributor Author

bep commented Jun 15, 2021

@mvdan no, that comment does not pinpoint the source of this data race, but this is rather old, so I'm not sure I can help further.

@empijei
Copy link
Contributor

empijei commented Jun 21, 2021

This package was changed recently to protect concurrent Execute calls, does this issue still reproduce on Go 1.17?

@bep bep mentioned this issue Feb 24, 2022
Closed
@seankhliao
Copy link
Member

I think this can be closed as is. If anyone has a proper reproducer, feel free to file a new issue.

@golang golang locked and limited conversation to collaborators Feb 24, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@davecheney @bep @empijei @mvdan @cuonglm @crvv @gopherbot @seankhliao