-
Notifications
You must be signed in to change notification settings - Fork 17.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
html/template: data race in Execute #39807
Comments
It also fails on go1.13.12. |
/cc @empijei |
/cc @mikesamuel |
Some additional info:
As a quick workaround running the templates once before they are run concurrently addresses the issue (see code below), but we need to properly fix this. Thanks for reporting. package main
import (
"fmt"
"html/template"
"io/ioutil"
"log"
"sync"
)
func main() {
var wg sync.WaitGroup
jsTempl := `
{{- define "jstempl" -}}
var foo = "bar";
{{- end -}}
<script type="application/javascript">
{{ template "jstempl" $ }}
</script>
`
tpl := template.New("")
_, err := tpl.New("templ.html").Parse(jsTempl)
if err != nil {
log.Fatal(err)
}
const numTemplates = 20
for i := 0; i < numTemplates; i++ {
_, err = tpl.New(fmt.Sprintf("main%d.html", i)).Parse(`{{ template "templ.html" . }}`)
if err != nil {
log.Fatal(err)
}
}
// Temporary workaround to prevent the race.
for j := 0; j < numTemplates; j++ {
if err := tpl.ExecuteTemplate(ioutil.Discard, fmt.Sprintf("main%d.html", j), nil); err != nil {
log.Fatal(err)
}
}
for i := 1; i <= 10; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for j := 0; j < numTemplates; j++ {
if err := tpl.ExecuteTemplate(ioutil.Discard, fmt.Sprintf("main%d.html", j), nil); err != nil {
log.Fatal(err)
}
}
}()
}
wg.Wait()
} |
This appears to be the same data race causing gohugoio/hugo#7293, where it is triggered from text/template. Hugo have a temporary fork of text/template included in their codebase. I attempted to fix the data race in the fork: gohugoio/hugo#7507. They'd rather have it upstreamed though. |
This adds a new lock protecting "tmpl". See golang#39807
I pulled the patch out of Hugo's fork of text/template and changed it a bit:
Personally, I'd prefer a single lock as it's simpler and safer. What do you think, should I file a PR? |
Do you have a proposal to fix the race described in this bug report? If so please do send a CL my way 😄 |
Change https://golang.org/cl/257817 mentions this issue: |
This adds a new lock protecting "tmpl". Thanks to @bep for providing the test case. Fixes golang#39807
This adds a new lock protecting "tmpl". Thanks to @bep for providing the test case. Fixes golang#39807
This adds a new lock protecting "tmpl". Thanks to @bep for providing the test case. Fixes golang#39807
This adds a new lock protecting "tmpl". Fixes golang#39807
This adds a new lock protecting "tmpl". Fixes golang#39807
It looks to me like html/template's nameSpace.mu needs to be an RWLock and must be RLock'ed during t.text.Execute. |
I guess this will not get fixed in Go 1.16? |
The problem in text/template seems different to me. That seems to require calling |
Change https://golang.org/cl/274450 mentions this issue: |
See previously #2439, which was apparently the same problem but regressed due to the lack of a regression test (which is understandable given that the race detector debuted over a year later). |
@bcmills sure, but you (and with you I don't mean specifically you) cannot at the sam time say that data races are always a serious bug which need fixing and then brush this on to a future release when there is a working fix in place. This whole situation means that I need to somehow maintain a fix for this in a fork, which is not something I do with joy and certainly not something I get paid to do. /cc @spf13 |
I understand that the length of time this has been open is frustrating, but I don't see anybody saying this. Are you able to check whether https://golang.org/cl/274450 fixes the problems that you are seeing? |
This reverts CLs 274450 and 279492, except for the new tests. The new race test is changed to skip, as it now fails. We can try again for 1.17. Original CL descriptions: html/template: attach functions to namespace The text/template functions are stored in a data structure shared by all related templates, so do the same with the original, unwrapped, functions on the html/template side. html/template: avoid race when escaping updates template For #39807 Fixes #43855 Change-Id: I2ce91321ada06ea496a982aefe170eb5af9ba847 Reviewed-on: https://go-review.googlesource.com/c/go/+/285957 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
The basic issue here is that html/template expects to be able to call the text/template My attempt to fix this, following the suggestion at #39807 (comment), failed because nothing released the read lock held during a method call, so if the method call called Given html/template's needs, I think we kind of do have to fix this in text/template, along the lines of https://golang.org/cl/257817. It's certainly an abstraction violation to change text/template to fix an html/template problem. But given that html/template needs to concurrently call |
Following my comment in https://go-review.googlesource.com/c/go/+/274450/4/src/html/template/template.go#138, I think it is possible to fix this race by escaping all of the templates at the first call to I suspect that the failure observed in However, I suspect that that failure indicates a fundamental inconsistency in the API: if the template |
I don't think there is any inconsistency in the API. When a top-level template is escaped, all the A topological sort might work, if we don't care about the extra work. |
@ianlancetaylor if a building is about to tip over, you put some support pillars in place. I reviewed a perfectly fine CR (https://golang.org/cl/257817) that would have made a great pillar until you could get a proper foundation in place. Would you have released Go 1.16 with a similar bug in the net/http package? |
If the net/http bug had existed for several releases, and if it were this late in the release cycle, then, yes, I would not have blocked Go 1.16 from being released with such a bug. |
@bep I'm sorry you are upset about this issue. I understand that it is frustrating. It's frustrating to me too. I've probably put 40 hours into fixing this issue, and so far I have failed. That's a lot of time for me. That said, the comment you mention is different. I'm not blocking 1.16 because of that bug. And, to use your metaphor, that CL is not a case of putting in support pillars until there is a proper foundation. That CL is, I believe, the correct fix. |
Change https://golang.org/cl/316669 mentions this issue: |
OK, I think this is finally fixed, and the fix will be in the 1.17 release. I hope. Sorry for the lengthy work on this. |
… race This adds a new lock protecting "tmpl". This is a copy of https://golang.org/cl/257817 by Andreas Fleig, updated for current tip, and updated to start running the html/template TestEscapeRace test. Thanks to @bep for providing the test case. Fixes golang#39807 (cherry picked from golang.org/cl/331511) Change-Id: Ic8874484290283a49116812eeaffb8608346dc70 Reviewed-on: https://go-review.googlesource.com/c/go/+/316669 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Change https://golang.org/cl/348580 mentions this issue: |
…fix data race This adds a new lock protecting "tmpl". This is a copy of https://golang.org/cl/257817 by Andreas Fleig, updated for current tip, and updated to start running the html/template TestEscapeRace test. Thanks to @bep for providing the test case. For #39807 Fixes #47042 Change-Id: Ic8874484290283a49116812eeaffb8608346dc70 Reviewed-on: https://go-review.googlesource.com/c/go/+/316669 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> (cherry picked from commit 496d7c6) Reviewed-on: https://go-review.googlesource.com/c/go/+/348580 Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Change https://golang.org/cl/350730 mentions this issue: |
Change https://golang.org/cl/351115 mentions this issue: |
Change https://golang.org/cl/351116 mentions this issue: |
For #39807 Fixes #48436 Change-Id: I75f82fd8738dd2f11f0c69b1230e1be1abc36024 Reviewed-on: https://go-review.googlesource.com/c/go/+/350730 Trust: Ian Lance Taylor <iant@golang.org> Trust: Daniel Martí <mvdan@mvdan.cc> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
…king it For #39807 For #48436 Fixes #48444 Change-Id: I75f82fd8738dd2f11f0c69b1230e1be1abc36024 Reviewed-on: https://go-review.googlesource.com/c/go/+/350730 Trust: Ian Lance Taylor <iant@golang.org> Trust: Daniel Martí <mvdan@mvdan.cc> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Daniel Martí <mvdan@mvdan.cc> (cherry picked from commit ba1c52d) Reviewed-on: https://go-review.googlesource.com/c/go/+/351115
…king it For #39807 For #48436 Fixes #48443 Change-Id: I75f82fd8738dd2f11f0c69b1230e1be1abc36024 Reviewed-on: https://go-review.googlesource.com/c/go/+/350730 Trust: Ian Lance Taylor <iant@golang.org> Trust: Daniel Martí <mvdan@mvdan.cc> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Daniel Martí <mvdan@mvdan.cc> (cherry picked from commit ba1c52d) Reviewed-on: https://go-review.googlesource.com/c/go/+/351116
Building the below program with
go build -race
and then running it fails:WARNING: DATA RACE
The text was updated successfully, but these errors were encountered: