Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not sending non-exit code on fatal errors #8749

Open
aileen opened this issue Jul 13, 2021 · 1 comment
Open

Not sending non-exit code on fatal errors #8749

aileen opened this issue Jul 13, 2021 · 1 comment
Assignees
Labels
Milestone

Comments

@aileen
Copy link

@aileen aileen commented Jul 13, 2021

What version of Hugo are you using (hugo version)?

$ hugo version
hugo v0.85.0+extended darwin/amd64 BuildDate=unknown

Does this issue reproduce with the latest release?

Yes, this is for us reproducible since 0.84.0 and persists with 0.85.0.


Our site is quite large and does more than 1k image resizes with HUGO which sometimes leads to a race condition (unrelated issue that we need to fix, probably caused by wrong usage). This is something we were ok dealing with, but since 0.84.0 we were seeing issues that Netlify was deploying our failed builds causing our site to be down.

Looking further into this, it seems that HUGO is not sending a non-exit code on build failure due to the race condition. This was working correctly in 0.83.1.

I can't link to the repository of the site, as it's a private one, but here are some example log snippets:

v0.83.1

INFO 2021/07/13 13:15:49 Using config file:
Start building sites …
fatal error: concurrent map read and map write

goroutine 385 [running]:
runtime.throw(0x5b7c497, 0x21)
	/usr/local/go/src/runtime/panic.go:1117 +0x72 fp=0xc0021dd030 sp=0xc0021dd000 pc=0x40386d2
runtime.mapaccess1_faststr(0x5aa1000, 0xc000780570, 0x5afed6b, 0x7, 0x0)
	/usr/local/go/src/runtime/map_faststr.go:21 +0x465 fp=0xc0021dd0a0 sp=0xc0021dd030 pc=0x4014a45
github.com/gohugoio/hugo/hugofs.FileMeta.GetInt(...)
	/root/project/hugo/hugofs/fileinfo.go:67
github.com/gohugoio/hugo/hugofs.FileMeta.Ordinal(...)
	/root/project/hugo/hugofs/fileinfo.go:150
github.com/gohugoio/hugo/hugofs.(*Walkway).walk.func1(0x1, 0x0, 0x57e22e0)
	/root/project/hugo/hugofs/walk.go:231 +0x367 fp=0xc0021dd160 sp=0xc0021dd0a0 pc=0x4568aa7
sort.insertionSort_func(0xc0021dd3b8, 0xc003f35aa0, 0x0, 0x2)
	/usr/local/go/src/sort/zfuncversion.go:12 +0xab fp=0xc0021dd198 sp=0xc0021dd160 pc=0x40a5ceb
sort.quickSort_func(0xc0021dd3b8, 0xc003f35aa0, 0x0, 0x2, 0x4)
	/usr/local/go/src/sort/zfuncversion.go:158 +0x1f6 fp=0xc0021dd1f0 sp=0xc0021dd198 pc=0x40a6856
sort.Slice(0x57d0d60, 0xc00521ec78, 0xc0021dd3b8)
	/usr/local/go/src/sort/slice.go:20 +0xe8 fp=0xc0021dd248 sp=0xc0021dd1f0 pc=0x40a4048
github.com/gohugoio/hugo/hugofs.(*Walkway).walk(0xc003f3d830, 0xc00010fae8, 0x9, 0x5f5b998, 0xc00521ec00, 0xc0026d2440, 0x2, 0x2, 0xc00280e380, 0x0, ...)
...
goroutine 375 [chan receive]:
github.com/gohugoio/hugo/hugolib.(*HugoSites).Build.func2(0xc0002ccc00, 0xc0004ffc20, 0xc0000505a0)
	/root/project/hugo/hugolib/hugo_sites_build.go:66 +0x7d
created by github.com/gohugoio/hugo/hugolib.(*HugoSites).Build
	/root/project/hugo/hugolib/hugo_sites_build.go:63 +0x230

goroutine 374 [chan send]:
github.com/gohugoio/hugo/hugolib.(*Site).renderPages.func1(0xc0008fc780, 0x25, 0xc000a6e720, 0xc001c2a100)
	/root/project/hugo/hugolib/site_render.go:84 +0xa7
github.com/gohugoio/hugo/hugolib.contentTrees.Walk.func1(0xc0008fc780, 0x25, 0x58669c0, 0xc000a6e720, 0xc0008fc700)
	/root/project/hugo/hugolib/content_map.go:877 +0x54
github.com/armon/go-radix.recursiveWalk(0xc000a6e750, 0xc001243a18, 0x5866900)
	/go/pkg/mod/github.com/armon/go-radix@v1.0.0/radix.go:519 +0xd7
github.com/armon/go-radix.recursiveWalk(0xc000a6e6c0, 0xc001243a18, 0x203000)
	/go/pkg/mod/github.com/armon/go-radix@v1.0.0/radix.go:525 +0x74
github.com/armon/go-radix.recursiveWalk(0xc000cf3bc0, 0xc001243a18, 0xc000bd9900)
	/go/pkg/mod/github.com/armon/go-radix@v1.0.0/radix.go:525 +0x74
github.com/armon/go-radix.recursiveWalk(0xc000907e30, 0xc001243a18, 0x0)
	/go/pkg/mod/github.com/armon/go-radix@v1.0.0/radix.go:525 +0x74
github.com/armon/go-radix.recursiveWalk(0xc00084f800, 0xc001243a18, 0xc000c2fa48)
	/go/pkg/mod/github.com/armon/go-radix@v1.0.0/radix.go:525 +0x74
github.com/armon/go-radix.(*Tree).Walk(...)
	/go/pkg/mod/github.com/armon/go-radix@v1.0.0/radix.go:447
github.com/gohugoio/hugo/hugolib.contentTrees.Walk(0xc000420a38, 0x3, 0x3, 0xc001243ab0)
	/root/project/hugo/hugolib/content_map.go:875 +0x68
github.com/gohugoio/hugo/hugolib.(*Site).renderPages(0xc0003746c0, 0xc000fad7c0, 0x0, 0xc000c2fa00)
	/root/project/hugo/hugolib/site_render.go:78 +0x209
github.com/gohugoio/hugo/hugolib.(*Site).render(0xc0003746c0, 0xc000fad7c0, 0x0, 0x0)
	/root/project/hugo/hugolib/site.go:1221 +0x59
github.com/gohugoio/hugo/hugolib.(*HugoSites).render(0xc0002ccc00, 0xc000c13f40, 0x6b76c80, 0x1)
	/root/project/hugo/hugolib/hugo_sites_build.go:320 +0x3d9
github.com/gohugoio/hugo/hugolib.(*HugoSites).Build.func5()
	/root/project/hugo/hugolib/hugo_sites_build.go:149 +0x3c
runtime/trace.WithRegion(0x5f4ffd8, 0xc00054e270, 0x5af8eda, 0x6, 0xc00128fe50)
	/usr/local/go/src/runtime/trace/annotation.go:141 +0xf8
github.com/gohugoio/hugo/hugolib.(*HugoSites).Build(0xc0002ccc00, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/root/project/hugo/hugolib/hugo_sites_build.go:151 +0x8a9
github.com/gohugoio/hugo/commands.(*commandeer).buildSites(...)
	/root/project/hugo/commands/hugo.go:722
github.com/gohugoio/hugo/commands.(*commandeer).fullBuild.func3(0xc0001aba98, 0x8)
	/root/project/hugo/commands/hugo.go:302 +0x7c
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc00054e240, 0xc000c090b0)
	/go/pkg/mod/golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:57 +0x59
created by golang.org/x/sync/errgroup.(*Group).Go
	/go/pkg/mod/golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:54 +0x66
error Command failed with exit code 2.

v0.84.0

Start building sites …
hugo v0.84.0-2C4689F7B+extended darwin/amd64 BuildDate=2021-06-18T17:14:27Z VendorInfo=gohugoio
fatal error: concurrent map writes

goroutine 77 [running]:
runtime.throw(0x5b355e6, 0x15)
	/usr/local/go/src/runtime/panic.go:1117 +0x72 fp=0xc002f8f0e0 sp=0xc002f8f0b0 pc=0x4039612
runtime.mapassign_faststr(0x5a7d4a0, 0xc00012e150, 0x5acaeb3, 0x4, 0x16)
	/usr/local/go/src/runtime/map_faststr.go:291 +0x3d8 fp=0xc002f8f148 sp=0xc002f8f0e0 pc=0x4016238
github.com/gohugoio/hugo/hugofs.(*Walkway).walk(0xc00351e510, 0xc0001183c8, 0x9, 0x5eff698, 0xc0025f9f38, 0xc002c8c820, 0x2, 0x2, 0xc002153940, 0x0, ...)
	/hugo/hugofs/walk.go:270 +0x37a fp=0xc002f8f318 sp=0xc002f8f148 pc=0x450075a
github.com/gohugoio/hugo/hugofs.(*Walkway).walk(0xc00351e510, 0x0, 0x0, 0x5eff698, 0xc0025f9d70, 0xc00236c480, 0x6, 0x6, 0xc002153940, 0x5f09938, ...)
	/hugo/hugofs/walk.go:298 +0x7ae fp=0xc002f8f4e8 sp=0xc002f8f318 pc=0x4500b8e
github.com/gohugoio/hugo/hugofs.(*Walkway).Walk(0xc00351e510, 0xc000263be0, 0x0)
	/hugo/hugofs/walk.go:137 +0x1c5 fp=0xc002f8f588 sp=0xc002f8f4e8 pc=0x44ffdc5
github.com/gohugoio/hugo/hugofs.Glob(0x5f068c0, 0xc000263be0, 0xc0017e6240, 0x22, 0xc002c8c4a0, 0xc0006ba660, 0xc002d30840)
	/hugo/hugofs/glob.go:77 +0x2cf fp=0xc002f8f6f0 sp=0xc002f8f588 pc=0x44f896f
github.com/gohugoio/hugo/resources/resource_factories/create.(*Client).match.func1(0xc002d30840, 0x38, 0x6b15420, 0xc002d30800, 0x38)
	/hugo/resources/resource_factories/create/create.go:107 +0xe5 fp=0xc002f8f760 sp=0xc002f8f6f0 pc=0x4d5cfa5
github.com/gohugoio/hugo/resources.(*ResourceCache).GetOrCreateResources.func1(0xc000142a00, 0xc002d30840, 0x38, 0x0)
	/hugo/resources/resource_cache.go:170 +0x26 fp=0xc002f8f7a8 sp=0xc002f8f760 pc=0x4a01ba6
github.com/gohugoio/hugo/resources.(*ResourceCache).getOrCreate(0xc000142a00, 0xc002d30840, 0x38, 0xc002f8f868, 0x0, 0x0, 0x0, 0x0)
	/hugo/resources/resource_cache.go:196 +0x173 fp=0xc002f8f828 sp=0xc002f8f7a8 pc=0x49f7433
github.com/gohugoio/hugo/resources.(*ResourceCache).GetOrCreateResources(0xc000142a00, 0xc002d30840, 0x38, 0xc002f8f920, 0x38, 0x3, 0xc00236a930, 0x3, 0x3)
...

github.com/gohugoio/hugo/hugolib.newPageContentOutput.func1(0x0, 0x0)
	/hugo/hugolib/page__per_output.go:104 +0x178
github.com/gohugoio/hugo/hugolib.newPageContentOutput.func2(0x5ef4420, 0xc003058d80, 0x0, 0x0, 0x0, 0x0)
	/hugo/hugolib/page__per_output.go:205 +0x26
github.com/gohugoio/hugo/lazy.(*Init).withTimeout.func1(0xc00116d320, 0x5ef4420, 0xc003058d80, 0xc003058de0)
	/hugo/lazy/init.go:176 +0x47
created by github.com/gohugoio/hugo/lazy.(*Init).withTimeout
	/hugo/lazy/init.go:175 +0x110
✨  Done in 6.85s.

I'm happy to provide more information and help debugging this, just let me know what is needed.

@bep bep added the Bug label Jul 13, 2021
@bep bep added this to the v0.86 milestone Jul 13, 2021
@bep bep self-assigned this Jul 13, 2021
@bep bep changed the title Regression: HUGO not sending non-exit code when build fails Not sending non-exit code when build fails Jul 13, 2021
@bep bep changed the title Not sending non-exit code when build fails Not sending non-exit code on panics Jul 13, 2021
@bep bep changed the title Not sending non-exit code on panics Not sending non-exit code on fatal errors Jul 13, 2021
@bep
Copy link
Member

@bep bep commented Jul 13, 2021

I'm having trouble reproducing the "exit code" issue (I suspected 04dc469 -- but that was added in 0.85). Every error situation I throw at it exits != 0. But I suspect I can fix the origin of this error. I'm a little puzzled why you're seeing this and not me (and others), and I don't understand the how, but I do understand the why, which I guess is good enough to fix it.

bep added a commit to bep/hugo that referenced this issue Jul 13, 2021
bep added a commit to bep/hugo that referenced this issue Jul 15, 2021
bep added a commit to bep/hugo that referenced this issue Jul 15, 2021
This commit started out investigating a `concurrent map read write` issue, ending by replacing the map with a struct.

This is easier to reason about, and it's more effective:

```
name                                  old time/op    new time/op    delta
SiteNew/Regular_Deep_content_tree-16    71.5ms ± 3%    69.4ms ± 5%    ~     (p=0.200 n=4+4)

name                                  old alloc/op   new alloc/op   delta
SiteNew/Regular_Deep_content_tree-16    29.7MB ± 0%    27.9MB ± 0%  -5.82%  (p=0.029 n=4+4)

name                                  old allocs/op  new allocs/op  delta
SiteNew/Regular_Deep_content_tree-16      313k ± 0%      303k ± 0%  -3.35%  (p=0.029 n=4+4)
```

See gohugoio#8749
bep added a commit to bep/hugo that referenced this issue Jul 15, 2021
This commit started out investigating a `concurrent map read write` issue, ending by replacing the map with a struct.

This is easier to reason about, and it's more effective:

```
name                                  old time/op    new time/op    delta
SiteNew/Regular_Deep_content_tree-16    71.5ms ± 3%    69.4ms ± 5%    ~     (p=0.200 n=4+4)

name                                  old alloc/op   new alloc/op   delta
SiteNew/Regular_Deep_content_tree-16    29.7MB ± 0%    27.9MB ± 0%  -5.82%  (p=0.029 n=4+4)

name                                  old allocs/op  new allocs/op  delta
SiteNew/Regular_Deep_content_tree-16      313k ± 0%      303k ± 0%  -3.35%  (p=0.029 n=4+4)
```

See gohugoio#8749
bep added a commit to bep/hugo that referenced this issue Jul 15, 2021
This commit started out investigating a `concurrent map read write` issue, ending by replacing the map with a struct.

This is easier to reason about, and it's more effective:

```
name                                  old time/op    new time/op    delta
SiteNew/Regular_Deep_content_tree-16    71.5ms ± 3%    69.4ms ± 5%    ~     (p=0.200 n=4+4)

name                                  old alloc/op   new alloc/op   delta
SiteNew/Regular_Deep_content_tree-16    29.7MB ± 0%    27.9MB ± 0%  -5.82%  (p=0.029 n=4+4)

name                                  old allocs/op  new allocs/op  delta
SiteNew/Regular_Deep_content_tree-16      313k ± 0%      303k ± 0%  -3.35%  (p=0.029 n=4+4)
```

See gohugoio#8749
bep added a commit to bep/hugo that referenced this issue Jul 15, 2021
This commit started out investigating a `concurrent map read write` issue, ending by replacing the map with a struct.

This is easier to reason about, and it's more effective:

```
name                                  old time/op    new time/op    delta
SiteNew/Regular_Deep_content_tree-16    71.5ms ± 3%    69.4ms ± 5%    ~     (p=0.200 n=4+4)

name                                  old alloc/op   new alloc/op   delta
SiteNew/Regular_Deep_content_tree-16    29.7MB ± 0%    27.9MB ± 0%  -5.82%  (p=0.029 n=4+4)

name                                  old allocs/op  new allocs/op  delta
SiteNew/Regular_Deep_content_tree-16      313k ± 0%      303k ± 0%  -3.35%  (p=0.029 n=4+4)
```

See gohugoio#8749
@bep bep removed this from the v0.86 milestone Jul 26, 2021
@bep bep added this to the v0.87 milestone Jul 26, 2021
@bep bep removed this from the v0.87 milestone Jul 30, 2021
@bep bep added this to the v0.88 milestone Jul 30, 2021
@bep bep removed this from the v0.88 milestone Sep 2, 2021
@bep bep added this to the v0.89 milestone Sep 2, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants