x/pkgsite: panic for some packages with syntax errors with locally running pkgsite

[mjl-]

What is the URL of the page with the issue?

http://localhost:8080/github.com/mjl-/mox

What is your user agent?

firefox (not relevant)

Screenshot

The connection was reset
The connection to the server was reset while the page was loading.

What did you do?

To reproduce, create a local directory somewhere, with two files (the form of the module path seems relevant):

$ cat go.mod
module github.com/mjl-/mox

go 1.24.2

Malformed go file:

$ cat test.go 
package test
bad

Run latest pkgsite:

go run -trimpath golang.org/x/pkgsite/cmd/pkgsite@v0.0.0-20250414131012-01dc430a4e83

Open http://localhost:8080/github.com/mjl-/mox

What did you see happen?

An error message about the invalid package.

What did you expect to see?

An aborted HTTP request due to an unhandled panic of a nil pointer dereference:

2025/04/14 20:58:29 Warning: fetching url from deps.dev: Get "https://deps.dev/_/s/go/p/github.com%2Fmjl-%2Fmox/v/v0.0.0/exists": context canceled
2025/04/14 20:58:29 http: panic serving 127.0.0.1:55874: runtime error: invalid memory address or nil pointer dereference
goroutine 697 [running]:
net/http.(*conn).serve.func1()
        net/http/server.go:1947 +0xbe
panic({0xb84260?, 0x1708220?})
        runtime/panic.go:792 +0x132
golang.org/x/pkgsite/internal/fetchdatasource.(*FetchDataSource).populateUnitSubdirectories(...)
        golang.org/x/pkgsite@v0.0.0-20250414131012-01dc430a4e83/internal/fetchdatasource/fetchdatasource.go:161
golang.org/x/pkgsite/internal/fetchdatasource.(*FetchDataSource).findUnit(0xc0002f8fc0, {0x124a100?, 0xc000333ce0?}, 0xc000486000, {0xc000416000?, 0xc4cf01?})
        golang.org/x/pkgsite@v0.0.0-20250414131012-01dc430a4e83/internal/fetchdatasource/fetchdatasource.go:233 +0x65
golang.org/x/pkgsite/internal/fetchdatasource.(*FetchDataSource).GetUnit(0xc0002f8fc0, {0x124a100, 0xc000333ce0}, 0xc0003485a0, 0x49dd59?, {{0x0?, 0xc00087abb0?}, {0x0?, 0x49dd9e?}})
        golang.org/x/pkgsite@v0.0.0-20250414131012-01dc430a4e83/internal/fetchdatasource/fetchdatasource.go:214 +0x17f
golang.org/x/pkgsite/internal/frontend.fetchMainDetails({0x124a100, 0xc000333ce0}, {0x124d620, 0xc0002f8fc0}, 0xc0003485a0, {0xc4ced7, 0x6}, 0x0, {{0x0, 0x0}, ...})
        golang.org/x/pkgsite@v0.0.0-20250414131012-01dc430a4e83/internal/frontend/main.go:117 +0x16d
golang.org/x/pkgsite/internal/frontend.fetchDetailsForUnit({0x124a100, 0xc000333ce0}, 0xc00040d040, {0x0, 0x0}, {0x124d620, 0xc0002f8fc0}, 0xc0003485a0, {0xc4ced7, 0x6}, ...)
        golang.org/x/pkgsite@v0.0.0-20250414131012-01dc430a4e83/internal/frontend/tabs.go:87 +0x305
golang.org/x/pkgsite/internal/frontend.(*Server).serveUnitPage(0xc00079ca20, {0x124a100, 0xc000333ce0}, {0x1248878, 0xc000712380}, 0xc00040d040, {0x124d620, 0xc0002f8fc0}, 0xc002746510)
        golang.org/x/pkgsite@v0.0.0-20250414131012-01dc430a4e83/internal/frontend/unit.go:147 +0x3a8
golang.org/x/pkgsite/internal/frontend.(*Server).serveDetails(0xc00079ca20, {0x1248878, 0xc000712380}, 0xc00040d040, {0x124d620, 0xc0002f8fc0})
        golang.org/x/pkgsite@v0.0.0-20250414131012-01dc430a4e83/internal/frontend/details.go:72 +0x691
golang.org/x/pkgsite/internal/frontend.(*Server).Install.(*Server).errorHandler.func6({0x1248878, 0xc000712380}, 0xc00040d040)
        golang.org/x/pkgsite@v0.0.0-20250414131012-01dc430a4e83/internal/frontend/server.go:504 +0x6c
net/http.HandlerFunc.ServeHTTP(0xc0000d83c0?, {0x1248878?, 0xc000712380?}, 0x6e9b4f?)
        net/http/server.go:2294 +0x29
net/http.(*ServeMux).ServeHTTP(0xc00040cf00?, {0x1248878, 0xc000712380}, 0xc00040d040)
        net/http/server.go:2822 +0x1c4
main.main.main.Timeout.func3.func4({0x1248878, 0xc000712380}, 0xc00040cf00)
        golang.org/x/pkgsite@v0.0.0-20250414131012-01dc430a4e83/internal/middleware/timeout/timeout.go:20 +0x94
net/http.HandlerFunc.ServeHTTP(0x476739?, {0x1248878?, 0xc000712380?}, 0xc001807b70?)
        net/http/server.go:2294 +0x29
net/http.serverHandler.ServeHTTP({0xc002746420?}, {0x1248878?, 0xc000712380?}, 0x6?)
        net/http/server.go:3301 +0x8e
net/http.(*conn).serve(0xc0000e73b0, {0x124a058, 0xc0008aae70})
        net/http/server.go:2102 +0x625
created by net/http.(*Server).Serve in goroutine 1
        net/http/server.go:3454 +0x485

The nil pointer dereference is at https://github.com/golang/pkgsite/blob/01dc430a4e8339c44b283f075264c5c5dd47621c/internal/fetchdatasource/fetchdatasource.go#L161

The nil u is coming from https://github.com/golang/pkgsite/blob/01dc430a4e8339c44b283f075264c5c5dd47621c/internal/fetchdatasource/fetchdatasource.go#L233 (nil unit)

[gabyhelp]

Related Issues

• x/pkgsite: panic in localdatasource #44704 (closed)
• x/pkgsite: panic when running in direct_proxy mode #42886 (closed)
• x/pkgsite: Server.queue nil panic using @master #54478 (closed)
• x/tools/gopls: crashed when parsing invalid URL #37882 (closed)
• x/tools/gopls: automated issue report (crash) #40449 (closed)
• x/tools/gopls: RenderPkgDoc crashes on go.starlark.net/starlark #67287 (closed)
• x/net/http: panic when route ends with '/' #36275 (closed)
• x/net/http2: closeStream has panic #59601 (closed)
• Panic when trying to serve http with embedded assets #43761 (closed)
• x/pkgsite/cmd/pkgsite: panic for "multiple registrations" when running locally #64903 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[adonovan]

@jba

[gopherbot]

Change https://go.dev/cl/665638 mentions this issue: internal/fetchdatasource: check for error earlier