You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
proposal: x/tools/go/analysis/passes/shadow: deprecate it #75342
Background: The shadow analyzer aims to report mistakes related to shadowing of local variables, which is a not infrequent source of mistakes in Go. However, the shadow analyzer has at no point in its history gotten remotely close to the signal-to-noise ratio we expect of our static analysis tools, and that's why it was never enabled by default in the old cmd/vet, nor even included in the vet suite once vet was modularized using the Go analysis framework. Yet we still receive occasional CLs trying to improve the algorithm, and they take time to review.
If anyone wishes to develop their own shadow analyzer, either based on our code or from whole cloth, they are free to do so and we wish them luck; and if they succeed at eliminating false positives (for example, demonstrating a <5% false positive rate on the goalng.org/x repos), we could discuss replacing the current implementation and perhaps even adding it to vet. Until then:
Proposal: We propose to formally deprecate the shadow analyzer on the grounds that it is a failed experiment, and to summarily reject CLs to incrementally improve its algorithm.
I guess this qualifies as a Proposal, since we typically treat deprecations as API changes (albeit reversible ones).
Fully agree. We have somewhat different standards for gopls, where hint level diagnostics are less prominent, and even there it is disabled by default.
Does that also mean removal from gopls? I have this analyzer enabled all the time in gopls. I don’t really use it in the sense of renaming variables based on its results, but rather as a hint, so I can spot such cases more easily while debugging. I’m fine with all the false positives.
The description of the analyzer mentions that "check for possible unintended shadowing of variables", which i have been not aware of. I thought that it reports all shadowed variables.
Perhaps an inlay hint (similar to the recently added ignoredError) would have been more helpful in such cases.
That was my intent, yes; I should have been more explicit.
I have this analyzer enabled all the time in gopls. I don’t really use it in the sense of renaming variables based on its results, but rather as a hint, so I can spot such cases more easily while debugging. I’m fine with all the false positives.
Huh, interesting. And it's not even configured at level protocol.SeverityHint as it should have been, so presumably it pollutes the VS Code problems pane with non-problems?
I have this analyzer enabled all the time in gopls. I don’t really use it in the sense of renaming variables based on its results, but rather as a hint, so I can spot such cases more easily while debugging. I’m fine with all the false positives.
I also have it enabled all the time, it's more distracting when browsing other people's code. For my own code, I'm only annoyed by the warnings for shadowing err.
If anyone wishes to develop their own shadow analyzer, either based on our code or from whole cloth, they are free to do so and we wish them luck; and if they succeed at eliminating false positives (for example, demonstrating a <5% false positive rate on the goalng.org/x repos), we could discuss replacing the current implementation and perhaps even adding it to vet.
I do have the one I wrote for typescript-go (explainer here) that I was hoping to upstream or similar, but I have not run it on a larger corpus. In our codebase, it found a lot of bugs and very few false positives.
I don't know if it meets the bar, though; how do I know if it's meeting the 5% rate?
Thanks, I get what you mean now. Mine is probably still on the noisy side. Maybe a couple of potential bugs, but the rest are just code that could be unclear (that I would probably have used new names for).
I have an updated version based on the usage logic discussed in the other PR. Shadowing is now reported when an inner variable is assigned but not used in the inner scope, while the outer variable is used afterwards.
I've run the analyzer on a number of repositories. So far, unless I'm missing something, there are only 2 reports, and these appear to be actually problematic cases:
https://github.com/golang/tools: internal/gcimporter/gcimporter.go:83:6: declaration of "err" shadows declaration at line 47
It's true that there are two err variables, the inner shadowing the outer, but the return statement is essentially a copy from the inner to the outer. This pattern is pretty common; it's not a bug.
https://github.com/golang/tools:
internal/gcimporter/gcimporter.go:83:6: declaration of "err" shadows declaration at line 47
I'll admit that the code is a little tricky, but I don't see any bugs here.
I think when defer func() will run it will take shadowing err (which is nil - otherwise it would return at 85) instead of top level err from line 98 or 105. So successfully open filename won't be prepended to the error.
It's true that there are two err variables, the inner shadowing the outer, but the return statement is essentially a copy from the inner to the outer. This pattern is pretty common; it's not a bug.
Here I think there is also at line 139 case similar to gcimporter.go where defer will use/update shadowing err.
I think when defer func() will run it will take shadowing err (which is nil - otherwise it would return at 85) instead of top level err from line 98 or 105. So successfully open filename won't be prepended to the error.
Ah, but this is the magic of defer and named result variables: when the return statement is executed, it augments err (fmt.Errorf(...)) and updates the named result variable. Then the deferred function runs, and it reads the named result variable again, augments the error, and updates the variable again. It is tricky, but I don't think it's incorrect.
Oh, you're absolutely right: in both cases the defer is operating on the inner (shadowing) var, not on the outer name result variable, so, as you say, it will never have any observable effect.
My apologies for not reading more closely. I think your analyzer has found two very subtle true positives. Well done, this is real progress. I'll run it over the corpus and see what else it turns up.
I ran it over the corpus and found 4100 matches, of which a random 50 are shown here. Of those, I sampled that first 10, and found that 8 were clear errors; the remaining two were not necessarily correctness issues but appeared in code that was clearly confused. I'll study another batch in more detail, and post the complete results in due course, though unfortunately our pipeline is in the middle of some upgrades at the moment. Promising results.
I think it might improve precision to ignore shadowing of package-level declarations.
Great, thanks for the update. I think checking that the shadowed identifier is a variable could help illuminate issues like shadowing of "type mapping".
Also, handling naked returns as outer variable usage could be added for cases like this:
Activity
[-]x/tools/go/analysis/passes/shadow: deprecate it[/-][+]Proposal: x/tools/go/analysis/passes/shadow: deprecate it[/+]findleyr commentedon Sep 9, 2025
I guess this qualifies as a Proposal, since we typically treat deprecations as API changes (albeit reversible ones).
Fully agree. We have somewhat different standards for gopls, where hint level diagnostics are less prominent, and even there it is disabled by default.
gabyhelp commentedon Sep 9, 2025
Related Issues
Related Documentation
(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)
mateusz834 commentedon Sep 10, 2025
Does that also mean removal from gopls? I have this analyzer enabled all the time in gopls. I don’t really use it in the sense of renaming variables based on its results, but rather as a hint, so I can spot such cases more easily while debugging. I’m fine with all the false positives.
The description of the analyzer mentions that "check for possible unintended shadowing of variables", which i have been not aware of. I thought that it reports all shadowed variables.
Perhaps an inlay hint (similar to the recently added
ignoredError) would have been more helpful in such cases.[-]Proposal: x/tools/go/analysis/passes/shadow: deprecate it[/-][+]proposal: x/tools/go/analysis/passes/shadow: deprecate it[/+]adonovan commentedon Sep 10, 2025
That was my intent, yes; I should have been more explicit.
Huh, interesting. And it's not even configured at level protocol.SeverityHint as it should have been, so presumably it pollutes the VS Code problems pane with non-problems?
mateusz834 commentedon Sep 10, 2025
No idea how it is displayed in VS Code, in
neovimi see it being displayed as warnings.mateusz834 commentedon Sep 10, 2025
Idea to make that use-case better: #75368
findleyr commentedon Sep 10, 2025
We should at the very least make shadow hint-level in gopls. IMO that provides the best UX in VS Code (it is, after all, a hint).
gopherbot commentedon Sep 10, 2025
Change https://go.dev/cl/702537 mentions this issue:
gopls/internal/settings: shadow.severity = hintseankhliao commentedon Sep 10, 2025
I also have it enabled all the time, it's more distracting when browsing other people's code. For my own code, I'm only annoyed by the warnings for shadowing err.
also using neovim, with https://git.sr.ht/~whynothugo/lsp_lines.nvim so the warnings take up an event line (motivation to not shadow variables in my own code).
gopls/internal/settings: shadow.severity = hint
jakebailey commentedon Sep 10, 2025
I do have the one I wrote for typescript-go (explainer here) that I was hoping to upstream or similar, but I have not run it on a larger corpus. In our codebase, it found a lot of bugs and very few false positives.
I don't know if it meets the bar, though; how do I know if it's meeting the 5% rate?
adonovan commentedon Sep 10, 2025
Run it on a random large repo and inspect a random 20 of the diagnostics. If only about one is false, you're doing well.
jakebailey commentedon Sep 10, 2025
Thanks, I get what you mean now. Mine is probably still on the noisy side. Maybe a couple of potential bugs, but the rest are just code that could be unclear (that I would probably have used new names for).
bkreitch commentedon Sep 26, 2025
I have an updated version based on the usage logic discussed in the other PR. Shadowing is now reported when an inner variable is assigned but not used in the inner scope, while the outer variable is used afterwards.
I've run the analyzer on a number of repositories. So far, unless I'm missing something, there are only 2 reports, and these appear to be actually problematic cases:
https://github.com/golang/tools:
internal/gcimporter/gcimporter.go:83:6: declaration of "err" shadows declaration at line 47https://github.com/prometheus/prometheus:
cmd/promtool/rules.go:118:18: declaration of "err" shadows declaration at line 101And it passes simple synthetic tests.
It still needs some optimizations and additions, but would the general functionality be acceptable?
adonovan commentedon Sep 26, 2025
FWIW, this one is a false positive:
I'll admit that the code is a little tricky, but I don't see any bugs here.
This one also doesn't seem problematic to me:
It's true that there are two err variables, the inner shadowing the outer, but the return statement is essentially a copy from the inner to the outer. This pattern is pretty common; it's not a bug.
bkreitch commentedon Sep 26, 2025
I think when defer func() will run it will take shadowing err (which is nil - otherwise it would return at 85) instead of top level err from line 98 or 105. So successfully open filename won't be prepended to the error.
Here I think there is also at line 139 case similar to gcimporter.go where defer will use/update shadowing err.
adonovan commentedon Sep 26, 2025
Ah, but this is the magic of defer and named result variables: when the return statement is executed, it augments err (fmt.Errorf(...)) and updates the named result variable. Then the deferred function runs, and it reads the named result variable again, augments the error, and updates the variable again. It is tricky, but I don't think it's incorrect.
bkreitch commentedon Sep 26, 2025
I seem to get different results when it's shadowed and when it's not:
Gives:
adonovan commentedon Sep 26, 2025
Oh, you're absolutely right: in both cases the defer is operating on the inner (shadowing) var, not on the outer name result variable, so, as you say, it will never have any observable effect.
My apologies for not reading more closely. I think your analyzer has found two very subtle true positives. Well done, this is real progress. I'll run it over the corpus and see what else it turns up.
adonovan commentedon Sep 30, 2025
I ran it over the corpus and found 4100 matches, of which a random 50 are shown here. Of those, I sampled that first 10, and found that 8 were clear errors; the remaining two were not necessarily correctness issues but appeared in code that was clearly confused. I'll study another batch in more detail, and post the complete results in due course, though unfortunately our pipeline is in the middle of some upgrades at the moment. Promising results.
I think it might improve precision to ignore shadowing of package-level declarations.
https://go-mod-viewer.appspot.com/github.com/uvalib/orcid-access-ws@v0.0.0-20250612130209-7d062dbabf9d/orcidaccessws/dao/db-datastore.go#L115: "declaration of ""err"" shadows declaration at line 87"
https://go-mod-viewer.appspot.com/github.com/nwidger/bat@v0.0.2/httplib/httplib.go#L467: "declaration of ""err"" shadows declaration at line 458"
https://go-mod-viewer.appspot.com/github.com/msinev/pmc@v0.0.0-20220512222710-eb40bf080315/pkg/uploader/xform.go#L192: "declaration of ""err"" shadows declaration at line 181"
https://go-mod-viewer.appspot.com/github.com/cwntr/go-defi@v0.0.0-20210629134751-07f9ec2f7e66/client/client.go#L990: "declaration of ""err"" shadows declaration at line 980"
https://go-mod-viewer.appspot.com/github.com/maikovskiys/l1tasks@v0.0.0-20230927052451-6436d7687dc9/learn/dev12/main.go#L11: "declaration of ""n"" shadows declaration at line 9"
https://go-mod-viewer.appspot.com/github.com/brioux/go-keycloak@v0.0.0-20240929191119-b54a3a01d90b/keycloak.go#L230: "declaration of ""err"" shadows declaration at line 213"
https://go-mod-viewer.appspot.com/github.com/andybalholm/giopdf@v0.0.0-20220317170119-aad9a095ad48/font.go#L324: "declaration of ""err"" shadows declaration at line 300"
https://go-mod-viewer.appspot.com/github.com/nizigama/simple-note@v0.0.0-20210501235539-2cc762e06fe8/services/database/bolt.go#L291: "declaration of ""err"" shadows declaration at line 15"
https://go-mod-viewer.appspot.com/github.com/informationsea/shellflow@v0.1.3/viewlog.go#L230: "declaration of ""err"" shadows declaration at line 227"
https://go-mod-viewer.appspot.com/github.com/pr2apisdk/testgo/sdk@v0.0.0-20240626131719-36a9d76518b2/sdk.go#L398: "declaration of ""err"" shadows declaration at line 359"
https://go-mod-viewer.appspot.com/github.com/10ego/gthp@v0.0.0-20241025155251-e1514fa71fbb/internal/templ/templates/index_templ.go#L33: "declaration of ""ctx"" shadows declaration at line 13"
https://go-mod-viewer.appspot.com/github.com/!aspie!soft/goutil/v6@v6.0.1/compress.go#L27: "declaration of ""q"" shadows declaration at line 25"
https://go-mod-viewer.appspot.com/github.com/!youth!camp!proj/douyin@v0.0.0-20220619060819-b7c3e9420930/service/feed.go#L23: "declaration of ""err"" shadows declaration at line 20"
https://go-mod-viewer.appspot.com/github.com/ngsalvo/datastar-rick-and-morty@v0.0.0-20250318143428-01c52f047cfc/web/pages/home_templ.go#L35: "declaration of ""ctx"" shadows declaration at line 15"
https://go-mod-viewer.appspot.com/github.com/ars0915/glossika-exercise@v0.0.0-20240904180321-a5617a00d892/pkg/db/db.go#L31: "declaration of ""err"" shadows declaration at line 14"
https://go-mod-viewer.appspot.com/github.com/zartbot/goflow@v0.0.0-20221001080520-456f7dfad288/service/optiontemplatemap/option_template_viptela_tloc.go#L174: "declaration of ""err"" shadows declaration at line 129"
https://go-mod-viewer.appspot.com/github.com/germla/auth@v0.0.0-20230505173936-bc9a4159cf17/providers/password/confirm.go#L82: "declaration of ""err"" shadows declaration at line 74"
https://go-mod-viewer.appspot.com/github.com/systematiccaos/gorm@v1.22.6/schema/relationship.go#L417: "declaration of ""primaryFields"" shadows declaration at line 358"
https://go-mod-viewer.appspot.com/github.com/jasoet/gopki@v1.15.0/signing/signer.go#L347: "declaration of ""err"" shadows declaration at line 343"
https://go-mod-viewer.appspot.com/github.com/alexander-fernandez/go-group@v0.0.0-20220317235734-2d15c54e6f5f/UtmStack-Setup-Linux.go#L775: "declaration of ""err"" shadows declaration at line 381"
https://go-mod-viewer.appspot.com/bitbucket.org/lygo/lygo_ext_dbbolt@v0.1.3/lygo_db_bolt_collection.go#L307: "declaration of ""err"" shadows declaration at line 299"
https://go-mod-viewer.appspot.com/github.com/brioux/go-keycloak@v0.0.0-20240929191119-b54a3a01d90b/keycloak.go#L220: "declaration of ""err"" shadows declaration at line 213"
https://go-mod-viewer.appspot.com/github.com/tanqiangyes/govalidator@v0.0.5/converter.go#L46: "declaration of ""err"" shadows declaration at line 28"
https://go-mod-viewer.appspot.com/github.com/miguelangel-nubla/oauth2commander@v0.3.7/oauth2server/command.go#L49: "declaration of ""err"" shadows declaration at line 47"
https://go-mod-viewer.appspot.com/github.com/cwntr/go-defi@v0.0.0-20210629134751-07f9ec2f7e66/client/client.go#L648: "declaration of ""err"" shadows declaration at line 634"
https://go-mod-viewer.appspot.com/github.com/korsakjakub/snk@v0.0.0-20230625212658-1d6c119c67ee/main.go#L27: "declaration of ""game"" shadows declaration at line 18 in game.go"
https://go-mod-viewer.appspot.com/github.com/topi314/pogo-icons@v0.0.0-20250403082451-64aa0bf0322a/pogoicons/commands.go#L213: "declaration of ""err"" shadows declaration at line 168"
https://go-mod-viewer.appspot.com/github.com/litongjava/hfile@v0.0.0-20250829082833-a538817521ac/main.go#L98: "declaration of ""repoDir"" shadows declaration at line 78"
https://go-mod-viewer.appspot.com/github.com/cwntr/go-defi@v0.0.0-20210629134751-07f9ec2f7e66/client/client.go#L1003: "declaration of ""err"" shadows declaration at line 980"
https://go-mod-viewer.appspot.com/github.com/qinghon/debinterface@v0.0.6/Interfaces.go#L155: "declaration of ""err"" shadows declaration at line 113"
https://go-mod-viewer.appspot.com/github.com/cwntr/go-defi@v0.0.0-20210629134751-07f9ec2f7e66/client/client.go#L781: "declaration of ""err"" shadows declaration at line 764"
https://go-mod-viewer.appspot.com/github.com/agrigoryan/aoc_2023_go@v0.0.0-20231216221323-4ace361ec685/day5/d5p1.go#L37: "declaration of ""mapping"" shadows declaration at line 19"
https://go-mod-viewer.appspot.com/github.com/timoth-y/kicksware-api/order-service@v0.0.0-20201002192818-87b546a7ae5a/api/rest/handler.go#L57: "declaration of ""err"" shadows declaration at line 53"
https://go-mod-viewer.appspot.com/github.com/cwntr/go-defi@v0.0.0-20210629134751-07f9ec2f7e66/client/client.go#L774: "declaration of ""err"" shadows declaration at line 764"
https://go-mod-viewer.appspot.com/github.com/predbdotovh/pre-api@v0.0.0-20211230223408-096f265fde6a/handlers.go#L85: "declaration of ""err"" shadows declaration at line 72"
https://go-mod-viewer.appspot.com/github.com/dallinallred/goftp@v0.0.0-20240626003405-61936bc079f1/server/ftpServer.go#L70: "declaration of ""err"" shadows declaration at line 47"
https://go-mod-viewer.appspot.com/github.com/meowstercatel/discordgo@v0.0.0-20250615204139-29b43d676735/state.go#L1072: "declaration of ""err"" shadows declaration at line 954"
https://go-mod-viewer.appspot.com/github.com/!chronostasys/calc@v0.0.1/compiler/ast/function.go#L786: "declaration of ""i"" shadows declaration at line 869 in ast.go"
https://go-mod-viewer.appspot.com/github.com/!jack-!z!l/go_rookie@v1.0.40/rpc/tcp.go#L332: "declaration of ""err"" shadows declaration at line 319"
https://go-mod-viewer.appspot.com/github.com/hashicorp/golang-math-big@v0.0.0-20180316142257-561262b71329/big/prime.go#L289: "declaration of ""t2"" shadows declaration at line 252"
https://go-mod-viewer.appspot.com/github.com/msinev/pmc@v0.0.0-20220512222710-eb40bf080315/pkg/uploader/result.go#L188: "declaration of ""err"" shadows declaration at line 177"
https://go-mod-viewer.appspot.com/github.com/omnigres/cli@v0.1.4/cmd/run.go#L37: "declaration of ""err"" shadows declaration at line 26"
https://go-mod-viewer.appspot.com/github.com/aerth/userinfo@v0.0.0-20160917202326-6c837af3a47b/userinfo.go#L64: "declaration of ""err"" shadows declaration at line 29"
https://go-mod-viewer.appspot.com/github.com/krzko/gosh@v0.0.0-20241128233202-5abd4ef06633/internal/shell/builtins/ls.go#L124: "declaration of ""err"" shadows declaration at line 112"
https://go-mod-viewer.appspot.com/github.com/gorocks/apikit@v0.0.0-20170808011444-396ab031afbd/apikit.go#L119: "declaration of ""err"" shadows declaration at line 108"
https://go-mod-viewer.appspot.com/github.com/googollee/clic@v0.1.5/source/clone_test.go#L60: "declaration of ""a3"" shadows declaration at line 10 in source_fields_test.go"
https://go-mod-viewer.appspot.com/github.com/tacerus/ldifdiff@v0.0.0-20181030102753-4dccbe38183b/input_test.go#L24: "declaration of ""entries"" shadows declaration at line 26 in types.go"
https://go-mod-viewer.appspot.com/github.com/nesv/gothub@v0.0.0-20140406181320-f0c824044d5f/gothub.go#L258: "declaration of ""err"" shadows declaration at line 247"
https://go-mod-viewer.appspot.com/github.com/googollee/clic@v0.1.5/source/clone_test.go#L60: "declaration of ""a2"" shadows declaration at line 10 in source_fields_test.go"
https://go-mod-viewer.appspot.com/github.com/junaozun/protoc-gen-go-clone@v0.0.0-20231229025521-89199567f720/generator/ext.go#L39: "declaration of ""err"" shadows declaration at line 35"
bkreitch commentedon Oct 1, 2025
Great, thanks for the update. I think checking that the shadowed identifier is a variable could help illuminate issues like shadowing of "type mapping".
Also, handling naked returns as outer variable usage could be added for cases like this: