There are probably other places in the stdlib, as well as in 3rd party packages, where mutating a struct field/package level variable, intended only for reading, will create mess. That's why I think some //go:readonly or similar annotation considered by go vet might be of more general use.
On the surface, a vet check is plausible. On the other hand, if you're new enough to Go to consider setting b.N then you are likely to not even know about vet. I think the best vet checks are those that will catch things that even experienced Go programmers can make.