x/tools/go/analysis: identify test-only packages

[lfolger]

Proposal Details

Some analyzers need to customize their behavior based on the context in which the code is running, e.g. to restrict certain (inefficient) APIs to only be used in tests.

At the moment analyzers have no way to determine if the file they are analyzing is only used in tests. One way of achieving this today is to check if the file ends in _test.go. This works for the most part. However it fails for libraries that are not tests themselves but can only be used in tests. Some build systems can enforce this. See bazels testonly attribute.

One way to support this would be similar to the version support based on go/types.Info.FileVersions which is a map from *ast.File to the version string. A map in the analysis.Pass from *ast.File to a bollean that says if the file is only used in tests or not would solve this problem.

Note: it is not a property of the package to be analyzed but the individual files because when analyzing a test package it consists of test files and the actual package source files.

[lfolger]

cc: @adonovan @timothy-king @findleyr

[adonovan]

This feature would first need to be added to x/tools/go/packages, which the multichecker driver uses for package information. So let's expand this proposal to consider the API changes to both packages at once.

[timothy-king]

(Reopening. Fat fingers closed this.)

[timothy-king]

3 candidate implementations:

1. Add a field Test map[*ast.File]bool to analysis.Pass and packages.Package (original proposal)
2. Add a field Test func(*ast.File)bool to analysis.Pass and packages.Package
3. Add a field Tests []*ast.File to analysis.Pass and packages.Package

Thoughts:

• 1 and 2 both strike me as user friendly for analyzer writers.
• I have a sneaking suspicion that packages will be slightly more natural to express with 3. Packages can be given a list of the testing files and forwards it along.
• 3 might be slightly inefficient if analyzers do O(|#test files||#files||#analyzers|*|#packages|) to check these, but this is easily to fix with a utility Analyzer if this becomes a problem.
• Options 2 strikes me as not a natural fit for packages.
• I do not see a particularly compelling reason to work over strings instead of *ast.File (packages has a few places strings appear though).

[adonovan]

A utility analyzer is overkill; we should just get the API right. The client needs to know "is this file a test file?" or, more often, "is this node or position within a test file?", even when they don't have the File to hand. The API should support those queries directly.

I suspect the frequency of queries will not be very great: typically once per file (if deciding to skip some files) or once per finding (if deciding whether to suppress a diagnostic). So this suggests a representation like option #3, with a helper function such as:

// InTestFile reports whether the position is within
// (including EOF) a file that is only used during testing.
func (pass *Pass) InTestFile(pos token.Pos) bool {
    for _, f := range pass.TestFiles {
        if f.FileStart <= pos && pos <= f.FileEnd { return true }
    }
    return false
}

Clients can build a hash table if they need more frequent access.

[lfolger]

I might be missing something but is there a reason this needs to be part of the analysis.Pass at all?
Is this to support files that are not part of the package under active analysis?
Otherwise it should be enough to have this information in the types.Package or types.Info which is always available for the package under analysis (even for packages that fail type checking it should be possible to fill in this data).

Or going the other direction. Does it need to be part of the types.Package or types.Info at all? Wouldn't it be sufficient to be part of the analysis.Pass only? (It might be hard for analysis drivers to get at this information).

I think what I'm trying to say is: is there a reason to duplicate this information?