proposal: all: add mechanism to remove API as a function of caller's go.mod "go 1.xx" version

[bradfitz]

When reviewing a number of the Go2 issues, we keep finding that we're basically unable to clean up past mistakes.

On the language side, the go.mod file's "go 1.xxx" declaration lets users declare their expected Go language semantics, which lets us remove Go language features over time, but we have nothing equivalent for removing standard library symbols, short of simply making new major versions of all packages, which gets contagiously invasive very quickly with the dependencies between the std packages. For redesigns, new major versions works, but it doesn't work well for cleanups.

It would be nice to have a mechanism that's similar (but different) to +build go1.x build tags, but are added implicitly by the caller module module's expected language version.

That'd let a package have different exports depending on who was using it.

Rather than propose an exact solution, this bug is more generally about tracking how (and whether) we can remove things from a package over time.

/cc @ianlancetaylor @griesemer @bcmills

[bcmills]

I don't think this is a problem that actually needs to be solved for package-level identifiers (types, constants, and functions). For those, we still have to provide an implementation anyway, and given that we have that implementation, we may as well just add a // Deprecated comment, and make lint and godoc warn everyone not to use it.

The interesting problem, then, is how to remove methods, because they aren't necessarily visible at the point where they are used.

Suppose I have:

package mine

type Thing struct{ […] }

// Deprecated: use NewThing instead.
func (t *Thing) OldThing() {}

func (t *Thing) NewThing() {}

package mediator

import (
	[…]
)

func […] {
	// This looks like it avoids the deprecated OldThing method...
	var t theirs.NewThinger = new(mine.Thing)
	theirs.Oops(t)
}

package theirs

type NewThinger interface { NewThing() }

func Oops(t NewThinger) {
	if x, ok := t.(interface { OldThing() }); ok {
		// ...but we would need whole-program analysis
		// to realize that OldThing is still in use.
		x.OldThing()
	} else {
		t.NewThing()
	}
}

[bcmills]

I have some ideas about how to address this, but I don't have the time to write them up this week. 😕

[ianlancetaylor]

@bcmills Any ideas on mechanism? Thanks.

[ianlancetaylor]

If we use our standard fallback of a magic comment, then it could look like:

//go:lang < 1.14
type ClientConn struct {
}

This means that ClientConn is normally exported, but it can not be seen when compiling at Go language level 1.14 or higher. This can also apply to struct fields.

type Request struct {
    ...
    //go:lang < 1.14
    Cancel <-chan struct{}
    ...
}

The type Request is visible at all language levels, but it can not be seen at language level 1.14 or higher. (The struct field would still exist at run time and would still show up when using the reflect package.)

The magic comment could be used with package scope types, consts, variables, functions, and methods, and also with fields of struct types and methods of interface types.

It could be used to cover an entire set of types, consts, or variables.

//go:lang < 1.14
const (
    SEEK_SET int = 0
    SEEK_CUR int = 1
    ...
)

In this example SEEK_SET and SEEK_CUR (and any other constants in the group) can not be seen at language level 1.14 or above.

[bcmills]

Yes, that's the thing that works for everything but methods — but it's also mostly redundant with // Deprecated comments.

The problem is, what happens if you do a type-assertion that checks for the removed method in a pre-1.14 module, and then pass the value as an interface{} to some other package that has since been upgraded? When the second package does the same type-assertion it will presumably fail, unless that method has somehow been marked as contaminated by the pre-1.14 package.

[bradfitz]

but it's also mostly redundant with // Deprecated comments.

Deprecated comments have no machine-readable structure, though. Parsing English sentences sounds super gross.

[bcmills]

Why is machine-readable structure important? Either way, the content is merely informational: it's nearly trivial to construct a “backdated” forwarding module to access the removed definitions.

[bradfitz]

Either way, the content is merely informational

That's not true. The whole point of this bug is to remove access to symbols from people. It's not to write English at them. We can already write English at them in comments.

[ianlancetaylor]

The problem is, what happens if you do a type-assertion that checks for the removed method in a pre-1.14 module, and then pass the value as an interface{} to some other package that has since been upgraded? When the second package does the same type-assertion it will presumably fail, unless that method has somehow been marked as contaminated by the pre-1.14 package.

The suggestion is that, if T.M is marked as removed in 1.14, then direct references to T.M will fail. And storing a value of type T in an interface that requires method M will fail. However, storing a value of type T in an interface, and then asserting to a different interface that checks for M will succeed. It may be that this inconsistency is enough to decide against this proposal, or at least to decide against permitting it to be used for methods, but I don't think there is any other plausible approach.

[bcmills]

The whole point of this bug is to remove access to symbols from people. It's not to write English at them.

The go directive doesn't tell us when the code was written, only which version it was written against. Given that, it seems like there is a fundamental tradeoff: we must either make breaking changes (presumably following a deprecation period), or use a mechanism that can be trivially defeated by a backdated intermediary.

Given that we don't want to take the breaking-change approach, I don't see why the defeat mechanism of “ignoring the linter” is all that much worse than “using a backdated trampoline module” — and the comment-and-lint approach has the advantage of already being in wide use.

[bcmills]

Thinking about the linting approach some more: we do have one interesting option, at least. We run vet for tests, but not lint — so perhaps we could adopt some sort of convention that promotes use of a // Deprecated: identifier from a mere lint warning to a stronger vet warning.

For example:
// Deprecated (go 1.14): […]

could be a lint warning if used in a pre-1.14 module, but a vet warning if used in a module with a later go version.