proposal: cmd/doc: add "// Unstable:" prefix convention

[arnottcr]

What did you see today?

Many projects have different stability guarantees for the exported symbols in a package. Others rely on generated code that cannot, or will not, give stability guarantees. As such, most authors will document this pre-release or instability in doc strings, but the syntax and conventions are all over the place.

The standard library likes to reference the Go compatibility promise or Go 1 compatibility guidelines, since it is bound by them, however these do not work well for community packages. Other terms like non-portable and EXPERIMENTAL are descriptive and well explained in unsafe and syscall/js respectively.

Some community libraries have used terms like // Alpha: ..., // Beta: ..., and // Unstable: ..., which work as well. There could be an argument for not releasing pre-release features on a stable branch, but other times like with the proto client/server interfaces, instability is a guarantee that must be worked around.

What would you like to see?

Similar to // Deprecated: ..., I would like to see the stabilization of supported comment tags for unstable symbols.

They support the same three formats that same three formats that deprecation has.

These tags should also allow such symbols to be excluded from the gorelease tool.

A single tag should be sufficient:

// Unstable: ...

When interacting with released, finalized symbol that cannot or will not be stabilized, the description can provide stability workarounds, alternatives, or what guarantees should be expected.

When interacting with pre-release features, a proposed timeline can be given or alternatives for customers requiring stability guarantees.

What did not work?

The // Alpha: ... and // Beta: ... options looked promising, since they would only be used for temporary instability as part of the release process. The two terms overload one another (what is the difference between alpha, beta, and // PreRelease: ...?), leading to confusion. Furthermore, the programmatic benefits of knowing an API will stabilize in a future release is not that useful, "is it unstable now?" is more important.

The // Experimental: ... syntax used by the standard library implies the notion that the feature will eventually be stabilized. This further overloads it with alpha, beta, etc. and does not fit the needs of the above gRPC interfaces.

The // NonPortable: ... syntax is too domain specific to unsafe to make sense for purely semantic changes to packages. It makes sense for unsafe, but does not generalize

[jayconrod]

cc @jba

We've talked about having apidiff ignore changes to definitions that are annotated like this. I imagine it would be useful for other tools as well.

[carnott-snap]

What are next steps for getting approval for this proposal?

[jayconrod]

@carnott-snap I don't think this really needs to go through the formal proposal process. It's something we agree on and discussed already as part of the apidiff and gorelease design.

I think the next step would be for this to be implemented in golang.org/x/exp/apidff. gorelease will use that.

This should be documented somewhere, but I'm not sure where yet. We should have documentation on preparing a release, linked from https://golang.org/doc/. That should basically be a checklist, and gorelease can refer to it in error messages. These stability comments could be mentioned there.

About the specifics on the comments: I agree that // Unstable: is the best single marker for this. I wouldn't mind having // Experimental: as a synonym, but it's probably better to have one word instead of two. We should require the word to be at the beginning of a paragraph, though it doesn't have to be (and usually should not be) the first paragraph.

[carnott-snap]

@carnott-snap I don't think this really needs to go through the formal proposal process. It's something we agree on and discussed already as part of the apidiff and gorelease design.

Is it worth getting this closed/tagged as Proposal-Accepted, or should we start work to prove the benefit first?

I think the next step would be for this to be implemented in golang.org/x/exp/apidff. gorelease will use that.

Does Google want to own the dev work? If not, I am happy to contribute.

This should be documented somewhere, but I'm not sure where yet. We should have documentation on preparing a release, linked from https://golang.org/doc/. That should basically be a checklist, and gorelease can refer to it in error messages. These stability comments could be mentioned there.

Long term it would be nice to mentioning this as part of the module release docs item in #33637. But for now, I think we can follow the lead that // Deprecated: ... has set:

• Create https://github.com/golang/go/wiki/Unstable.
• Amend the godoc blog post: https://blog.golang.org/godoc-documenting-go-code.

About the specifics on the comments: I agree that // Unstable: is the best single marker for this. I wouldn't mind having // Experimental: as a synonym, but it's probably better to have one word instead of two.

Totally agree, may be worth getting a wider audience, but one acceptable word seems better than two perfect ones.

We should require the word to be at the beginning of a paragraph, though it doesn't have to be (and usually should not be) the first paragraph.

IIRC, doc comments rules will not allow it to be the first paragraph, though type unstable struct{} // Unstable: do not use. should be fine. I think we should follow the // Deprecated: ... conventions, and require the keyword be the first word of the paragraph.

[rsc]

Adding "// Unstable:" has a much larger effect than "// Deprecated:".
"// Deprecated" is basically a courtesy: it says "this will keep working but just so you know there is a newer thing you might be happier with." If you don't see the comment, no big deal.

"// Unstable" is much more invasive. It says "even though you might think otherwise, I reserve the right to change the types/code here in the future or delete it entirely and break your uses." If you don't see the comment, it's a very big deal: you upgrade and your code breaks.

The big question is not what the comment syntax should be but whether we want to bless this kind of user-hurting behavior at all.

An alternative would be to develop experimental changes like this on explicitly experimental version tags (v1.6.1-unstable, for example), keeping the unstable code completely out of the stable tag.

Another alternative would be to put the name into the defined symbols, like "mypkg.UnstableFoo", like we did for types like testing.InternalBenchmark (before internal directories). It's impossible to miss the Unstable when it's in the name.

We should very carefully consider what the right way is to let package authors experiment without hurting users. A simple comment does not seem like enough. (I realize that the idea is tools would surface the comment etc but then that's just more mechanism on top, whereas versions or symbol names that explicitly say unstable reuse existing mechanism.)

[bcmills]

The big question is not what the comment syntax should be but whether we want to bless this kind of user-hurting behavior at all.

Note that there are already at least a few examples of unstable definitions in the standard library.

For example, consider:

• unicode.Version (which changes every time the version is updated)
• os.ModeType (which would change if a new type bit is added)
• text/scanner.GoTokens (which will change if a new kind of token is added to the Go grammar, for example in order to handle generics).

[jayconrod]

As another data point, there are also several symbols in the testing package: RegisterCover, Cover, CoverBlock, MainStart. These have comments like:

It is not meant to be called directly and is not subject to the Go 1 compatibility document. It may change signature from release to release.

These definitions are only meant to be called by generated code that is tightly coupled with the testing package, so in this case, it might be fine to name things Unstable. I imagine you might see the same pattern any time you have a code generator and a library that go together.

However, some definitions like the ones Bryan mentioned may have their own definition of compatibility. According to apidiff, changing the value of an integer constant is an incompatible change because it may be used as an array length. Using unicode.Version as an array length would be quite strange though, and I don't think there's a need to report a change to that constant as an error in gorelease. I'd like to have some annotation like // Unstable: that gives authors a chance to suppress false positives like this.

[jayconrod]

@rsc I'm not sure what the next step is. It doesn't sound like the proposal committee has accepted or rejected this. What information would be useful in making a decision?

[dsnet]

whether we want to bless this kind of user-hurting behavior at all.

As a data point. In the protobuf module, there are several types or functions explicitly marked as being for internal use only. However, they must be exported in order for generated code to access them. We reserve the right to change those APIs with the caveat that we don't break existing usages that were properly generated by protoc-gen-go.

Whether this convention is adopted or not, this type of sharp edge already exists. In v1 the sharp edge is awful since the internal functions is placed alongside public functions in the proto package. In v2, the sharp edge is isolated to a protoimpl package which is explicitly documented as being unstable.

[ianlancetaylor]

@jayconrod I think that what might be helpful is a reason why you can't use names like ExperimentalFoo.

[jayconrod]

@ianlancetaylor My expectation is that gorelease via apidiff will identify a number of changes as technically incompatible, but developers will want a way to annotate definitions to suppress false positives.

I'm planning to ship an experimental version of gorelease soon ("soon" depends on what other cmd/go issues need to be fixed before the 1.14 beta). Hopefully I'll have some useful feedback to share after that.

[ianlancetaylor]

Sorry if this is an unhelpful comment, but the cmd/api tool in the standard library does have a way to suppress false positives, for the kinds of examples that @bcmills cites above. See the files in https://golang.org/api/ .

[carnott-snap]

I think that what might be helpful is a reason why you can't use names like ExperimentalFoo.

• In the description I call out why Experimental is problematic for long term, intentional instability: is Unstable equally acceptable?
  • If so, s/Experimental/Unstable/g.
• Experimental is quite long to add to every unstable symbol.
• Would const experimentalFoo = "bar" be stable?
• Would this apply to functions retroactively?
  • type ExperimentalRoute interface{} where the link hardware is experimental, not the Go api.
  • github.com/moby/moby@v1.13.1/api/server/router.ExperimentalRoute is stable
• What is the migration path for symbols that are partially stable? (read: custom compatibility guarantees)
  • It would be a breaking change to convert gRPC code generator's logic fromtype XxxService interface into type ExperimentalXxxService.
• Must Experimental be the first word, or simply prefix in the symbol?
  • func ExperimentallyInvestigate() {}
  • func GetExperimentalFoo() {}
• How do I mark a whole package or module?
  • Can we reserve exp or experimental, like we did internal?
  • Would a lowercase prefix be clear?
    • syscall/experimentaljs
    • golang.org/x/exp/experimentalapidiff
  • Is documentation sufficient?
    • See the current state of syscall/js, golang.org/x/exp, etc.
  • Currently apidiff and gorelease are not mindful of this.
  • golang.org/x/exp/apidiff.ExperimentalChange has stutter.
  • What protects you from accidentally committing to something: golang.org/x/exp/apidiff.Change.

[ianlancetaylor]

I didn't mean anything special about the exact word Experimental. I just meant, as in #34409 (comment), to put the instability into the symbol name itself rather than a comment. A comment can be missed. It's much harder to miss the symbol name.

[bcmills]

The (IMO pretty major) downside of putting words like Experimental in the identifiers (or in the package path, for that matter) is that it imposes an additional breaking change if/when the API is promoted from experimental to stable.

(It's the same reason we don't distinguish between v0 and v1 in module paths.)

[cristaloleg]

It's much better to use branches for experimental features. This makes 2 worlds(stable and experimental) independent and reduces possible mistakes for the user.

As Russ mentioned above: comment is too weak to protect user from incorrect usage.

Extending previous comment by Bryan: v0 and v1 means that the module can be experimental, but in the same time we can use replace statement in go.mod to use a module from specific branch with experimental/unstable/ things.

In other words: comments or unstable api increases code entropy which increases code maintainability.

[carnott-snap]

I built this proposal around documentation to make adoption less invasive. This was partially based on experience with the protobuf libraries, where they define custom compatibility guarantees. I agree condoning unstable interfaces is troublesome, but many projects require it, and I wanted a canonical way to label/identify it, both for users and tooling.

It may be helpful to isolate the use cases that exist for instability: (please correct me if I left anything out or if this is a false trichotomy)

1. We have experimental things and want to change/remove them.
   • Call this the pre-release case (or alpha, beta).
   • packages
     • golang.org/x/exp
     • syscall/js
   • symbols
     • golang.org/x/tools/go/analysis.ObjectFact
2. We would like to define our own compatibility guarantees that differ from the apidiff/Go 1 compatibility.
   • Call this the we know better than the language case.
   • unicode.Version
     • This will not randomly become a func, but the value and length may change.
   • Call site compatibility
     • Protobuf generated code type XxxServer interface must be implementable and allow adding methods.
   • Considering the complexity of the apidiff spec, this will keep cropping up; sometimes it could be useful to break some guarantees to give extra features: e.g. change a constant's value when you know it will not be used as an array length.
3. We want to share an internal interface between two packages.
   • Call this the protected api surface case.
   • testing.RegisterCover
   • gRPC generated methods against message structs: e.g. XXX_Unmarshal.
   • Some of these are required because of the limited nature of the Go language, others are bad practice, frequently it is hard to tell the difference.

I think we all agree that we want stable packages, but even the standard library is developing experimental or long term unstable features. Most examples use documentation to signal their stability, so this seemed intuitive and canonical. What emphasis should be placed on supporting existing patterns, as opposed to preventing undesirable behaviour?

Outstanding concerns:

• Branches and tags will not work well with 2. or 3. Authors want the ability to ship official, (partially) unstable releases.
• The Experimental proposal does not appear to be a specification, so much as a convention. This lack of tooling support makes it incompatible with my intentions.

[rsc]

@rsc I'm not sure what the next step is. It doesn't sound like the proposal committee has accepted or rejected this. What information would be useful in making a decision?

The proposal committee's job is to help steer a discussion toward consensus, not to decide on their own. It sounds like there is no consensus here yet.

I wrote a lot about this topic at https://research.swtch.com/proposals-experiment. It's very important that people know they are using experimental features. Otherwise we run the risk of hitting the situation Rust did where everyone was on "unstable". I have the same concern about the "exp" build tag in #35704: it's too easy to lapse into where everything uses this.

Actually the build tag may be worse, since if any dependency super far down uses exp, you have to do the same. So basically everyone will have to use it.

[carnott-snap]

I would like to be clear about the purposes of the two tickets being discussed:

• #34409: documenting any form of feature instability
• #35704: gating alpha, beta, testing, pre-release features that will be stabilised

The proposal committee's job is to help steer a discussion toward consensus, not to decide on their own. It sounds like there is no consensus here yet.

My concern is that the current state of things is not good. Major tools are actively developing and releasing unstable features, by apidiff's measure, that have a myriad of ways to signal their stability guarantees. This is hard on consumers, and may lead to more instability if we do nothing.

I wrote a lot about this topic at https://research.swtch.com/proposals-experiment. It's very important that people know they are using experimental features.

Would a go vet lint be sufficient? We already do this for // Deprecated: . What do you define as know they are using? This seems like a very tricky litmus test, e.g. dep was relied upon heavily by the community, despite never leaving experiment status.

Otherwise we run the risk of hitting the situation Rust did where everyone was on "unstable".

I have worked with Rust myself and do not see this today. Are there any lessons we can learn from their experiences? My understanding was that they had a lot of important features that needed to be stabilised, e.g. async, and that finally unlocking things from the nightly compiler was the fix.

I have the same concern about the "exp" build tag in #35704: it's too easy to lapse into where everything uses this.

Actually the build tag may be worse, since if any dependency super far down uses exp, you have to do the same. So basically everyone will have to use it.

Since #35704 is trying to solve a different problem, would you mind continuing the discussion there? I broke it out partially because I saw two heterogeneous problems (pre-release and custom compatibility) that felt like they may should be solved differently.