New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doc: mention "purego" build tag convention somewhere #23172

Open
dsnet opened this Issue Dec 18, 2017 · 16 comments

Comments

Projects
None yet
@dsnet
Member

dsnet commented Dec 18, 2017

As the number of Go implementations continues to increase, the number of cases where the unsafe package is unlikely to work properly also rises. Currently, there is appengine, gopherjs, and possibly wasm where pointer arithmetic is not allowed.

Currently, protobuf and other packages special cases build tags for appengine and js and may need to add others in the near future. It does not scale very well to blacklist specific known Go implementations where unsafe does not work.

My proposal is to document safe as a community agreed upon tag meaning that unsafe should not be used. It is conceivable that this concept be extended to the compiler rejecting programs that use unsafe when the safe tag is set, but I'm currently more interested as a library owner in knowing whether to avoid unsafe in my own packages.

\cc @zombiezen @dneil @neelance @shurcooL

@gopherbot gopherbot added this to the Proposal milestone Dec 18, 2017

@gopherbot gopherbot added the Proposal label Dec 18, 2017

@dsnet

This comment has been minimized.

Member

dsnet commented Dec 18, 2017

The code history of protobuf seems to indicate that this very same concept was discussed but not pursued further. I'd like to push this more since I see this distinction in at least 2 packages I own.

golang/protobuf#154

@dsnet dsnet closed this Dec 18, 2017

@dsnet dsnet reopened this Dec 18, 2017

@mdempsky

This comment has been minimized.

Member

mdempsky commented Dec 18, 2017

I think standardizing a build tag to indicate whether package unsafe is available makes sense.

It is conceivable that this concept be extended to the compiler rejecting programs that use unsafe when the safe tag is set

I disagree. Currently, build tags are strictly a build-system concept. I'd argue the compiler should remain ignorant of them. cmd/compile already has a -u flag that prevents importing package unsafe, and the build system can arrange to pass -u as appropriate.

@dmitshur

This comment has been minimized.

Member

dmitshur commented Dec 19, 2017

/cc @bradfitz who also ran into this with go4.org/reflectutil, and seemed to like "safe" at the time.

@bradfitz

This comment has been minimized.

Member

bradfitz commented Dec 19, 2017

@shurcooL, I'm still fine with "safe", as long as it's defined (i.e. "code that doesn't import "unsafe").

But does it also mean no assembly?

Those are the sorts of things that should be clarified, if this is to be blessed somehow. (our own use, wiki page, etc)

@dmitshur

This comment has been minimized.

Member

dmitshur commented Dec 19, 2017

@dsnet Can you clarify if your proposal is about documenting safe to have a very specific meaning and applied to all packages?

Or is it about documenting the fact that safe is a commonly used build tag for a given purpose, but individual projects still have get final say on the exact meaning of the safe build tag for their own needs?

@flimzy

This comment has been minimized.

flimzy commented Dec 19, 2017

Would this proposal be codified in the standard library somehow, perhaps by adding a !safe build tag to the unsafe package? Or would it live purely in documentation?

@davecb

This comment has been minimized.

davecb commented Dec 19, 2017

In a previous life, we had to identify versions of libraries, and rapidly found out that that was too coarse a measure. We eventually attached a label via the linker to each entry point*, and could tell if, for example, a call to memcpy allowed overap or not. '

We also used it in migration work, to identify parts of programs that could not be supported on a different OS or hardware platform.

You arguably should consider labelling parts of the unsafe library with supported and unsupported by target OS, language or whatever, not the whole library if only one operation is unavailable.

--dave
[* a description of using per-entry-point labels for a different purposes is at https://leaflessca.wordpress.com/2017/02/12/dll-hell-and-avoiding-an-np-complete-problem/ ]

@andlabs

This comment has been minimized.

Contributor

andlabs commented Jan 16, 2018

Would this unsafe build tag also affect code that uses cgo? SWIG?

How would this build tag interact with the standard library, where both are used a lot? Does no unsafe mean no reflect as well?

What is the unsafe policy on nacl? is there anything in nacl that we could use for this?

@shurcooL it sounds like the latter at minimum, the former ideally.

@dsnet

This comment has been minimized.

Member

dsnet commented Jan 16, 2018

I propose that "safe" be soft signal that a library should have memory safety (i.e., makes no assumptions about how objects are laid out in memory, the architecture endianess, semantics regarding registers, etc).

Thus, the "safe" tag has the following properties:

  • This is just a hint for library authors who want to write code that is highly portable. There is no logic in the compiler or the build tool to enforce this.
    • Thus, the standard library doesn't have to use it. Portability is achieve by either requiring a fork of the standard library (as is the case for gopherjs) or via a series of build tags as the mainline standard library does for various architectures.
  • Use of reflect is allowed since it doesn't allow you to violate memory safety.
  • Use of cgo is allowed. We already have a build tag for that, which is cgo. In practice, safe implies that cgo is not used since it is difficult to use cgo without pointers (for which you need unsafe).
  • Use of assembly is forbidden. Any reasonable use of assembly makes assumption about how memory is laid out on the stack and/or heap.

Thus, appengine and gopherjs are example toolchains that would always set the safe tag.

@rsc

This comment has been minimized.

Contributor

rsc commented Jan 22, 2018

Based on discussion with proposal-review:

  • It seems reasonable for appengine, gopherjs, and wasm, all of which declare their own "restricted build" tag, to agree on a common one.
  • It should mean no asm, no cgo, no unsafe. (It's impossible to use cgo without unsafe.)
  • Nothing in the standard distribution would care; this is really about coordination between these other non-standard environments. Where do you propose to document this?
  • A better name than "safe" would be nice. "purego"?
@neelance

This comment has been minimized.

Member

neelance commented Jan 23, 2018

On WebAssembly: The wasm backend that I'm working on uses a linear memory, so unsafe is fully supported. It also has its own asm instructions, just like other architectures. Cgo is not supported (unless someone wants to do a crazy integration with emscripten).

@cznic

This comment has been minimized.

Contributor

cznic commented Jan 23, 2018

A better name than "safe" would be nice. "purego"?

👍 for purego, already using it in my projects for some years.

@dsnet

This comment has been minimized.

Member

dsnet commented Jan 23, 2018

I support purego as well. Even if WebAssembly supports unsafe (which is great to hear!), there is always still the use case where someone wants to compile with a pure-Go version for a variety of reasons.

I don't have any great suggestions for where to document this, but perhaps the godoc for go/build?

@dmitshur

This comment has been minimized.

Member

dmitshur commented Jan 23, 2018

I wanted to point out that in colloquial usage, I've seen "pure Go" most commonly refer to packages that don't use cgo (but can use unsafe, assembly). Seeing it mean "no unsafe and no assembly as well" would require some calibration. But maybe it's fine.

The math/big package contains some precedent on this: it defines a math_big_pure_go tag, which is being used as proposed here (no assembly, no unsafe, no cgo).

@rsc

This comment has been minimized.

Contributor

rsc commented Jan 29, 2018

OK, purego it is.

@rsc rsc changed the title from proposal: doc: agree upon a "safe" build tag to proposal: doc: mention "purego" build tag convention somewhere Jan 29, 2018

@rsc rsc changed the title from proposal: doc: mention "purego" build tag convention somewhere to doc: mention "purego" build tag convention somewhere Jan 29, 2018

@rsc rsc modified the milestones: Proposal, Go1.11 Jan 29, 2018

cespare added a commit to cespare/xxhash that referenced this issue Jan 29, 2018

Rename the 'noasm' tag to 'purego'
This is the convention that the Go ecosystem is standardizing on.

See golang/go#23172.

dmitshur added a commit to gopherjs/gopherjs that referenced this issue Feb 2, 2018

build: Add "purego" build tag to default context.
The proposal in golang/go#23172 was accepted. The "purego" build tag
is intended to be a community agreed upon soft-signal to indicate the
forbidden use of unsafe, assembly, or cgo.

Since GopherJS doesn't support those, it should always use "purego"
versions of Go packages when they're available. This is done by always
setting the "purego" build constraint.

Fixes #746.

dsnet added a commit to google/go-cmp that referenced this issue Feb 2, 2018

Support purego build tag
The proposal in golang/go#23172 was accepted. The "purego" build tag
is intended to be a community agreed upon soft-signal to indicate the
forbidden use of unsafe, assembly, or cgo.

A change in the future will remove special-casing the appengine and js tags
once the related toolchains support purego (possibly after some bake-in period).

dsnet added a commit to golang/protobuf that referenced this issue Feb 2, 2018

proto: support purego build tag
The proposal in golang/go#23172 was accepted. The "purego" build tag
is intended to be a community agreed upon soft-signal to indicate the
forbidden use of unsafe, assembly, or cgo.

A change in the future will remove special-casing the appengine and js tags
once the related toolchains support purego (possibly after some bake-in period).

dmitshur added a commit to gopherjs/gopherjs that referenced this issue Feb 2, 2018

build: Add "purego" build tag to default context. (#747)
The proposal in golang/go#23172 was accepted. The "purego" build tag
is intended to be a community agreed upon soft-signal to indicate the
forbidden use of unsafe, assembly, or cgo.

Since GopherJS doesn't support those, it should always use "purego"
versions of Go packages when they're available. This is done by always
setting the "purego" build constraint.

Document build tags that are set.

Bump GopherJS version to 1.9-2. Setting the "purego" build constraint
can change behavior in theory. It's better to have a distinct GopherJS
version to attribute it to.

Fixes #746.

dsnet added a commit to google/go-cmp that referenced this issue Feb 2, 2018

Support purego build tag
The proposal in golang/go#23172 was accepted. The "purego" build tag
is intended to be a community agreed upon soft-signal to indicate the
forbidden use of unsafe, assembly, or cgo.

A change in the future will remove special-casing the appengine and js tags
once the related toolchains support purego (possibly after some bake-in period).

dsnet added a commit to google/go-cmp that referenced this issue Feb 2, 2018

Support purego build tag (#68)
The proposal in golang/go#23172 was accepted. The "purego" build tag
is intended to be a community agreed upon soft-signal to indicate the
forbidden use of unsafe, assembly, or cgo.

A change in the future will remove special-casing the appengine and js tags
once the related toolchains support purego (possibly after some bake-in period).

damz added a commit to damz/msgp that referenced this issue Feb 14, 2018

Support GopherJS and other pure Go environments
Unsafe pointer arithmetic is not supported under GopherJS,
and currently fails silently.

Add the `purego` tag, standardized as part of golang/go#23172,
in addition to the `appengine` tag kept for backwards compatibility.

philhofer added a commit to tinylib/msgp that referenced this issue Feb 15, 2018

Support GopherJS and other pure Go environments (#218)
Unsafe pointer arithmetic is not supported under GopherJS,
and currently fails silently.

Add the `purego` tag, standardized as part of golang/go#23172,
in addition to the `appengine` tag kept for backwards compatibility.
@gopherbot

This comment has been minimized.

gopherbot commented Mar 28, 2018

Change https://golang.org/cl/103239 mentions this issue: go/build: document purego convention

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment