proposal: spec: byte view: type that can represent a []byte or string #5376

Open
bradfitz opened this Issue Apr 30, 2013 · 7 comments

Comments

Projects
None yet
4 participants
@bradfitz
Member

bradfitz commented Apr 30, 2013

Go has no built-in type which can represent a read-only view of possibly-changing bytes.

We have two current types which internally are a byte pointer and a length:  (the first
also has a capacity, irrelevant to this issue) 


[]byte

   Allowance: "you can mess with this memory"
   Restriction: "this memory isn't necessarily forever; anybody else might be changing it now (race) or later (iterator becoming invalidated after this call, etc)"

string

   Allowance: "if you have a reference to this, it's good forever, and nobody will ever mess with it".
   Restriction: "you can NOT change"


And because of their conflicting allowances and restrictions, any conversion from
string->[]byte or []byte->string necessarily has to make a copy and often
generates garbage. 

Both are great, but there's a missing piece:

Many places in Go's API want a type with *neither* of those allowances, and are happy
with both of those restrictions:

-- much of pkg strings
-- much of pkg bytes
-- all the strconv.Parse* functions (issue #2632)
-- most of the string arguments to system calls
-- the io.Writer interface's argument (no need then for io.WriteString)
-- leveldb's Key/Value accessors.

Look at leveldb's Iterator:

http://godoc.org/code.google.com/p/leveldb-go/leveldb/db#Iterator

It has to go out of its way to say "please do not corrupt my memory".  If
somebody uses memdb directly
(http://godoc.org/code.google.com/p/leveldb-go/leveldb/memdb) and misuses the Iterator
type, all the sudden the memory corruption and stack traces implicate the memdb package,
even though it's a caller of memdb who violated the db.Iterator contract.

All leveldb really wants is to give callers a view of the bytes.

That is, in addition to "string" with its promise A (handle for life) and
"[]byte" with its promise B (you can mutate), we need a a common type to both
of those with neither promise, what is constant-time assignable from a string or a
[]byte.

s := "string"
b := []byte(s)
var v byteview = s // constant time assignment
var v byteview = b // constant time assignment

A byteview would be represented in memory just like a string (*byte, int), but the
compiler would prevent mutations or addressing (like string), and callers would always
treat its backing data as ephemeral, like a []byte, unless negotiated otherwise.

Obviously this isn't a Go 1.n item, considering the impact it would have on the standard
library.

A good name for byteview would be "bytes", but we have a package
"bytes" already. Maybe we can get rid of that package.
@bradfitz

This comment has been minimized.

Show comment Hide comment
@bradfitz

bradfitz Apr 30, 2013

Member

Comment 1:

Status changed to LongTerm.

Member

bradfitz commented Apr 30, 2013

Comment 1:

Status changed to LongTerm.

@rsc

This comment has been minimized.

Show comment Hide comment
@rsc

rsc Nov 27, 2013

Contributor

Comment 3:

Labels changed: added go1.3maybe.

Contributor

rsc commented Nov 27, 2013

Comment 3:

Labels changed: added go1.3maybe.

@rsc

This comment has been minimized.

Show comment Hide comment
@rsc

rsc Dec 4, 2013

Contributor

Comment 4:

Labels changed: added release-none, removed go1.3maybe.

Contributor

rsc commented Dec 4, 2013

Comment 4:

Labels changed: added release-none, removed go1.3maybe.

@rsc

This comment has been minimized.

Show comment Hide comment
@rsc

rsc Dec 4, 2013

Contributor

Comment 5:

Labels changed: added repo-main.

Contributor

rsc commented Dec 4, 2013

Comment 5:

Labels changed: added repo-main.

@cespare

This comment has been minimized.

Show comment Hide comment
@cespare

cespare Oct 28, 2015

Contributor

@rsc rsc changed the title from spec: byte view: type that can represent a []byte or string to proposal: spec: byte view: type that can represent a []byte or string Jun 20, 2017

@rsc rsc added the Go2 label Jun 20, 2017

@ianlancetaylor

This comment has been minimized.

Show comment Hide comment
@ianlancetaylor

ianlancetaylor Nov 29, 2017

Contributor

Certain implementations of generics (#15292) would provide this. But not all--it's possible to imagine generics proposals that do not support this.

Contributor

ianlancetaylor commented Nov 29, 2017

Certain implementations of generics (#15292) would provide this. But not all--it's possible to imagine generics proposals that do not support this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment