cmd/gc: optimized map[string] lookup from []byte key

[bradfitz]

key := []byte("some key")  // from network, etc

var m map[string]T
....
v, ok := m[string(key)]  // It would be nice if this were optimized away and didn't
actually make a copy & garbage
....

[rsc]

Comment 1:

In general this might introduce races.  If you pass in a []byte to the key lookup,
there is no guarantee that something is not editing the bytes during the lookup.
But it could probably be done so that it would just fail the lookup instead of 
causing corruption.

Labels changed: added priority-later, removed priority-triage.

Owner changed to @rsc.

Status changed to Thinking.

[bradfitz]

Comment 2:

Failing the lookup in that case would be fine & expected.
In fact, I think the naive implementation would get that for free: if the memory is
scanned once for the hashcode and once for the equality, it seems like it just might not
match if the []byte changed.  I don't see how it could do something worse.  The []byte
couldn't be freed by GC and reclaimed to the OS in the meantime because the faked
non-escaping stringheader would still pin it.

[rsc]

Comment 3:

Labels changed: added go1.1maybe.

[robpike]

Comment 4:

Labels changed: removed go1.1maybe.

[bradfitz]

Comment 5:

Answering Dmitry's question from https://golang.org/issue/5160?c=4
here:
Consider:
m := make(map[string]T)
var b []byte = slice of some buffer
go someFunc(b)
t, ok := m[string(b)] 
I would like:
1) string(b) to not make a copy. It should instead make a non-escaping fake String
header to pass to the map lookup function, using the same byte data & length as the
[]byte b.
2) the map lookup to be guaranteed to not crash, even if b is concurrently mutated.
3) t to be valid, iff ok == true.
I don't think that would violate any  of the current Go semantics, since any ordering
between mutations from "go someFunc" and string(b) are already not specified by the
memory model.
Correct?

[dvyukov]

Comment 6:

How is it related to interning? There is no sense to intern temps.
Are talking only about map indexing m[string(b)], or string(b) temps in general?

[bradfitz]

Comment 7:

This bug has nothing to do with interning except that if I were to implement interning
myself in pure Go, without the help of the runtime (which is issue #5160), then I'd want
to lookup interned strings from []byte from the network, without making copies.
That is, I'd have:
  var internTable map[string]string  // identity map. from key to same value.
But I'd want to access it like:
   var b []byte = slice of bytes from the network in a bytes.Buffer
   _ = internTable[string(b)]
... reusing memory from another similar request (e.g. caching HTTP Request MIME headers)
I'm only talking about indexing m[string(b)] in this bug, for Go->runtime maps.
For Go->Go, that is issue #2205 (cmd/gc: read-only escape analysis and avoiding string ->
[]byte copies)

[dvyukov]

Comment 8:

Ah, I see, so you want on intern []byte and get string as the result.
Yes, I think creating a temp string from []byte in this context should be safe even in
complex statements like:
m[f(...)] = g(h(...), k(intern(b)))

[bradfitz]

Comment 9:

Please, ignore everything about interning.  That was just one example of a user of this.
I think you're making this sound more complicated than it is.  And I think one of us is
using the wrong vocabulary.  (I don't want to "intern []byte" because I can't intern a
mutable thing)
I just want m[string(byteSlice)] to avoid a copy and to instead create a fake String
header temporary.  It never escapes the hashmap functions, so it's safe.
And no language semantics change which weren't already racy and undefined.
Example:
m := map[string]int{
   "X": 1,
   "Y": 2,
}
mutate := func(b []byte) { b[0] = 'Y' }
key := []byte{'X'}
go mutate(key)
value, ok := m[string(key)]
What is value & ok?
  1, true?
  2, true?
  0, false?
It's already undefined.

[bradfitz]

Comment 10:

Patch at https://golang.org/cl/8090046

Owner changed to ---.

[bradfitz]

Comment 11:

Labels changed: added performance, garbage.

[rsc]

Comment 12:

Labels changed: added go1.3maybe.

[rsc]

Comment 13:

Labels changed: added release-none, removed go1.3maybe.

[rsc]

Comment 14:

Labels changed: added repo-main.

[bradfitz]

Comment 15:

I still want this, and it would improve Camlistore start-up time when we slurp a big
on-disk index into memory.
Is this definitely off the table for Go 1.3?