Skip to content

proposal: maps,cmd/compile: special case and/or add function for getting an arbitrary entry from a map #66361

Open

Description

@bradfitz

Proposal Details

Sometimes people try to get a random key/value out of a map, writing:

       for k, v = range m {
            break
       }

One example is/was in database/sql. Unfortunately, it uses a lot of CPU:

Screenshot 2024-03-16 at 8 57 01 PM

So I rewrote it in https://go-review.googlesource.com/c/go/+/572119

An alternative is the compiler could recognize that pattern: a range with a body containing only a break.

If so, the compiler could insert a call to a runtime func that takes 2 pointers (either/both of which may be nil) and updates them to a random map value if the map has non-zero length. If it's empty, the pointers are unmodified.

/cc @golang/compiler @maisem @raggi

Activity

  1. gopherbot commented on Mar 17, 2024

    @gopherbot
    Contributor

    Change https://go.dev/cl/572119 mentions this issue: database/sql: optimize connection request pool

  2. mdempsky commented on Mar 17, 2024

    @mdempsky
    Contributor

    We could add:

    package maps
    
    func AnyElem[Key comparable, Val any](m map[Key, Val]) (Key, Val, bool)
    

    And happen to implement that with a compiler optimization that recognizes the way users often write it today. That way existing users benefit, and future users have a more convenient and reliable spelling.

  3. seankhliao commented on Mar 17, 2024

    @seankhliao
    Member
  4. jimmyfrasche commented on Mar 17, 2024

    @jimmyfrasche
    Member

    An alternate, more general, approach would be to expose some way to get the nth map entry. That may be too general but it's worth exploring, if just for contrast.

    Let's say there's

    package unsafe
    func Entry(m MapType, n IntegerType) (ArbitraryType, ArbitraryType)

    It does not provide a fixed order over the map, but as long as you do not modify or iterate over the map the same n will return the same entry. It need not be O(1).

    With that selecting an arbitrary element is unsafe.Entry(m, 0) and choosing a random element is unsafe.Entry(m, rand.IntN(len(m))). This would also let you draw k>1 elements from the map.

    Of course if you want an arbitrary element a random one will do and you could avoid exposing it by just having a func in rand that calls the equivalent of unsafe.Entry(m, rand.IntN(len(m))) for you, but the lack of generic methods make that a bit cumbersome.

  5. mdempsky commented on Mar 17, 2024

    @mdempsky
    Contributor

    An alternate, more general, approach would be to expose some way to get the nth map entry.

    As exposed to users today, map entries are unordered, so there's no unique "nth map entry." We could change that and define some order, but that seems like a substantially broader change than proposed here, and I'm skeptical that's something we want to pursue without use cases.

  6. Jorropo commented on Mar 17, 2024

    @Jorropo
    Member

    A function would be weird because it becomes O(cap(m)) on map with small ratios.
    If you grow a map and remove many elements, this would be expensive operation (because we never shrink maps).

  7. Jorropo commented on Mar 17, 2024

    @Jorropo
    Member

    @bradfitz I'm not convinced a fast path would improve anything here, scanning the empty slots seems like a slow thing to do.

    package a
    
    import "fmt"
    import "testing"
    
    var doNotOptimizeK, doNotOptimizeV uint
    
    func benchmark(b *testing.B, cap, len uint) {
    	b.Run(fmt.Sprintf("%d/%d", cap, len), func(b *testing.B) {
    		m := make(map[uint]uint, cap)
    		for i := range len {
    			m[i] = i
    		}
    		b.ResetTimer()
    		for range b.N {
    			for doNotOptimizeK, doNotOptimizeV = range m {
    				break
    			}
    		}
    	})
    }
    
    func BenchmarkMap(b *testing.B) {
    	for cap := uint(1); cap < 1000000; cap *= 7 {
    		for len := uint(1); len <= cap; len *= 7 {
    			benchmark(b, cap, len)
    		}
    	}
    }
    goos: linux
    goarch: amd64
    cpu: AMD Ryzen 5 3600 6-Core Processor              
    BenchmarkMap/1/1-12         	39464058	        30.63 ns/op
    BenchmarkMap/7/1-12         	40099461	        30.96 ns/op
    BenchmarkMap/7/7-12         	56495338	        21.27 ns/op
    BenchmarkMap/49/1-12        	20483455	        56.14 ns/op
    BenchmarkMap/49/7-12        	29672593	        40.95 ns/op
    BenchmarkMap/49/49-12       	46328062	        26.57 ns/op
    BenchmarkMap/343/1-12       	 5146122	       239.7 ns/op
    BenchmarkMap/343/7-12       	13583094	        80.50 ns/op
    BenchmarkMap/343/49-12      	28516389	        39.72 ns/op
    BenchmarkMap/343/343-12     	42983257	        28.27 ns/op
    BenchmarkMap/2401/1-12      	  722722	      1616 ns/op
    BenchmarkMap/2401/7-12      	 2753096	       521.7 ns/op
    BenchmarkMap/2401/49-12     	14052568	        86.84 ns/op
    BenchmarkMap/2401/343-12    	29775351	        38.74 ns/op
    BenchmarkMap/2401/2401-12   	41705300	        30.31 ns/op
    BenchmarkMap/16807/1-12     	   94954	     12596 ns/op
    BenchmarkMap/16807/7-12     	  406827	      3432 ns/op
    BenchmarkMap/16807/49-12    	 2267532	       465.1 ns/op
    BenchmarkMap/16807/343-12   	10755490	        96.90 ns/op
    BenchmarkMap/16807/2401-12  	27650230	        43.03 ns/op
    BenchmarkMap/16807/16807-12 	37117978	        32.77 ns/op
    BenchmarkMap/117649/1-12    	   11602	    100873 ns/op
    BenchmarkMap/117649/7-12    	   38211	     30643 ns/op
    BenchmarkMap/117649/49-12   	  325615	      4009 ns/op
    BenchmarkMap/117649/343-12  	 1842660	       640.4 ns/op
    BenchmarkMap/117649/2401-12 	 9241438	       128.8 ns/op
    BenchmarkMap/117649/16807-12         	21193573	        50.61 ns/op
    BenchmarkMap/117649/117649-12        	31642425	        38.37 ns/op
    BenchmarkMap/823543/1-12             	    2823	    426807 ns/op
    BenchmarkMap/823543/7-12             	   10000	    118543 ns/op
    BenchmarkMap/823543/49-12            	   67795	     17367 ns/op
    BenchmarkMap/823543/343-12           	  466880	      2473 ns/op
    BenchmarkMap/823543/2401-12          	 2689250	       422.2 ns/op
    BenchmarkMap/823543/16807-12         	 8709276	       124.7 ns/op
    BenchmarkMap/823543/117649-12        	18674678	        62.27 ns/op
    BenchmarkMap/823543/823543-12        	24420856	        48.04 ns/op
    PASS
    ok  	command-line-arguments	60.335s
    

    It easily do ~30ns when the fill up ratio is good, so your ~10s suggest you are doing 329 million requests (assuming the ratio is good). I know this is an extremely limited comparison, but are you doing in the range of 3b~32m requests ? If you are doing far less, your map is poorly filled up so there is gonna be lots of scanning of empty cells involved even with a compiler help (more state could be added to skip some scanning but that a change to the runtime more than the compiler).

  8. bradfitz commented on Mar 17, 2024

    @bradfitz
    ContributorAuthor

    @Jorropo, it's a very heavily loaded server. I did the optimization based on data, not on a whim or for fun.

  9. Jorropo commented on Mar 17, 2024

    @Jorropo
    Member

    The patch you sent make sense.
    I'm saying I don't think adding a compiler fast path is gonna be a substantial improvement because a mostly empty sparse map is the wrong datastructure to pick random elements from.
    How long was the profile you took ? or how many requests did it handled ? this can be used to estimate if the map is poorly filled up or not.

  10. pkositsyn commented on Mar 18, 2024

    @pkositsyn
    Contributor

    I think it is generally unobvious, that it's not an even distribution by taking the first element of iteration (https://go.dev/play/p/n50sJuL1O6s and for other lens it's still not even).

    Do we actually want to encourage "getting random element directly from a map"? This clearly adds a confusion about the distribution

  11. gophun commented on Mar 18, 2024

    @gophun

    @pkositsyn I assume that the intention is to get an 'arbitrary' element (which could be the same each time) instead of a 'random' element. The specification leaves the order undefined. So, it is theoretically possible that another Go implementation (or this one in the future, albeit unlikely) implements an ordered map.

  12. bboreham commented on Mar 18, 2024

    @bboreham
    Contributor

    Anecdata: I spent many hours tracking down odd behaviour in a gossip system due to the skewed distribution of this pattern.

  13. pkositsyn commented on Mar 18, 2024

    @pkositsyn
    Contributor

    If it is about getting "any" element I suggest renaming the issue to "getting arbitrary element entry in map"

  14. 8 remaining items

  15. added this to the Backlog milestone on Mar 20, 2024
  16. changed the title [-]cmd/compile: special case getting a random map entry?[/-] [+]proposal: maps,cmd/compile: special case and/or add function for getting an arbitrary entry from a map[/+] on Mar 20, 2024
  17. moved this to Incoming in Proposalson Mar 20, 2024
  18. adonovan commented on Mar 27, 2024

    @adonovan
    Member

    Most often when I use range m { ...; break}, I know that the map contains a single element and wish there was a library function maps.Sole:

    package maps
    
    // Sole returns the sole entry of map m. It panics if len(m) != 1.
    func Sole[K comparable, V any](m map[K]V) (K, V) {
       if n := len(m); n > 1 {
         panic(fmt.Sprintf("map contains %d entries", n))
       }
       for k, v := range m { return k, v }
       panic("empty map")
    }
  19. rsc commented on Apr 3, 2024

    @rsc
    Contributor

    I don't think we need to worry about giving every element exactly the same chance for the start of an iteration.

    Special-casing this:

       for k, v = range m {
            break
       }
    

    to be faster seems reasonable and does not require a proposal. Is anyone interested?

  20. nanasi880 commented on Jan 9, 2025

    @nanasi880

    There is nothing wrong with the code being fast. However, using a for loop to pick a random value from a map in the first place seems inappropriate, considering the reason why Go made map iteration random.
    Before Go 1, we faced portability issues because we relied on a specific order when looping over maps. Now we are trying to rely on the property that a random value is returned when looping over maps.
    I think what we really need here is a function that can quickly return a single random value, not to strengthen the for loop.
    Again, I think that making code faster is a good thing. But isn't it bad to rely on the property of looping over maps?

  21. nanasi880 commented on Jan 9, 2025

    @nanasi880

    I have checked the Go language specification.
    https://go.dev/ref/spec#For_statements

    The iteration order over maps is not specified and is not guaranteed to be the same from one iteration to the next.

    The language specification does not guarantee that the order will be the same for each iteration. However, it also does not guarantee that different values will be produced. It only means that the order does not have to be the same.
    I think that expecting randomness by looping over a map is a bad idea that relies too heavily on internal implementation details that are not part of the language specification.

  22. mateusz834 commented on Apr 15, 2026

    @mateusz834
    Member

    Well today I saw CL 745440 from Alan. And thought to myself that maybe we should have an iter.First(). So it could be used as: maps.All(map).First().

    func (seq Seq2[K, V]) First() (k K, v V, ok bool) {
       for k, v = range seq {
            ok = true
            break
       }
    }

    That might be a better place for such thing.

    But note that this would not solve the performance, but might result with a clearer/simpler code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Proposalcompiler/runtimeIssues related to the Go compiler and/or runtime.

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions