strconv: consider speeding up formatting of small integers with cache

[ericlagergren]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

1.8

What operating system and processor architecture are you using (go env)?

GOOS=darwin
GOARCH=amd64

What, why, etc.

While profiling an ORM-like program, I discovered most of the 100+ allocs/op were from calls to strconv.Itoa. By implementing a simple cache in front of strconv.Itoa, the number of allocs dropped by over half and so did the runtime of the function (7 to 3 μs). I looked at some other of our projects and noticed most of the Itoa-like calls were used to format smaller numbers (i.e., < 100). I'd imagine other project that generate SQL or graph query code and need to format parameters ($1, $2, ...) would see decent performance improvements as well.

A small test I ran shows no appreciable difference when the majority of N is > 99 or N is a pseudo-random number in [0, b.N), but runs in 1/10 the time where N < 100 and doesn't incur any allocations.

My issues with this idea:

• Small range of numbers (assumes that most calls are < 100)
• Only works with base 10
• Negative numbers need their own cache or an alloc needs to be made to contact the number with "-"

FWIW the cache only needs to be a 190-byte long string.

name             old time/op    new time/op    delta
SmallItoaAll-4     45.8ns ± 0%    47.4ns ± 0%   ~     (p=1.000 n=1+1)
SmallItoaRand-4    90.9ns ± 0%    90.5ns ± 0%   ~     (p=1.000 n=1+1)
SmallItoa-4        31.4ns ± 0%     3.4ns ± 0%   ~     (p=1.000 n=1+1)

name             old alloc/op   new alloc/op   delta
SmallItoaAll-4      8.00B ± 0%     8.00B ± 0%   ~     (all equal)
SmallItoaRand-4     7.00B ± 0%     7.00B ± 0%   ~     (all equal)
SmallItoa-4         1.00B ± 0%     0.00B        ~     (p=1.000 n=1+1)

name             old allocs/op  new allocs/op  delta
SmallItoaAll-4       1.00 ± 0%      1.00 ± 0%   ~     (all equal)
SmallItoaRand-4      1.00 ± 0%      0.00        ~     (p=1.000 n=1+1)
SmallItoa-4          1.00 ± 0%      0.00        ~     (p=1.000 n=1+1)

[josharian]

I recently looked into doing something similar but lower level. When calling string(b) where b is a byte slice of length exactly one, construct a string using the recently introduced runtime.staticbytes, thus avoiding an allocation. This would eliminate the allocation for single-digit numbers.

I gave it up because I wasn't convinced it was worth the impact on other []byte to string conversions, but in the meantime, I saw some other optimizations, leading to CL 37791. It is possible that the other optimizations would pay for a "single digit" code path.

What do you think, @randall77 @mdempsky?

[ericlagergren]

Here's a sample implementation (from the ORM-project):

// smallInts is all integers [0, 99] and only 190 bytes.
const smallInts = "0123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899"

func itoa(i int64) string {
	if i < 10 {
		return smallInts[i : i+1]
	}
	if i < 100 {
		return smallInts[2*i-10 : 2*i-8]
	}
	return strconv.FormatInt(i, 10)
}

@josharian I like the idea of single digits! I was a bit bummed when string(b) still allocated.

[gopherbot]

CL https://golang.org/cl/37963 mentions this issue.

[bradfitz]

Your SQL database and network is fast enough that you notice a 7μs to 3μs drop when formatting SQL queries?

I'm not totally against this, and I usually like to help fight the allocation battles, but I'm a little surprised this comes up often enough to matter. Does it help on any of our existing benchmarks?

[ericlagergren]

@bradfitz of course not. I wish it was, though. 😉 I just happened to run pprof (as part of tests after making a change) and notice strconv came up a lot. That's all. It doesn't really affect many existing strconv benchmarks because they deal with too many numbers > 99. And I haven't tested the entire stdlib.

…

On Thu, Mar 9, 2017 at 8:29 AM Brad Fitzpatrick ***@***.***> wrote: Your SQL database and network is fast enough that you notice a 7μs to 3μs drop when formatting SQL queries? I'm not totally against this, and I usually like to help fight the allocation battles, but I'm a little surprised this comes up often enough to matter. Does it help on any of our existing benchmarks? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#19445 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFnwZ1YioEBp63Rkf4SpGUvrvnDiiibAks5rkCjogaJpZM4MWGfA> .

[dlsniper]

Sorry if this will sound bad, but will we now start seeing and accepting patches that further add numbers to this? Why stop at 99 and not at 2147483648? That seems to cover a lot more of the frequent use-cases where timestamps are converted from strings to numbers.

[mvdan]

@dlsniper do you think that's worth the large amount of binary data and compiler work? 99 numbers, in comparison, is a tiny amount.

[bradfitz]

@mvdan, what's wrong with a 20 GB binary? Disk space is cheap. We could avoid allocations! :)

[bradfitz]

Actually it'd only be about 18 GB. Even better.

[ericlagergren]

since I'm not sure if I can comment on the CL: why a ~1.7kb (total size, including string headers) array instead of a 190 byte const string? Just to save two subtractions?

[bradfitz]

@ericlagergren, you can comment on the CL.

I don't see a 2.4k array.

Let me turn your question around: why a 190 byte const string over the existing code?

[ericlagergren]

@bradfitz cool, didn't know that–thanks!

I typoed that number. I meant ~1.7k (16 bytes for each string header * 100 elements + the strings' data).

The const string is smaller.

[mvdan]

@ericlagergren if you come up with a CL that doesn't complicate the code and keeps performance, I don't see why it wouldn't get merged.