time: creates lots of garbage with custom errors

[mzimmerman]

Go version

go version go1.24.1 linux/amd64

Output of go env in your module/workspace:

AR='ar'
CC='gcc'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='1'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='g++'
GCCGO='gccgo'
GO111MODULE=''
GOAMD64='v1'
GOARCH='amd64'
GOAUTH='netrc'
GOBIN=''
GOCACHE='/home/mzimmerman/.cache/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='/home/mzimmerman/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build739745044=/tmp/go-build -gno-record-gcc-switches'
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMOD='/home/mzimmerman/timegarbage/go.mod'
GOMODCACHE='/home/mzimmerman/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/mzimmerman/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/snap/go/10866'
GOSUMDB='sum.golang.org'
GOTELEMETRY='local'
GOTELEMETRYDIR='/home/mzimmerman/.config/go/telemetry'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/snap/go/10866/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.24.1'
GOWORK=''
PKG_CONFIG='pkg-config'

What did you do?

I have an application that deals with lots of different log files and events from disparate systems. The application is to correlate those activities together and so parsing times is important and relevant.

There's ~100 different log types many with their own (awful) format for time. I can't control the upstream applications; I've tried. The best I can do is support what they give me.

So, I have a lot of calls to time.Parse() that fail and I just repeat until I succeed (or eventually fail).

I am throwing away the error value and just checking if time.IsZero() for determining whether the call was successful or not and I did see the compiler optimize it some, but there's still a lot of garbage that is unneeded. Temporarily modifying the time library to not generate an error dropped memory usage significantly.

What did you see happen?

During parsing, the time library creates custom errors for each call such that it is the highest user of allocated memory in my entire application by 10x.

What did you expect to see?

Lower memory usage

[mzimmerman]

Running the benchmark for 10s I get 4.6 gigs of memory allocated -- I would love for the compiler to optimize away the generation of this error at all (if it's unread by the application), or alternatively if there was a ParseWithoutError() type method that would lessen memory usages.

https://go.dev/play/p/M9swxO5s-1R

[ianlancetaylor]

This is a valid request but it's an unusual one. I want to set expectations that this will be difficult to fix and we are unlikely to do anything any time soon. For your immediate needs I would recommend copying the code in time/format.go to write your own format validity checker.

[gabyhelp]

Related Issues

• time.String produces garbage out for time.Parse (garbage in) #17254 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[thepudds]

Hi @mzimmerman, FWIW, I've encountered this exact scenario before. As Ian said, it probably wouldn't be too bad to just copy and tweak the code from the time package.

Another approach I've used (in a different language) is to pair each time format with a corresponding regex that first identifies candidate matches prior to attempting a full time parse, which cuts down quite a bit how often the full time parse fails.

So if we were to update a portion of your benchmark, it could be something like (playground link):

	formats := []pattern{
		{rePattern: `^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}-\d{4}$`, format: "2006-01-02T15:04:05.999-0700"},
		{rePattern: `^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3} -\d{4}$`, format: "2006-01-02T15:04:05.999 -0700"},
		{rePattern: `^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}$`, format: "2006-01-02T15:04:05.999"},
	}

with sample results:

       │  original   │      check-regexp-first           │
       │   sec/op    │   sec/op     vs base               │
Time-4   6.650m ± 5%   2.033m ± 3%  -69.43% (p=0.002 n=6)

       │  original   │      check-regexp-first           │
       │     B/op     │     B/op      vs base                │
Time-4   5.951Mi ± 0%   0.000Mi ± 0%  -100.00% (p=0.002 n=6)

Go doesn't have the fastest regex engine out there, but it's reasonable, and it usually would be faster than allocating errors.

And of course, you could an even lighter first check, like a string contains check or check specific offsets for specific literals, etc.

That said, maybe I misunderstood your situation or maybe that doesn't otherwise help.

[mzimmerman]

@ianlancetaylor Thank you for the response. Years ago when I came to Go, I was initially frustrated with the io.Reader/io.Writer interface, but I see the wisdom in it now in that it doesn't require any allocations. Being that those are potentially heavy memory areas, the design really makes sense. I thought the same thing (sort of) could be applied to the time package. Only allocating as minimal memory as is required to get the job done. I get that this isn't high priority and is niche.

@thepudds You're spot on. I've thought about doing similar, like comparing length, but that breaks down when hours/minutes/days/months can be one (or two) digits. Also, January and March are different also. The simplest (for me) is if the time package supported what I want :) A colleague already started to bring it in and modify the time package and I think that's what we'll end up doing, cause at the end of the day the time package is doing some of those same string comparisons again, just at a later time. I don't want to have to do it twice if I don't have to anyway.

[seankhliao]

You might also want to look at other implementations, like https://pkg.go.dev/github.com/araddon/dateparse

[seankhliao]

I'm not sure if there's anything to do here? ignoring the error is unusual in go code, and you're better off using a more flexible parser anyway.

[mzimmerman]

I was actually using the error, not ignoring it, and I'm just pointing out how much garbage the error creates when called at scale. Perhaps it could return "known" errors instead of allocating new memory.

…

On Tue, Mar 25, 2025, 3:38 PM Sean Liao ***@***.***> wrote: I'm not sure if there's anything to do here? ignoring the error is unusual in go code, and you're better off using a more flexible parser anyway. — Reply to this email directly, view it on GitHub <#72951 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AASNEU2K7ZICZTWTNBGUSKT2WGPCDAVCNFSM6AAAAABZLB2RHOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDONJSGMZTGMZRHA> . You are receiving this because you were mentioned.Message ID: ***@***.***> [image: seankhliao]*seankhliao* left a comment (golang/go#72951) <#72951 (comment)> I'm not sure if there's anything to do here? ignoring the error is unusual in go code, and you're better off using a more flexible parser anyway. — Reply to this email directly, view it on GitHub <#72951 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AASNEU2K7ZICZTWTNBGUSKT2WGPCDAVCNFSM6AAAAABZLB2RHOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDONJSGMZTGMZRHA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[seankhliao]

given that it's a ParseError, it seems unavoidable.
Changing it to sentinel error is likely to be considered a breaking change, and less helpful to most programs.
https://pkg.go.dev/time#ParseError