Directly generate batch without intermediate entries #12
Conversation
mstoykov
force-pushed
the
optimizeGenerateEntries
branch
2 times, most recently
from
a55bf6d
to
74b587f
Compare
batch.go
Outdated
| line = flog.NewLog(string(labels[model.LabelName("format")]), time.Now()) | ||
| stream.Entries = append(stream.Entries, logproto.Entry{ | ||
| Timestamp: time.Now(), |
There was a problem hiding this comment.
These two calls to time.Now() are currently the biggest CPU user if you just run the examples/simple.js - we can make it into one in the begging of the for loop, but I guess you want to have the time in the line?
|
In general LGTM! |
|
I did add a basic benchmark, I would've made it a table-driven one and added a bunch of variants, but I have no idea what will be good values for that, so if you provide more I will update it. I can rearrange and squash commits if we want to keep some kind of history or not if we are just going to merge squash this in the end - I don't know what you prefer, but I have no problems with either. p.s. the commit message has the results |
They look great! Great work! 🥳 |
Fixes #10 Basic benchmark results: old are the results with making entries and then batch, old- is only making the entries new is the new variant where we directly make the batch as can be seen there is around 50% reduction name \ time/op old.bench old-.bench new.bench GenerateEntries-8 255µs ± 9% 202µs ± 4% 138µs ± 5% name \ alloc/op old.bench old-.bench new.bench GenerateEntries-8 20.2kB ± 0% 12.5kB ± 0% 9.3kB ± 0% name \ allocs/op old.bench old-.bench new.bench GenerateEntries-8 435 ± 0% 213 ± 0% 216 ± 0% Co-authored-by: Christian Haudum <christian.haudum@gmail.com>
mstoykov
force-pushed
the
optimizeGenerateEntries
branch
from
57db64d
to
f46fe53
Compare
|
@chaudum merge when you want if you don't want more changes |
Fixes #10