Make summary, wordcount etc. more effective #2378

Closed
wants to merge 3 commits into
from

Projects

None yet

2 participants

@bep
Collaborator
bep commented Aug 20, 2016 edited

In total, the effect with this branch compared with master:

benchmark           old ns/op      new ns/op      delta
BenchmarkHugo-4     3166246627     3038050615     -4.05%

benchmark           old allocs     new allocs     delta
BenchmarkHugo-4     10557420       10540152       -0.16%

benchmark           old bytes      new bytes      delta
BenchmarkHugo-4     1621203992     1605716520     -0.96%

See: https://github.com/bep/hugo-benchmark

Word counting benchmark:

BenchmarkTotalWords-4            100000         18795 ns/op           0  B/op           0 allocs/op 
BenchmarkTotalWordsOld-4          30000         46751 ns/op           6400 B/op           1 allocs/op

New auto-summary benchmark:

BenchmarkTestTruncateWordsToWholeSentence-4            300000         4720 ns/op         0 B/op              0 allocs/op
BenchmarkTestTruncateWordsToWholeSentenceOld-4         100000         17699 ns/op        3072 B/op           3 allocs/op
@bep bep changed the title from WIP: Make summary, wordcount more effective to WIP: Make summary, wordcount etc. more effective Aug 20, 2016
@bep bep changed the title from WIP: Make summary, wordcount etc. more effective to Make summary, wordcount etc. more effective Aug 24, 2016
@spf13 spf13 was assigned by bep Aug 24, 2016
@bep bep referenced this pull request Aug 24, 2016
Closed

Finish the multilingual feature #2303

27 of 27 tasks complete
bep added some commits Aug 16, 2016
@bep bep Avoid splitting words for summary
For people using autogenerated summaries, this is one of the hot spots in the memory department.

We don't need to split al the content into words to do proper summary truncation.

This is obviously more effective:

```
BenchmarkTestTruncateWordsToWholeSentence-4            300000          4720 ns/op           0 B/op           0 allocs/op
BenchmarkTestTruncateWordsToWholeSentenceOld-4         100000         17699 ns/op        3072 B/op           3 allocs/op
```
44e1121
@bep bep Improve TotalWords counter func
It is obviously more efficient when we do not care about the actual words.

```
BenchmarkTotalWords-4            100000         18795 ns/op           0 B/op           0 allocs/op
BenchmarkTotalWordsOld-4          30000         46751 ns/op        6400 B/op           1 allocs/op
```
ea0e8c5
@bep bep Lazy calculate WordCount, ReadingTime and FuzzyWordCount
This avoids having to execute these expensive operations for sites not using these values.

This commit sums up a set of wordcounting and autosummary related performance improvements.

The effect of these kind of depends on what features your site use, but a benchmark from 4 Hugo sites in the wild shows promise:

```
benchmark           old ns/op       new ns/op       delta
BenchmarkHugo-4     21293005843     20032857342     -5.92%

benchmark           old allocs     new allocs     delta
BenchmarkHugo-4     65290922       65186032       -0.16%

benchmark           old bytes      new bytes      delta
BenchmarkHugo-4     9771213416     9681866464     -0.91%
```
dfdcbe0
@bep bep added a commit that closed this pull request Sep 14, 2016
@bep bep Lazy calculate WordCount, ReadingTime and FuzzyWordCount
This avoids having to execute these expensive operations for sites not using these values.

This commit sums up a set of wordcounting and autosummary related performance improvements.

The effect of these kind of depends on what features your site use, but a benchmark from 4 Hugo sites in the wild shows promise:

```
benchmark           old ns/op       new ns/op       delta
BenchmarkHugo-4     21293005843     20032857342     -5.92%

benchmark           old allocs     new allocs     delta
BenchmarkHugo-4     65290922       65186032       -0.16%

benchmark           old bytes      new bytes      delta
BenchmarkHugo-4     9771213416     9681866464     -0.91%
```

Closes #2378
dd45e6d
@bep bep closed this in dd45e6d Sep 14, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment