New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make summary, wordcount etc. more effective #2378

Closed
wants to merge 3 commits into
base: master
from

Conversation

Projects
None yet
2 participants
@bep
Member

bep commented Aug 20, 2016

In total, the effect with this branch compared with master:

benchmark           old ns/op      new ns/op      delta
BenchmarkHugo-4     3166246627     3038050615     -4.05%

benchmark           old allocs     new allocs     delta
BenchmarkHugo-4     10557420       10540152       -0.16%

benchmark           old bytes      new bytes      delta
BenchmarkHugo-4     1621203992     1605716520     -0.96%

See: https://github.com/bep/hugo-benchmark

Word counting benchmark:

BenchmarkTotalWords-4            100000         18795 ns/op           0  B/op           0 allocs/op 
BenchmarkTotalWordsOld-4          30000         46751 ns/op           6400 B/op           1 allocs/op

New auto-summary benchmark:

BenchmarkTestTruncateWordsToWholeSentence-4            300000         4720 ns/op         0 B/op              0 allocs/op
BenchmarkTestTruncateWordsToWholeSentenceOld-4         100000         17699 ns/op        3072 B/op           3 allocs/op

@bep bep changed the title from WIP: Make summary, wordcount more effective to WIP: Make summary, wordcount etc. more effective Aug 20, 2016

@bep bep changed the title from WIP: Make summary, wordcount etc. more effective to Make summary, wordcount etc. more effective Aug 24, 2016

@bep bep added NeedsReview and removed status/in progress labels Aug 24, 2016

@bep bep referenced this pull request Aug 24, 2016

Closed

Finish the multilingual feature #2303

27 of 27 tasks complete

bep added some commits Aug 16, 2016

Avoid splitting words for summary
For people using autogenerated summaries, this is one of the hot spots in the memory department.

We don't need to split al the content into words to do proper summary truncation.

This is obviously more effective:

```
BenchmarkTestTruncateWordsToWholeSentence-4            300000          4720 ns/op           0 B/op           0 allocs/op
BenchmarkTestTruncateWordsToWholeSentenceOld-4         100000         17699 ns/op        3072 B/op           3 allocs/op
```
Improve TotalWords counter func
It is obviously more efficient when we do not care about the actual words.

```
BenchmarkTotalWords-4            100000         18795 ns/op           0 B/op           0 allocs/op
BenchmarkTotalWordsOld-4          30000         46751 ns/op        6400 B/op           1 allocs/op
```
Lazy calculate WordCount, ReadingTime and FuzzyWordCount
This avoids having to execute these expensive operations for sites not using these values.

This commit sums up a set of wordcounting and autosummary related performance improvements.

The effect of these kind of depends on what features your site use, but a benchmark from 4 Hugo sites in the wild shows promise:

```
benchmark           old ns/op       new ns/op       delta
BenchmarkHugo-4     21293005843     20032857342     -5.92%

benchmark           old allocs     new allocs     delta
BenchmarkHugo-4     65290922       65186032       -0.16%

benchmark           old bytes      new bytes      delta
BenchmarkHugo-4     9771213416     9681866464     -0.91%
```

@bep bep closed this in dd45e6d Sep 14, 2016

@bep bep deleted the bep:truncate-whole-sentence branch Apr 18, 2017

tychoish added a commit to tychoish/hugo that referenced this pull request Aug 13, 2017

Lazy calculate WordCount, ReadingTime and FuzzyWordCount
This avoids having to execute these expensive operations for sites not using these values.

This commit sums up a set of wordcounting and autosummary related performance improvements.

The effect of these kind of depends on what features your site use, but a benchmark from 4 Hugo sites in the wild shows promise:

```
benchmark           old ns/op       new ns/op       delta
BenchmarkHugo-4     21293005843     20032857342     -5.92%

benchmark           old allocs     new allocs     delta
BenchmarkHugo-4     65290922       65186032       -0.16%

benchmark           old bytes      new bytes      delta
BenchmarkHugo-4     9771213416     9681866464     -0.91%
```

Closes gohugoio#2378
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment