New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up taxonomies #3464

Open
bep opened this Issue May 14, 2017 · 2 comments

Comments

Projects
None yet
1 participant

@bep bep self-assigned this May 14, 2017

@bep

This comment has been minimized.

Show comment
Hide comment
@bep

bep May 30, 2017

Member

Running some benchmarks shows an increase in processing time and memory allocations that should not be needed. In the numbers below, we skip rendering.

▶  go test -run="NONE" -bench="BenchmarkSiteBuilding/num_root_sections=1.*tags_per_page=1.*shortcodes=false.*render=false" -test.benchmem=true ./hugolib
BenchmarkSiteBuilding/num_root_sections=1|num_pages=10|tags_per_page=1|shortcodes=false|render=false-4         	    1000	   1727309 ns/op	  782833 B/op	    8401 allocs/op
BenchmarkSiteBuilding/num_root_sections=1|num_pages=100|tags_per_page=1|shortcodes=false|render=false-4        	     100	  14908393 ns/op	 7612525 B/op	   79569 allocs/op
BenchmarkSiteBuilding/num_root_sections=1|num_pages=500|tags_per_page=1|shortcodes=false|render=false-4        	      20	  66474194 ns/op	37819696 B/op	  396557 allocs/op
BenchmarkSiteBuilding/num_root_sections=1|num_pages=1000|tags_per_page=1|shortcodes=false|render=false-4       	      10	 125817355 ns/op	75889336 B/op	  794273 allocs/op
BenchmarkSiteBuilding/num_root_sections=1|num_pages=5000|tags_per_page=1|shortcodes=false|render=false-4       	       2	 580504415 ns/op	379782296 B/op	 3981619 allocs/op
PASS
ok  	github.com/spf13/hugo/hugolib	15.329s

github.com/spf13/hugo  benchmark-site ✗                                                                                                                                                      3h53m ⚑
▶  go test -run="NONE" -bench="BenchmarkSiteBuilding/num_root_sections=1.*tags_per_page=5.*shortcodes=false.*render=false" -test.benchmem=true ./hugolib
BenchmarkSiteBuilding/num_root_sections=1|num_pages=10|tags_per_page=5|shortcodes=false|render=false-4         	    1000	   2058662 ns/op	  847723 B/op	   10865 allocs/op
BenchmarkSiteBuilding/num_root_sections=1|num_pages=100|tags_per_page=5|shortcodes=false|render=false-4        	     100	  15602630 ns/op	 8153934 B/op	  102924 allocs/op
BenchmarkSiteBuilding/num_root_sections=1|num_pages=500|tags_per_page=5|shortcodes=false|render=false-4        	      20	  73528038 ns/op	40466519 B/op	  512729 allocs/op
BenchmarkSiteBuilding/num_root_sections=1|num_pages=1000|tags_per_page=5|shortcodes=false|render=false-4       	      10	 140013838 ns/op	81163656 B/op	 1026429 allocs/op
BenchmarkSiteBuilding/num_root_sections=1|num_pages=5000|tags_per_page=5|shortcodes=false|render=false-4       	       2	 651779072 ns/op	406896348 B/op	 5141880 allocs/op
PASS
ok  	github.com/spf13/hugo/hugolib	16.270s

github.com/spf13/hugo  benchmark-site ✗                                                                                                                                                      3h54m ⚑
▶  go test -run="NONE" -bench="BenchmarkSiteBuilding/num_root_sections=1.*tags_per_page=20.*shortcodes=false.*render=false" -test.benchmem=true ./hugolib
BenchmarkSiteBuilding/num_root_sections=1|num_pages=10|tags_per_page=20|shortcodes=false|render=false-4         	     500	   2716379 ns/op	 1104154 B/op	   20366 allocs/op
BenchmarkSiteBuilding/num_root_sections=1|num_pages=100|tags_per_page=20|shortcodes=false|render=false-4        	      50	  23806261 ns/op	10351286 B/op	  193507 allocs/op
BenchmarkSiteBuilding/num_root_sections=1|num_pages=500|tags_per_page=20|shortcodes=false|render=false-4        	      10	 108307095 ns/op	51317777 B/op	  963909 allocs/op
BenchmarkSiteBuilding/num_root_sections=1|num_pages=1000|tags_per_page=20|shortcodes=false|render=false-4       	       5	 217514323 ns/op	102422284 B/op	 1926085 allocs/op
BenchmarkSiteBuilding/num_root_sections=1|num_pages=5000|tags_per_page=20|shortcodes=false|render=false-4       	       2	 987713311 ns/op	515377820 B/op	 9636490 allocs/op
PASS
ok  	github.com/spf13/hugo/hugolib	12.351s
Member

bep commented May 30, 2017

Running some benchmarks shows an increase in processing time and memory allocations that should not be needed. In the numbers below, we skip rendering.

▶  go test -run="NONE" -bench="BenchmarkSiteBuilding/num_root_sections=1.*tags_per_page=1.*shortcodes=false.*render=false" -test.benchmem=true ./hugolib
BenchmarkSiteBuilding/num_root_sections=1|num_pages=10|tags_per_page=1|shortcodes=false|render=false-4         	    1000	   1727309 ns/op	  782833 B/op	    8401 allocs/op
BenchmarkSiteBuilding/num_root_sections=1|num_pages=100|tags_per_page=1|shortcodes=false|render=false-4        	     100	  14908393 ns/op	 7612525 B/op	   79569 allocs/op
BenchmarkSiteBuilding/num_root_sections=1|num_pages=500|tags_per_page=1|shortcodes=false|render=false-4        	      20	  66474194 ns/op	37819696 B/op	  396557 allocs/op
BenchmarkSiteBuilding/num_root_sections=1|num_pages=1000|tags_per_page=1|shortcodes=false|render=false-4       	      10	 125817355 ns/op	75889336 B/op	  794273 allocs/op
BenchmarkSiteBuilding/num_root_sections=1|num_pages=5000|tags_per_page=1|shortcodes=false|render=false-4       	       2	 580504415 ns/op	379782296 B/op	 3981619 allocs/op
PASS
ok  	github.com/spf13/hugo/hugolib	15.329s

github.com/spf13/hugo  benchmark-site ✗                                                                                                                                                      3h53m ⚑
▶  go test -run="NONE" -bench="BenchmarkSiteBuilding/num_root_sections=1.*tags_per_page=5.*shortcodes=false.*render=false" -test.benchmem=true ./hugolib
BenchmarkSiteBuilding/num_root_sections=1|num_pages=10|tags_per_page=5|shortcodes=false|render=false-4         	    1000	   2058662 ns/op	  847723 B/op	   10865 allocs/op
BenchmarkSiteBuilding/num_root_sections=1|num_pages=100|tags_per_page=5|shortcodes=false|render=false-4        	     100	  15602630 ns/op	 8153934 B/op	  102924 allocs/op
BenchmarkSiteBuilding/num_root_sections=1|num_pages=500|tags_per_page=5|shortcodes=false|render=false-4        	      20	  73528038 ns/op	40466519 B/op	  512729 allocs/op
BenchmarkSiteBuilding/num_root_sections=1|num_pages=1000|tags_per_page=5|shortcodes=false|render=false-4       	      10	 140013838 ns/op	81163656 B/op	 1026429 allocs/op
BenchmarkSiteBuilding/num_root_sections=1|num_pages=5000|tags_per_page=5|shortcodes=false|render=false-4       	       2	 651779072 ns/op	406896348 B/op	 5141880 allocs/op
PASS
ok  	github.com/spf13/hugo/hugolib	16.270s

github.com/spf13/hugo  benchmark-site ✗                                                                                                                                                      3h54m ⚑
▶  go test -run="NONE" -bench="BenchmarkSiteBuilding/num_root_sections=1.*tags_per_page=20.*shortcodes=false.*render=false" -test.benchmem=true ./hugolib
BenchmarkSiteBuilding/num_root_sections=1|num_pages=10|tags_per_page=20|shortcodes=false|render=false-4         	     500	   2716379 ns/op	 1104154 B/op	   20366 allocs/op
BenchmarkSiteBuilding/num_root_sections=1|num_pages=100|tags_per_page=20|shortcodes=false|render=false-4        	      50	  23806261 ns/op	10351286 B/op	  193507 allocs/op
BenchmarkSiteBuilding/num_root_sections=1|num_pages=500|tags_per_page=20|shortcodes=false|render=false-4        	      10	 108307095 ns/op	51317777 B/op	  963909 allocs/op
BenchmarkSiteBuilding/num_root_sections=1|num_pages=1000|tags_per_page=20|shortcodes=false|render=false-4       	       5	 217514323 ns/op	102422284 B/op	 1926085 allocs/op
BenchmarkSiteBuilding/num_root_sections=1|num_pages=5000|tags_per_page=20|shortcodes=false|render=false-4       	       2	 987713311 ns/op	515377820 B/op	 9636490 allocs/op
PASS
ok  	github.com/spf13/hugo/hugolib	12.351s

@bep bep added the Enhancement label May 30, 2017

@bep bep changed the title from Check taxonomy creation algo performance to Speed up taxonomies May 30, 2017

@bep bep added this to the v0.22 milestone May 30, 2017

@bep

This comment has been minimized.

Show comment
Hide comment
@bep

bep May 31, 2017

Member

Current status:

▶ ./benchSite.sh "frontmatter=YAML,num_root_sections=1,num_pages=5000,tags_per_page=.*,shortcodes=false,render=false"
Running with BenchmarkSiteBuilding/frontmatter=YAML,num_root_sections=1,num_pages=5000,tags_per_page=.*,shortcodes=false,render=false
BenchmarkSiteBuilding/frontmatter=YAML,num_root_sections=1,num_pages=5000,tags_per_page=0,shortcodes=false,render=false-4         	       2	 525379904 ns/op	354275268 B/op	 2957422 allocs/op
BenchmarkSiteBuilding/frontmatter=YAML,num_root_sections=1,num_pages=5000,tags_per_page=1,shortcodes=false,render=false-4         	       2	 560668515 ns/op	355998064 B/op	 3065023 allocs/op
BenchmarkSiteBuilding/frontmatter=YAML,num_root_sections=1,num_pages=5000,tags_per_page=5,shortcodes=false,render=false-4         	       2	 583426642 ns/op	366766444 B/op	 3352551 allocs/op
BenchmarkSiteBuilding/frontmatter=YAML,num_root_sections=1,num_pages=5000,tags_per_page=20,shortcodes=false,render=false-4        	       2	 677635758 ns/op	400926544 B/op	 4339179 allocs/op
PASS
ok  	github.com/spf13/hugo/hugolib	7.577s

With TOML the number should be more comparable with the above:

▶ ./benchSite.sh "frontmatter=TOML,num_root_sections=1,num_pages=5000,tags_per_page=.*,shortcodes=false,render=false"
Running with BenchmarkSiteBuilding/frontmatter=TOML,num_root_sections=1,num_pages=5000,tags_per_page=.*,shortcodes=false,render=false
BenchmarkSiteBuilding/frontmatter=TOML,num_root_sections=1,num_pages=5000,tags_per_page=0,shortcodes=false,render=false-4         	       2	 569312727 ns/op	371997368 B/op	 3537972 allocs/op
BenchmarkSiteBuilding/frontmatter=TOML,num_root_sections=1,num_pages=5000,tags_per_page=1,shortcodes=false,render=false-4         	       2	 566052006 ns/op	375530568 B/op	 3755634 allocs/op
BenchmarkSiteBuilding/frontmatter=TOML,num_root_sections=1,num_pages=5000,tags_per_page=5,shortcodes=false,render=false-4         	       2	 625279854 ns/op	402182348 B/op	 4783130 allocs/op
BenchmarkSiteBuilding/frontmatter=TOML,num_root_sections=1,num_pages=5000,tags_per_page=20,shortcodes=false,render=false-4        	       2	 927723162 ns/op	503709824 B/op	 8749784 allocs/op
PASS
ok  	github.com/spf13/hugo/hugolib	8.551s

Which I guess shows that it is fairly stable for YAML even with 5000 pages with 20 tags each, but with TOML front matter it scales pretty poorly.

See #3541

Member

bep commented May 31, 2017

Current status:

▶ ./benchSite.sh "frontmatter=YAML,num_root_sections=1,num_pages=5000,tags_per_page=.*,shortcodes=false,render=false"
Running with BenchmarkSiteBuilding/frontmatter=YAML,num_root_sections=1,num_pages=5000,tags_per_page=.*,shortcodes=false,render=false
BenchmarkSiteBuilding/frontmatter=YAML,num_root_sections=1,num_pages=5000,tags_per_page=0,shortcodes=false,render=false-4         	       2	 525379904 ns/op	354275268 B/op	 2957422 allocs/op
BenchmarkSiteBuilding/frontmatter=YAML,num_root_sections=1,num_pages=5000,tags_per_page=1,shortcodes=false,render=false-4         	       2	 560668515 ns/op	355998064 B/op	 3065023 allocs/op
BenchmarkSiteBuilding/frontmatter=YAML,num_root_sections=1,num_pages=5000,tags_per_page=5,shortcodes=false,render=false-4         	       2	 583426642 ns/op	366766444 B/op	 3352551 allocs/op
BenchmarkSiteBuilding/frontmatter=YAML,num_root_sections=1,num_pages=5000,tags_per_page=20,shortcodes=false,render=false-4        	       2	 677635758 ns/op	400926544 B/op	 4339179 allocs/op
PASS
ok  	github.com/spf13/hugo/hugolib	7.577s

With TOML the number should be more comparable with the above:

▶ ./benchSite.sh "frontmatter=TOML,num_root_sections=1,num_pages=5000,tags_per_page=.*,shortcodes=false,render=false"
Running with BenchmarkSiteBuilding/frontmatter=TOML,num_root_sections=1,num_pages=5000,tags_per_page=.*,shortcodes=false,render=false
BenchmarkSiteBuilding/frontmatter=TOML,num_root_sections=1,num_pages=5000,tags_per_page=0,shortcodes=false,render=false-4         	       2	 569312727 ns/op	371997368 B/op	 3537972 allocs/op
BenchmarkSiteBuilding/frontmatter=TOML,num_root_sections=1,num_pages=5000,tags_per_page=1,shortcodes=false,render=false-4         	       2	 566052006 ns/op	375530568 B/op	 3755634 allocs/op
BenchmarkSiteBuilding/frontmatter=TOML,num_root_sections=1,num_pages=5000,tags_per_page=5,shortcodes=false,render=false-4         	       2	 625279854 ns/op	402182348 B/op	 4783130 allocs/op
BenchmarkSiteBuilding/frontmatter=TOML,num_root_sections=1,num_pages=5000,tags_per_page=20,shortcodes=false,render=false-4        	       2	 927723162 ns/op	503709824 B/op	 8749784 allocs/op
PASS
ok  	github.com/spf13/hugo/hugolib	8.551s

Which I guess shows that it is fairly stable for YAML even with 5000 pages with 20 tags each, but with TOML front matter it scales pretty poorly.

See #3541

bep added a commit that referenced this issue Jun 3, 2017

parser: Add BenchmarkFrontmatterTags
The list handling is surprisingly expensive:

```
▶ go test -run="NONE" -bench="BenchmarkFrontmatterTags" -test.benchmem=true ./parser | prettybench
PASS
benchmark                               iter         time/iter   bytes alloc           allocs
---------                               ----         ---------   -----------           ------
BenchmarkFrontmatterTags/JSON:1-4    1000000     2039.00 ns/op      912 B/op     20 allocs/op
BenchmarkFrontmatterTags/JSON:11-4    300000     5202.00 ns/op     1640 B/op     44 allocs/op
BenchmarkFrontmatterTags/JSON:21-4    200000     7993.00 ns/op     2392 B/op     65 allocs/op
BenchmarkFrontmatterTags/YAML:1-4     200000     9359.00 ns/op     5928 B/op     66 allocs/op
BenchmarkFrontmatterTags/YAML:11-4    100000    21218.00 ns/op     8408 B/op    140 allocs/op
BenchmarkFrontmatterTags/YAML:21-4     50000    32852.00 ns/op    10920 B/op    211 allocs/op
BenchmarkFrontmatterTags/TOML:1-4     100000    21505.00 ns/op     9231 B/op    173 allocs/op
BenchmarkFrontmatterTags/TOML:11-4     20000    82919.00 ns/op    19808 B/op    625 allocs/op
BenchmarkFrontmatterTags/TOML:21-4     10000   141847.00 ns/op    31200 B/op   1106 allocs/op
ok      github.com/spf13/hugo/parser    17.890s
```

See #3464

bep added a commit to bep/hugo that referenced this issue Jun 3, 2017

all: Temporarily revert to BurntSushi for TOML front matter handling
We still have go-toml as a transitive dependency, and it is the way to go eventually, but we care about speed, so let us wait that one out.

Note that the issue this fixes is about taxonomies, but I guess this is a general issue for sites with many pages that uses TOML as front matter.

```
benchmark                              old ns/op     new ns/op     delta
BenchmarkFrontmatterTags/TOML:1-4      23206         8543          -63.19%
BenchmarkFrontmatterTags/TOML:11-4     80117         18495         -76.92%
BenchmarkFrontmatterTags/TOML:21-4     140676        28727         -79.58%

benchmark                              old allocs     new allocs     delta
BenchmarkFrontmatterTags/TOML:1-4      173            60             -65.32%
BenchmarkFrontmatterTags/TOML:11-4     625            138            -77.92%
BenchmarkFrontmatterTags/TOML:21-4     1106           210            -81.01%

benchmark                              old bytes     new bytes     delta
BenchmarkFrontmatterTags/TOML:1-4      9231          2912          -68.45%
BenchmarkFrontmatterTags/TOML:11-4     19808         5184          -73.83%
BenchmarkFrontmatterTags/TOML:21-4     31200         7536          -75.85%
```

See #3541
Updates #3464

@bep bep closed this in #3545 Jun 3, 2017

bep added a commit that referenced this issue Jun 3, 2017

all: Temporarily revert to BurntSushi for TOML front matter handling
We still have go-toml as a transitive dependency, and it is the way to go eventually, but we care about speed, so let us wait that one out.

Note that the issue this fixes is about taxonomies, but I guess this is a general issue for sites with many pages that uses TOML as front matter.

```
benchmark                              old ns/op     new ns/op     delta
BenchmarkFrontmatterTags/TOML:1-4      23206         8543          -63.19%
BenchmarkFrontmatterTags/TOML:11-4     80117         18495         -76.92%
BenchmarkFrontmatterTags/TOML:21-4     140676        28727         -79.58%

benchmark                              old allocs     new allocs     delta
BenchmarkFrontmatterTags/TOML:1-4      173            60             -65.32%
BenchmarkFrontmatterTags/TOML:11-4     625            138            -77.92%
BenchmarkFrontmatterTags/TOML:21-4     1106           210            -81.01%

benchmark                              old bytes     new bytes     delta
BenchmarkFrontmatterTags/TOML:1-4      9231          2912          -68.45%
BenchmarkFrontmatterTags/TOML:11-4     19808         5184          -73.83%
BenchmarkFrontmatterTags/TOML:21-4     31200         7536          -75.85%
```

See #3541
Updates #3464

@bep bep reopened this Jun 3, 2017

@bep bep modified the milestones: v0.23, v0.22 Jun 7, 2017

@ghost ghost referenced this issue Jun 12, 2017

Open

spf13/hugo v0.22 released #11

@bep bep modified the milestones: v0.23, v0.24, v0.25 Jun 16, 2017

@bep bep modified the milestones: v0.25, v0.26 Jul 3, 2017

@bep bep modified the milestones: v0.26, v0.27 Aug 6, 2017

@bep bep modified the milestones: v0.27, v0.28 Sep 7, 2017

@bep bep modified the milestones: v0.28, v0.29, v0.30 Sep 21, 2017

@bep bep modified the milestones: v0.30, v0.31 Oct 13, 2017

@bep bep modified the milestones: v0.31, v0.32 Oct 30, 2017

@bep bep modified the milestones: v0.32, v0.33 Dec 16, 2017

@bep bep modified the milestones: v0.33, v0.34 Jan 11, 2018

@bep bep modified the milestones: v0.34, v0.35, v0.36 Jan 22, 2018

@bep bep modified the milestones: v0.36, v0.37 Feb 3, 2018

@bep bep modified the milestones: v0.37, v0.38 Feb 11, 2018

@bep bep modified the milestones: v0.38, v0.39 Feb 21, 2018

@bep bep modified the milestones: v0.39, v0.40 Apr 9, 2018

@bep bep modified the milestones: v0.40, v0.41 Apr 20, 2018

@bep bep modified the milestones: v0.41, v0.42 May 4, 2018

@bep bep modified the milestones: v0.42, v0.43 Jun 5, 2018

@bep bep modified the milestones: v0.43, v0.44 Jun 30, 2018

@bep bep modified the milestones: v0.44, v0.45, v0.46 Jul 10, 2018

@bep bep modified the milestones: v0.46, v0.47, v0.48 Aug 3, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment