New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slowish TOML frontmatter parsing #3541

Closed
bep opened this Issue May 30, 2017 · 2 comments

Comments

Projects
None yet
1 participant
@bep
Member

bep commented May 30, 2017

▶ go test -run="NONE" -bench="BenchmarkSiteBuilding/num_root_sections=1.*num_pages=1000.*shortcodes=false.*render=false"  -test.benchmem=true ./hugolib
BenchmarkSiteBuilding/frontmatter=YAML|num_root_sections=1|num_pages=1000|tags_per_page=0|shortcodes=false|render=false-4         	      10	 110588950 ns/op	71186219 B/op	  609989 allocs/op
BenchmarkSiteBuilding/frontmatter=YAML|num_root_sections=1|num_pages=1000|tags_per_page=1|shortcodes=false|render=false-4         	      10	 111165198 ns/op	71817096 B/op	  633223 allocs/op
BenchmarkSiteBuilding/frontmatter=YAML|num_root_sections=1|num_pages=1000|tags_per_page=5|shortcodes=false|render=false-4         	      10	 126947952 ns/op	73183604 B/op	  687713 allocs/op
BenchmarkSiteBuilding/frontmatter=YAML|num_root_sections=1|num_pages=1000|tags_per_page=20|shortcodes=false|render=false-4        	      10	 150970682 ns/op	79956410 B/op	  887734 allocs/op
BenchmarkSiteBuilding/frontmatter=TOML|num_root_sections=1|num_pages=1000|tags_per_page=0|shortcodes=false|render=false-4         	      10	 129999714 ns/op	74819504 B/op	  743911 allocs/op
BenchmarkSiteBuilding/frontmatter=TOML|num_root_sections=1|num_pages=1000|tags_per_page=1|shortcodes=false|render=false-4         	      10	 123858599 ns/op	75871137 B/op	  794189 allocs/op
BenchmarkSiteBuilding/frontmatter=TOML|num_root_sections=1|num_pages=1000|tags_per_page=5|shortcodes=false|render=false-4         	      10	 137144036 ns/op	81040482 B/op	 1025858 allocs/op
BenchmarkSiteBuilding/frontmatter=TOML|num_root_sections=1|num_pages=1000|tags_per_page=20|shortcodes=false|render=false-4        	      10	 206386658 ns/op	102723519 B/op	 1941946 allocs/op
@bep

This comment has been minimized.

Show comment
Hide comment
@bep

bep May 30, 2017

Member

Latest go-toml:

▶ go test -run="NONE" -bench="BenchmarkSiteBuilding/num_root_sections=1.*num_pages=1000.*tags_per_page=20.*shortcodes=false.*render=false"  -test.benchmem=true ./hugolib
BenchmarkSiteBuilding/frontmatter=YAML|num_root_sections=1|num_pages=1000|tags_per_page=20|shortcodes=false|render=false-4         	      10	 143671438 ns/op	79607541 B/op	  869858 allocs/op
BenchmarkSiteBuilding/frontmatter=TOML|num_root_sections=1|num_pages=1000|tags_per_page=20|shortcodes=false|render=false-4         	      10	 195634229 ns/op	99621562 B/op	 1746580 allocs/op
Member

bep commented May 30, 2017

Latest go-toml:

▶ go test -run="NONE" -bench="BenchmarkSiteBuilding/num_root_sections=1.*num_pages=1000.*tags_per_page=20.*shortcodes=false.*render=false"  -test.benchmem=true ./hugolib
BenchmarkSiteBuilding/frontmatter=YAML|num_root_sections=1|num_pages=1000|tags_per_page=20|shortcodes=false|render=false-4         	      10	 143671438 ns/op	79607541 B/op	  869858 allocs/op
BenchmarkSiteBuilding/frontmatter=TOML|num_root_sections=1|num_pages=1000|tags_per_page=20|shortcodes=false|render=false-4         	      10	 195634229 ns/op	99621562 B/op	 1746580 allocs/op

@bep bep changed the title from Slow TOML frontmatter parsing to Slowish TOML frontmatter parsing May 30, 2017

bep added a commit to bep/hugo that referenced this issue Jun 3, 2017

all: Temporarily revert to BurntSushi for TOML front matter handling
We still have go-toml as a transitive dependency, and it is the way to go eventually, but we care about speed, so let us wait that one out.

Note that the issue this fixes is about taxonomies, but I guess this is a general issue for sites with many pages that uses TOML as front matter.

```
benchmark                              old ns/op     new ns/op     delta
BenchmarkFrontmatterTags/TOML:1-4      23206         8543          -63.19%
BenchmarkFrontmatterTags/TOML:11-4     80117         18495         -76.92%
BenchmarkFrontmatterTags/TOML:21-4     140676        28727         -79.58%

benchmark                              old allocs     new allocs     delta
BenchmarkFrontmatterTags/TOML:1-4      173            60             -65.32%
BenchmarkFrontmatterTags/TOML:11-4     625            138            -77.92%
BenchmarkFrontmatterTags/TOML:21-4     1106           210            -81.01%

benchmark                              old bytes     new bytes     delta
BenchmarkFrontmatterTags/TOML:1-4      9231          2912          -68.45%
BenchmarkFrontmatterTags/TOML:11-4     19808         5184          -73.83%
BenchmarkFrontmatterTags/TOML:21-4     31200         7536          -75.85%
```

See #3541
Updates #3464

bep added a commit that referenced this issue Jun 3, 2017

all: Temporarily revert to BurntSushi for TOML front matter handling
We still have go-toml as a transitive dependency, and it is the way to go eventually, but we care about speed, so let us wait that one out.

Note that the issue this fixes is about taxonomies, but I guess this is a general issue for sites with many pages that uses TOML as front matter.

```
benchmark                              old ns/op     new ns/op     delta
BenchmarkFrontmatterTags/TOML:1-4      23206         8543          -63.19%
BenchmarkFrontmatterTags/TOML:11-4     80117         18495         -76.92%
BenchmarkFrontmatterTags/TOML:21-4     140676        28727         -79.58%

benchmark                              old allocs     new allocs     delta
BenchmarkFrontmatterTags/TOML:1-4      173            60             -65.32%
BenchmarkFrontmatterTags/TOML:11-4     625            138            -77.92%
BenchmarkFrontmatterTags/TOML:21-4     1106           210            -81.01%

benchmark                              old bytes     new bytes     delta
BenchmarkFrontmatterTags/TOML:1-4      9231          2912          -68.45%
BenchmarkFrontmatterTags/TOML:11-4     19808         5184          -73.83%
BenchmarkFrontmatterTags/TOML:21-4     31200         7536          -75.85%
```

See #3541
Updates #3464

@ghost ghost referenced this issue Jun 12, 2017

Open

spf13/hugo v0.22 released #11

@stale

This comment has been minimized.

Show comment
Hide comment
@stale

stale bot Dec 6, 2017

This issue has been automatically marked as stale because it has not had recent activity. The resources of the Hugo team are limited, and so we are asking for your help.
If this is a bug and you can still reproduce this error on the master branch, please reply with all of the information you have about it in order to keep the issue open.
If this is a feature request, and you feel that it is still relevant and valuable, please tell us why.
This issue will automatically be closed in the near future if no further activity occurs. Thank you for all your contributions.

stale bot commented Dec 6, 2017

This issue has been automatically marked as stale because it has not had recent activity. The resources of the Hugo team are limited, and so we are asking for your help.
If this is a bug and you can still reproduce this error on the master branch, please reply with all of the information you have about it in order to keep the issue open.
If this is a feature request, and you feel that it is still relevant and valuable, please tell us why.
This issue will automatically be closed in the near future if no further activity occurs. Thank you for all your contributions.

@stale stale bot added the Stale label Dec 6, 2017

@stale stale bot closed this Dec 27, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment