Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

html: reduce entity memory usage and init load #26727

Closed
bradfitz opened this issue Jul 31, 2018 · 2 comments
Closed

html: reduce entity memory usage and init load #26727

bradfitz opened this issue Jul 31, 2018 · 2 comments

Comments

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Jul 31, 2018

The html package has two large maps that are populated during an implicit init():

https://github.com/golang/go/blob/master/src/html/entity.go

Because of that, the memory is always in the heap and the linker can't discard it, even if the user never uses it. (via https://golang.org/pkg/html/#UnescapeString)

On linux/amd64 at tip, it adds 110 KB of heap just to import _ "html". A more realistic use case is code that only uses html.EscapeString. Such code should never need to run init() or retain that code in memory.

It seems this is a perfect use case of perfect hashing.

@cespare, it looks like your https://github.com/cespare/mph is nice & simple and suitably licensed.

It seems like we could pull entity.go into a gen-entity.go file and go generate an ugly zentity.go file that is just one big string literal, suitably packed.

Then there'd be no init work to construct the map, and the linker could discard it.

/cc @josharian

@gopherbot
Copy link

@gopherbot gopherbot commented Jul 31, 2018

Change https://golang.org/cl/127075 mentions this issue: html: lazily populate Unescape tables

Loading

@bradfitz
Copy link
Contributor Author

@bradfitz bradfitz commented Jul 31, 2018

Actually, https://go-review.googlesource.com/c/go/+/127075 ended up working too and is probably simpler.

Loading

@bradfitz bradfitz removed this from the Go1.12 milestone Jul 31, 2018
@bradfitz bradfitz added this to the Go1.11 milestone Jul 31, 2018
@gopherbot gopherbot closed this in 740e589 Jul 31, 2018
@golang golang locked and limited conversation to collaborators Jul 31, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants