Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/link: unexpectedly long symbol name #15104

Closed
bradfitz opened this issue Apr 4, 2016 · 11 comments

Comments

Projects
None yet
5 participants
@bradfitz
Copy link
Member

commented Apr 4, 2016

I caused a regression in 73edd7b (cmd/link: simplify readSymName, taking advantage of bufio.Reader) It made an assumption that symbol names were less than 4kb.

Failure seen here:

https://storage.googleapis.com/go-build-log/1f5b1b2b/linux-amd64_8877cb41.log

# testmain
2016/04/04 12:24:45 $WORK/golang.org/x/text/unicode/cldr.a(_go_.o): unexpectedly long symbol name
FAIL    golang.org/x/text/internal/number [build failed]
ok      golang.org/x/text/internal/stringset    0.010s
ok      golang.org/x/text/internal/tag  0.003s
?       golang.org/x/text/internal/testtext [no test files]
ok      golang.org/x/text/internal/triegen  0.395s
ok      golang.org/x/text/internal/ucd  0.003s
?       golang.org/x/text/internal/utf8internal [no test files]
ok      golang.org/x/text/language  0.037s
# testmain
2016/04/04 12:24:49 $WORK/golang.org/x/text/unicode/cldr.a(_go_.o): unexpectedly long symbol name
FAIL    golang.org/x/text/language/display [build failed]
ok      golang.org/x/text/message   0.010s
ok      golang.org/x/text/runes 0.009s
ok      golang.org/x/text/search    0.012s
?       golang.org/x/text/secure    [no test files]
ok      golang.org/x/text/secure/precis 0.011s
ok      golang.org/x/text/transform 0.028s
?       golang.org/x/text/unicode   [no test files]
# testmain
2016/04/04 12:24:52 $WORK/golang.org/x/text/unicode/cldr.a(_go_.o): unexpectedly long symbol name
FAIL    golang.org/x/text/unicode/bidi [build failed]
# testmain
2016/04/04 12:24:54 $WORK/golang.org/x/text/unicode/cldr.a(_go_.o): unexpectedly long symbol name
# testmain
2016/04/04 12:24:54 $WORK/golang.org/x/text/unicode/cldr/_test/golang.org/x/text/unicode/cldr.a(_go_.o): unexpectedly long symbol name
FAIL    golang.org/x/text/unicode/cldr [build failed]
FAIL    golang.org/x/text/unicode/norm [build failed]
# testmain
2016/04/04 12:24:55 $WORK/golang.org/x/text/unicode/cldr.a(_go_.o): unexpectedly long symbol name
ok      golang.org/x/text/unicode/rangetable    1.353s
FAIL    golang.org/x/text/width [build failed]

/cc @crawshaw @nigeltao

@bradfitz bradfitz self-assigned this Apr 4, 2016

@bradfitz bradfitz added this to the Go1.7 milestone Apr 4, 2016

@crawshaw

This comment has been minimized.

Copy link
Contributor

commented Apr 4, 2016

I would love to see that symbol name. (It's a bug because big data should be SHAed into the name.)

Given that with position-independent code we have to keep a lot of symbol names around, I've been meaning to survey them, just haven't got there yet.

@bradfitz

This comment has been minimized.

Copy link
Member Author

commented Apr 4, 2016

Unfortunately I can't reproduce.

/cc @mpvl

@bradfitz

This comment has been minimized.

Copy link
Member Author

commented Apr 4, 2016

Oh, I have to go test golang.org/x/text/... to reproduce, not go build.

@bradfitz

This comment has been minimized.

Copy link
Member Author

commented Apr 4, 2016

The symbol name is 6K:

2016/04/04 15:48:29 $WORK/golang.org/x/text/unicode/cldr/_test/golang.org/x/text/unicode/cldr.a(go.o): unexpectedly long symbol name of 6397 bytes in golang.org/x/text/unicode/cldr: go.typelink.struct { cldr.Common; AttributeOrder *cldr.Common "xml:\"attributeOrder\""; ElementOrder *cldr.Common "xml:\"elementOrder\""; SerialElements *cldr.Common "xml:\"serialElements\""; Suppress *struct { cldr.Common; Attributes []*struct { cldr.Common; Element string "xml:\"element,attr\""; Attribute string "xml:\"attribute,attr\""; AttributeValue string "xml:\"attributeValue,attr\"" } "xml:\"attributes\"" } "xml:\"suppress\""; Validity *struct { cldr.Common; Variable []*struct { cldr.Common; Id string "xml:\"id,attr\"" } "xml:\"variable\""; AttributeValues []*struct { cldr.Common; Dtds string "xml:\"dtds,attr\""; Elements string "xml:\"elements,attr\""; Attributes string "xml:\"attributes,attr\""; Order string "xml:\"order,attr\"" } "xml:\"attributeValues\"" } "xml:\"validity\""; Alias *struct { cldr.Common; LanguageAlias []*struct { cldr.Common; Replacement string "xml:\"replacement,attr\""; Reason string "xml:\"reason,attr\"" } "xml:\"languageAlias\""; ScriptAlias []*struct { cldr.Common; Replacement string "xml:\"replacement,attr\""; Reason string "xml:\"reason,attr\"" } "xml:\"scriptAlias\""; TerritoryAlias []*struct { cldr.Common; Replacement string "xml:\"replacement,attr\""; Reason string "xml:\"reason,attr\"" } "xml:\"territoryAlias\""; SubdivisionAlias []*struct { cldr.Common; Replacement string "xml:\"replacement,attr\""; Reason string "xml:\"reason,attr\"" } "xml:\"subdivisionAlias\""; VariantAlias []*struct { cldr.Common; Replacement string "xml:\"replacement,attr\""; Reason string "xml:\"reason,attr\"" } "xml:\"variantAlias\""; ZoneAlias []*struct { cldr.Common; Replacement string "xml:\"replacement,attr\""; Reason string "xml:\"reason,attr\"" } "xml:\"zoneAlias\"" } "xml:\"alias\""; Deprecated *struct { cldr.Common; DeprecatedItems []*struct { cldr.Common; Elements string "xml:\"elements,attr\""; Attributes string "xml:\"attributes,attr\""; Values string "xml:\"values,attr\"" } "xml:\"deprecatedItems\"" } "xml:\"deprecated\""; Distinguishing *struct { cldr.Common; DistinguishingItems []*struct { cldr.Common; Exclude string "xml:\"exclude,attr\""; Elements string "xml:\"elements,attr\""; Attributes string "xml:\"attributes,attr\"" } "xml:\"distinguishingItems\"" } "xml:\"distinguishing\""; Blocking *struct { cldr.Common; BlockingItems []*struct { cldr.Common; Elements string "xml:\"elements,attr\"" } "xml:\"blockingItems\"" } "xml:\"blocking\""; CoverageAdditions *struct { cldr.Common; LanguageCoverage []*struct { cldr.Common; Values string "xml:\"values,attr\"" } "xml:\"languageCoverage\""; ScriptCoverage []*struct { cldr.Common; Values string "xml:\"values,attr\"" } "xml:\"scriptCoverage\""; TerritoryCoverage []*struct { cldr.Common; Values string "xml:\"values,attr\"" } "xml:\"territoryCoverage\""; CurrencyCoverage []*struct { cldr.Common; Values string "xml:\"values,attr\"" } "xml:\"currencyCoverage\""; TimezoneCoverage []*struct { cldr.Common; Values string "xml:\"values,attr\"" } "xml:\"timezoneCoverage\"" } "xml:\"coverageAdditions\""; SkipDefaultLocale *struct { cldr.Common; Services string "xml:\"services,attr\"" } "xml:\"skipDefaultLocale\""; DefaultContent *struct { cldr.Common; Locales string "xml:\"locales,attr\"" } "xml:\"defaultContent\"" } struct { "".Common; AttributeOrder *"".Common "xml:\"attributeOrder\""; ElementOrder *"".Common "xml:\"elementOrder\""; SerialElements *"".Common "xml:\"serialElements\""; Suppress *struct { "".Common; Attributes []*struct { "".Common; Element string "xml:\"element,attr\""; Attribute string "xml:\"attribute,attr\""; AttributeValue string "xml:\"attributeValue,attr\"" } "xml:\"attributes\"" } "xml:\"suppress\""; Validity *struct { "".Common; Variable []*struct { "".Common; Id string "xml:\"id,attr\"" } "xml:\"variable\""; AttributeValues []*struct { "".Common; Dtds string "xml:\"dtds,attr\""; Elements string "xml:\"elements,attr\""; Attributes string "xml:\"attributes,attr\""; Order string "xml:\"order,attr\"" } "xml:\"attributeValues\"" } "xml:\"validity\""; Alias *struct { "".Common; LanguageAlias []*struct { "".Common; Replacement string "xml:\"replacement,attr\""; Reason string "xml:\"reason,attr\"" } "xml:\"languageAlias\""; ScriptAlias []*struct { "".Common; Replacement string "xml:\"replacement,attr\""; Reason string "xml:\"reason,attr\"" } "xml:\"scriptAlias\""; TerritoryAlias []*struct { "".Common; Replacement string "xml:\"replacement,attr\""; Reason string "xml:\"reason,attr\"" } "xml:\"territoryAlias\""; SubdivisionAlias []*struct { "".Common; Replacement string "xml:\"replacement,attr\""; Reason string "xml:\"reason,attr\"" } "xml:\"subdivisionAlias\""; VariantAlias []*struct { "".Common; Replacement string "xml:\"replacement,attr\""; Reason string "xml:\"reason,attr\"" } "xml:\"variantAlias\""; ZoneAlias []*struct { "".Common; Replacement string "xml:\"replacement,attr\""; Reason string "xml:\"reason,attr\"" } "xml:\"zoneAlias\"" } "xml:\"alias\""; Deprecated *struct { "".Common; DeprecatedItems []*struct { "".Common; Elements string "xml:\"elements,attr\""; Attributes string "xml:\"attributes,attr\""; Values string "xml:\"values,attr\"" } "xml:\"deprecatedItems\"" } "xml:\"deprecated\""; Distinguishing *struct { "".Common; DistinguishingItems []*struct { "".Common; Exclude string "xml:\"exclude,attr\""; Elements string "xml:\"elements,attr\""; Attributes string "xml:\"attributes,attr\"" } "xml:\"distinguishingItems\"" } "xml:\"distinguishing\""; Blocking *struct { "".Common; BlockingItems []*struct { "".Common; Elements string "xml:\"elements,attr\"" } "xml:\"blockingItems\"" } "xml:\"blocking\""; CoverageAdditions *struct { "".Common; LanguageCoverage []*struct { "".Common; Values string "xml:\"values,attr\"" } "xml:\"languageCoverage\""; ScriptCoverage []*struct { "".Common; Values string "xml:\"values,attr\"" } "xml:\"scriptCoverage\""; TerritoryCoverage []*struct { "".Common; Values string "xml:\"values,attr\"" } "xml:\"territoryCoverage\""; CurrencyCoverage []*struct { "".Common; Values string "xml:\"values,attr\"" } "xml:\"currencyCoverage\""; TimezoneCoverage []*struct { "".Common; Values string "xml:\"values,attr\"" } "xml:\"timezoneCoverage\"" } "xml:\"coverageAdditions\""; SkipDefaultLocale *struct { "".Common; Services string "xml:\"services,attr\"" } "xml:\"skipDefaultLocale\""; DefaultContent *struct { "".Common; Locales string "xml:\"locales,attr\"" } "xml:\"defaultContent\"" }

@bradfitz

This comment has been minimized.

Copy link
Member Author

commented Apr 4, 2016

"Fix" the linker, or fix the compiler? @crawshaw, you want this one?

/cc @josharian

@crawshaw crawshaw assigned crawshaw and unassigned bradfitz Apr 4, 2016

@gopherbot

This comment has been minimized.

Copy link

commented Apr 4, 2016

CL https://golang.org/cl/21457 mentions this issue.

@mpvl

This comment has been minimized.

Copy link
Member

commented Apr 4, 2016

Yes, all this code is auto-generated. Lots of nested structs. I could hoist some of the nested structs, which should reduce it a bit, but that would be very far from ideal. It should be allowed in my opinion.

@bradfitz

This comment has been minimized.

Copy link
Member Author

commented Apr 4, 2016

@mpvl, nobody has suggested you need to change your code.

@thesyncim

This comment has been minimized.

Copy link

commented Apr 6, 2016

same problem here

@crawshaw

This comment has been minimized.

Copy link
Contributor

commented Apr 6, 2016

I had a followup to cl/21457 that introduces a new type printer for the type.* symbol names. It takes the type with placeholders for the package names and runs it through SHA-1, then prints the first few bytes of the hash (like we do with git commits). It then appends all the package names that appear in the type in appearance order.

The result is consistently small types and slightly less IO during compilation. (My secondary motivation was a stopgap measure for shrinking the symbol table for -buildmode=shared, but I now have a much simpler hack for that I'm going to send later today.) Unfortunately it also makes the toolchain just a little harder to debug. Given it's already pretty hard to understand and debug, I don't think it's worth it.

So I'd rather just land cl/21457 and make the linker's object reader handle long symbol names. I'll send another CL for that in a bit.

@gopherbot

This comment has been minimized.

Copy link

commented Apr 6, 2016

CL https://golang.org/cl/21581 mentions this issue.

AkihiroSuda added a commit to osrg/namazu that referenced this issue Apr 7, 2016

@gopherbot gopherbot closed this in 79048df Apr 13, 2016

gopherbot pushed a commit that referenced this issue Apr 18, 2016

cmd/link, cmd/compile: typelink sorting in linker
Instead of writing out the type almost twice in the symbol name,
teach the linker how to sort typelink symbols by their contents.

This ~halves the size of typelink symbol names, which helps very
large (6KB) names like those mentioned in #15104.

This does not increase the total sorting work done by the linker,
and makes it possible to use shorter symbol names for types. See
the follow-on CL 21583.

Change-Id: Ie5807565ed07d31bc477d20f60e4c0b47144f337
Reviewed-on: https://go-review.googlesource.com/21457
Reviewed-by: Ian Lance Taylor <iant@golang.org>

@golang golang locked and limited conversation to collaborators Apr 14, 2017

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.