Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/text: panic of index out of range in language.Matcher #26580

Open
yuanbai opened this issue Jul 25, 2018 · 4 comments
Open

x/text: panic of index out of range in language.Matcher #26580

yuanbai opened this issue Jul 25, 2018 · 4 comments

Comments

@yuanbai
Copy link

@yuanbai yuanbai commented Jul 25, 2018

What version of Go are you using (go version)?

go 1.9

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

amd64, darwin and linux.

What did you do?

language.MustParse accepts "lang_ZZ", e.g. en_ZZ, es_ZZ, etc. and returns a valid tag.
but m.Match(such a language tag) triggers a panic of "index out of range" at this lookup table: https://github.com/golang/text/blob/master/language/tables.go#L50

Sample code to reproduce the issue:

package main

import (
	"fmt"
	"golang.org/x/text/language"
)

func main() {
	supportedLanguages := []language.Tag{
		language.English,
		language.Arabic,
		language.BritishEnglish,
		language.Spanish,
		language.French,
		language.Italian,
		language.Japanese,
		language.Korean,
		language.Portuguese,
		language.Russian,
		language.SimplifiedChinese,
		language.TraditionalChinese,
	}

	tl := language.MustParse("en_ZZ")
	fmt.Printf("target language: %v\n", tl)

	m := language.NewMatcher(supportedLanguages)
	_, idx, _ := m.Match(tl)

	fmt.Printf("index matched: %v\n", idx)
}

Output:

target language: en-ZZ
panic: runtime error: index out of range

goroutine 1 [running]:
golang.org/x/text/language.regionGroupDist(0x139005701650135, 0x1650139)
	/Users/yb/go_root/src/golang.org/x/text/language/match.go:677 +0x13d
golang.org/x/text/language.(*bestMatch).update(0xc42003fbf0, 0xc4200722d0, 0x1650139, 0x0, 0x0, 0x101650057)
	/Users/ybi/go_root/src/golang.org/x/text/language/match.go:622 +0x28d
golang.org/x/text/language.(*matcher).getBest(0xc420072270, 0xc42000adc0, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/Users/yb/go_root/src/golang.org/x/text/language/match.go:504 +0x4a0
golang.org/x/text/language.(*matcher).Match(0xc420072270, 0xc42000adc0, 0x1, 0x1, 0x0, 0x0, 0x1161220, 0xc420072270, 0x2d952e1a19cc2e08)
	/Users/yb/go_root/src/golang.org/x/text/language/match.go:83 +0xd4
main.main()
	/Users/yb/test.go:28 +0x4d5
exit status 2

What did you expect to see?

either
(1) language.MustParse rejects "lang_ZZ" and throws an error;
or
(2) language.MustParse parses "lang_ZZ" to a tag of just "lang", similar to "lang_UND", e.g. language.MustParse of "en_ZZ" and "en" should return the identical result.
or
(3) language.Matcher supports ZZ country code and works properly.

What did you see instead?

language.MustParse accepts ZZ, but language.Matcher crashes on that.

@gopherbot gopherbot added this to the Unreleased milestone Jul 25, 2018
@schultz9999

This comment has been minimized.

Copy link

@schultz9999 schultz9999 commented Sep 12, 2018

Same reproduces in 1.10.3 darwin/amd64. Fails due to bad index:

image

@alexcons

This comment has been minimized.

Copy link

@alexcons alexcons commented Nov 28, 2018

This is still happening even on 1.11.1 darwin/amd64

@agnivade

This comment has been minimized.

Copy link
Contributor

@agnivade agnivade commented Nov 28, 2018

Yes, because it is not fixed yet.

/cc @mpvl

@rlk833

This comment has been minimized.

Copy link

@rlk833 rlk833 commented Nov 11, 2019

Has this been fixed yet? It is a year later and I'm at go1.12.4 darwin/amd64 and it is occurring to my code.

What is actually wrong with the input? Maybe if I knew what was wrong with the input I could screen for it.

This is our production code used by hundreds of different people so I don't know in each case what is sent in by them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants
You can’t perform that action at this time.