Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

unicode: support version 11 #27945

Closed
fgm opened this issue Sep 30, 2018 · 20 comments
Closed

unicode: support version 11 #27945

fgm opened this issue Sep 30, 2018 · 20 comments
Labels
FrozenDueToAge help wanted
Milestone
Go1.13

Comments

@fgm
Copy link

fgm commented Sep 30, 2018

see http://blog.unicode.org/2018/06/announcing-unicode-standard-version-110.html for the announcement and further information about what changed.

What version of Go are you using (go version)?
go version go1.11 darwin/amd64

What operating system and processor architecture are you using (go env)?
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/fgm/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/fgm/src/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/0g/p780bc554njc4qj110_8rmbr0000gn/T/go-build464389101=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?
println(unicode.Version)

as in https://play.golang.org/p/k69zvpQ97S

What did you expect to see?
11.0.0

What did you see instead?
10.0.0

This is a followup to #21471
@agnivade
Copy link
Contributor

/cc @mpvl

@ALTree ALTree added this to the Go1.12 milestone Sep 30, 2018
@odeke-em
Copy link
Member

odeke-em commented Feb 4, 2019

Thank you for filing the feature request report @fgm and welcome to the Go project!

We didn't get to work on this for Go1.12 so I'll move it to Go1.13 (or even Unreleased) as it is presumably a big change but also @mpvl is quite occupied.

@odeke-em odeke-em modified the milestones: Go1.12, Go1.13 Feb 4, 2019
@odeke-em
Copy link
Member

@mpvl just recently mailed https://go-review.googlesource.com/c/text/+/154440/ so we can say the work has began.

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/154441 mentions this issue: internal/export/idna: hoist conformance test

gopherbot pushed a commit to golang/text that referenced this issue Mar 27, 2019
@mpvl
internal/export/idna: hoist conformance test
e3703dc 
in preparation to deal with the new file format
in Unicode 11.

Updates golang/go#27945

Change-Id: I424b3891e852c87bea536646a98a96aae2446294
Reviewed-on: https://go-review.googlesource.com/c/text/+/154441
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@gopherbot
Copy link
Contributor

Change https://golang.org/cl/154443 mentions this issue: internal/export/unicode: simplified generation

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/169617 mentions this issue: unicode: update table using new generator in x/text

@mpvl
Copy link
Contributor

mpvl commented Mar 27, 2019

@odeke-em Just bear with me. This is going to be a very large change with CLs across core and x/text this time.

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/169637 mentions this issue: gen.go: update due to changes to core

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/154418 mentions this issue: vendor/golang.org/x: update generation for Unicode 10.0.0

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/169638 mentions this issue: internal/export/unicode: add scripts tests

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/169639 mentions this issue: cases: add bit for exception index

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/169640 mentions this issue: cases: titlecase may differ from lower case

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/169619 mentions this issue: unicode: remove script test

@odeke-em
Copy link
Member

@odeke-em Just bear with me. This is going to be a very large change with CLs across core and x/text this time.

@mpvl great and thank you! No rush at all :)

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/172757 mentions this issue: all: update to Unicode 11

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/154420 mentions this issue: all: update to Unicode 11

gopherbot pushed a commit to golang/text that referenced this issue Apr 24, 2019
@mpvl
internal/export/unicode: simplified generation
e46505f 
by using x/text’s infrastructure

Updates golang/go#27945

Change-Id: I070afd9cad4e367ac9085a66064f2bb47e5e225a
Reviewed-on: https://go-review.googlesource.com/c/text/+/154443
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
gopherbot pushed a commit to golang/text that referenced this issue Apr 24, 2019
@mpvl
internal/export/unicode: add scripts tests
c40a8cc 
This is to replace the script test in core that is used
to verify the generation, but it interupts the flow of
automation as they require manual upgrades.
This brute-force approach is better anyway.

Updates golang/go#27945

Change-Id: Ie7b8b257408c790c5547f03747fddc77c6c4ad68
Reviewed-on: https://go-review.googlesource.com/c/text/+/169638
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
gopherbot pushed a commit to golang/text that referenced this issue Apr 24, 2019
@mpvl
cases: add bit for exception index
009a891 
The limit for this will be reached by Unicode 11.

Updates golang/go#27945

Change-Id: I5add868e60ba632de61b1ff3cb520453759b95c4
Reviewed-on: https://go-review.googlesource.com/c/text/+/169639
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
gopherbot pushed a commit to golang/text that referenced this issue Apr 24, 2019
@mpvl
cases: fix case mappings to allow for Gregorian uppercase
c942b20 
Prepare for Unicode 11, where in some cases
an undefined title case is lower case, instead of
upper case.

Updates golang/go#27945

Change-Id: I96d6921434e344271c80dbc4033b5952b8284178
Reviewed-on: https://go-review.googlesource.com/c/text/+/169640
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
@gopherbot
Copy link
Contributor

Change https://golang.org/cl/154442 mentions this issue: all: upgrade to Unicode 11

gopherbot pushed a commit that referenced this issue Apr 24, 2019
@mpvl
unicode: remove script test
8a20fde 
The script test requires a manual update on each new
Unicode release, which interupts the automated flow.
The test is removed in favor of one that fits within the
automated scripts.
See https://go-review.googlesource.com/c/text/+/169638.

Updates #27945

Change-Id: I9c9f1e2a77f7baef17040ea09eec28d7cf55b5e4
Reviewed-on: https://go-review.googlesource.com/c/go/+/169619
Reviewed-by: Russ Cox <rsc@golang.org>
gopherbot pushed a commit to golang/text that referenced this issue Apr 24, 2019
@mpvl
gen.go: update due to changes to core
b1379a7 
Updates documentation and other changes not related to
a Unicode upgrade to minimize the diff.

This CL also updates gen.go to no longer copy packages
into the vendor directory. Henceforth, go mod vendor
should be used to copy packages.

Updates golang/go#27945

Change-Id: I59da1c56e81bec67979ce13d89af3a753d4809aa
Reviewed-on: https://go-review.googlesource.com/c/text/+/169637
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
gopherbot pushed a commit that referenced this issue Apr 24, 2019
@mpvl
unicode: update table using new generator in x/text
e308d41 
The changes in Unicode 11 exposes a bug in maketables.go.
We update the Unicode 10 tables using a new generator
to minimize the changes upgrading to Unicode 11.

This change switches over the generation from core to that in
x/text. To properly update the tables one needs to run the generate
in x/text anyway, so this makes that official.

The RangeTable generator in x/text also generates slightly compacter
tables.

Updates #27945

See CL 154443

Change-Id: I6c59e082d5b8cd9e9332a32d8971061228581d66
Reviewed-on: https://go-review.googlesource.com/c/go/+/169617
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
gopherbot pushed a commit that referenced this issue Apr 25, 2019
@mpvl
all: update to Unicode 11
34fc6ce 
This does *not* update the vendored tables.
A commit updating these tables should follow soon,

Mostly generated running UNICODE_VERSION=11.0.0 in x/text.

Manually updated next.txt file.

Updates #27945.

Change-Id: I939a01e235aeca898ee9afc99a531e7ad8444e12
Reviewed-on: https://go-review.googlesource.com/c/go/+/154420
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
gopherbot pushed a commit to golang/text that referenced this issue Apr 25, 2019
@mpvl
all: upgrade to Unicode 11
342b2e1 
by running go generate at the top level.

IDNA still uses the old test format. The new
format tests more cases. The 11 format causes
test failure, but only for erroneous input that
result in errors. The package will in all such
cases still report an error, but the value result
may differ.

Updates golang/go#27945

Change-Id: Idb94e78411c2cb9f8e829b09d3d8de7807829076
Reviewed-on: https://go-review.googlesource.com/c/text/+/154442
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
@gopherbot
Copy link
Contributor

Change https://golang.org/cl/174057 mentions this issue: vendor: pull in text@v0.3.2

gopherbot pushed a commit that referenced this issue Apr 26, 2019
@mpvl
vendor: pull in text@v0.3.2
2ae5e7f 
This updates the packages from text and net to use
the Unicode 11 tables.

Updates #27945

Change-Id: I096e80283f37b596f2aa8bd3e5ffec6b7045469a
Reviewed-on: https://go-review.googlesource.com/c/go/+/174057
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@mpvl
Copy link
Contributor

mpvl commented Apr 26, 2019

This concludes the Unicode 11 upgrades.

Feel free to open a new one for Unicode 12. :)

@mpvl mpvl closed this as completed Apr 26, 2019
@odeke-em
Copy link
Member

Thank you and well done @mpvl!

@cuonglm cuonglm mentioned this issue May 26, 2019
Closed
@rsc rsc mentioned this issue Jun 27, 2019
Closed
@andybons andybons mentioned this issue Aug 5, 2019
Closed
@golang golang locked and limited conversation to collaborators Apr 25, 2020
@rsc rsc unassigned mpvl Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge help wanted
Projects
None yet
Milestone
Go1.13
Development

No branches or pull requests
6 participants
@fgm @agnivade @ALTree @odeke-em @mpvl @gopherbot