Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Manual cherry pick of #89833: preserve integers decoding raw JSON values #90020

Merged
merged 2 commits into from Apr 28, 2020
Merged

Manual cherry pick of #89833: preserve integers decoding raw JSON values #90020

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1ca9e6f
Preserve int/float distinction when decoding raw values
liggitt Apr 3, 2020
65319f4
kube-openapi bump to release-1.17
liggitt Apr 9, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 2 additions & 2 deletions go.mod
View file
Open in desktop
Expand Up @@ -152,7 +152,7 @@ require (
k8s.io/klog v1.0.0
k8s.io/kube-aggregator v0.0.0
k8s.io/kube-controller-manager v0.0.0
k8s.io/kube-openapi v0.0.0-20200316234421-82d701f24f9d
k8s.io/kube-openapi v0.0.0-20200410145947-bcb3869e6f29 // release-1.17
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does go keep these comments now or even adds them autmatically?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added them, go preserves them

k8s.io/kube-proxy v0.0.0
k8s.io/kube-scheduler v0.0.0
k8s.io/kubectl v0.0.0
Expand Down Expand Up @@ -540,7 +540,7 @@ replace (
k8s.io/klog => k8s.io/klog v1.0.0
k8s.io/kube-aggregator => ./staging/src/k8s.io/kube-aggregator
k8s.io/kube-controller-manager => ./staging/src/k8s.io/kube-controller-manager
k8s.io/kube-openapi => k8s.io/kube-openapi v0.0.0-20200316234421-82d701f24f9d
k8s.io/kube-openapi => k8s.io/kube-openapi v0.0.0-20200410145947-bcb3869e6f29 // release-1.17
k8s.io/kube-proxy => ./staging/src/k8s.io/kube-proxy
k8s.io/kube-scheduler => ./staging/src/k8s.io/kube-scheduler
k8s.io/kubectl => ./staging/src/k8s.io/kubectl
Expand Down
4 changes: 2 additions & 2 deletions go.sum
View file
Open in desktop
Expand Up @@ -589,8 +589,8 @@ k8s.io/heapster v1.2.0-beta.1 h1:lUsE/AHOMHpi3MLlBEkaU8Esxm5QhdyCrv1o7ot0s84=
k8s.io/heapster v1.2.0-beta.1/go.mod h1:h1uhptVXMwC8xtZBYsPXKVi8fpdlYkTs6k949KozGrM=
k8s.io/klog v1.0.0 h1:Pt+yjF5aB1xDSVbau4VsWe+dQNzA0qv1LlXdC2dF6Q8=
k8s.io/klog v1.0.0/go.mod h1:4Bi6QPql/J/LkTDqv7R/cd3hPo4k2DG6Ptcz060Ez5I=
k8s.io/kube-openapi v0.0.0-20200316234421-82d701f24f9d h1:jocF7XFucw2pEiv2wS7wk2FRFCjDFGV1oa4TMs0SAT0=
k8s.io/kube-openapi v0.0.0-20200316234421-82d701f24f9d/go.mod h1:F+5wygcW0wmRTnM3cOgIqGivxkwSWIWT5YdsDbeAOaU=
k8s.io/kube-openapi v0.0.0-20200410145947-bcb3869e6f29 h1:NeQXVJ2XFSkRoPzRo8AId01ZER+j8oV4SZADT4iBOXQ=
k8s.io/kube-openapi v0.0.0-20200410145947-bcb3869e6f29/go.mod h1:F+5wygcW0wmRTnM3cOgIqGivxkwSWIWT5YdsDbeAOaU=
k8s.io/repo-infra v0.0.1-alpha.1 h1:2us1n30u3cOcoPsacNfCvCssS9B9Yldr1ZGOdK0728U=
k8s.io/repo-infra v0.0.1-alpha.1/go.mod h1:wO1t9WaB99V80ljbeENTnayuEEwNZt7gECYh/CEyOJ8=
k8s.io/system-validators v1.0.4 h1:sW57tJ/ciqOVbbTLN+ZNy64MJMNqUuiwrirQv8IR2Kw=
Expand Down
2 changes: 1 addition & 1 deletion staging/src/k8s.io/api/go.sum
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion staging/src/k8s.io/apiextensions-apiserver/go.mod
View file
Open in desktop
Expand Up @@ -28,7 +28,7 @@ require (
k8s.io/code-generator v0.0.0
k8s.io/component-base v0.0.0
k8s.io/klog v1.0.0
k8s.io/kube-openapi v0.0.0-20200316234421-82d701f24f9d
k8s.io/kube-openapi v0.0.0-20200410145947-bcb3869e6f29 // release-1.17
k8s.io/utils v0.0.0-20191114184206-e782cd3c129f
sigs.k8s.io/yaml v1.1.0
)
Expand Down
4 changes: 2 additions & 2 deletions staging/src/k8s.io/apiextensions-apiserver/go.sum
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 4 additions & 2 deletions staging/src/k8s.io/apiextensions-apiserver/pkg/apiserver/schema/goopenapi_test.go
View file
Open in desktop
Expand Up @@ -38,9 +38,9 @@ func TestStructuralRoundtrip(t *testing.T) {
f.RandSource(rand.New(rand.NewSource(seed)))
f.Funcs(
func(s *JSON, c fuzz.Continue) {
switch c.Intn(6) {
switch c.Intn(7) {
case 0:
s.Object = float64(42.0)
s.Object = float64(42.2)
case 1:
s.Object = map[string]interface{}{"foo": "bar"}
case 2:
Expand All @@ -51,6 +51,8 @@ func TestStructuralRoundtrip(t *testing.T) {
s.Object = map[string]interface{}{}
case 5:
s.Object = nil
case 6:
s.Object = int64(42)
}
},
)
Expand Down
32 changes: 30 additions & 2 deletions staging/src/k8s.io/apiextensions-apiserver/test/integration/defaulting_test.go
View file
Open in desktop
Expand Up @@ -55,6 +55,10 @@ var defaultingFixture = &apiextensionsv1.CustomResourceDefinition{
Served: true,
Subresources: &apiextensionsv1.CustomResourceSubresources{
Status: &apiextensionsv1.CustomResourceSubresourceStatus{},
Scale: &apiextensionsv1.CustomResourceSubresourceScale{
SpecReplicasPath: ".spec.replicas",
StatusReplicasPath: ".status.replicas",
},
},
},
{
Expand All @@ -63,6 +67,10 @@ var defaultingFixture = &apiextensionsv1.CustomResourceDefinition{
Served: false,
Subresources: &apiextensionsv1.CustomResourceSubresources{
Status: &apiextensionsv1.CustomResourceSubresourceStatus{},
Scale: &apiextensionsv1.CustomResourceSubresourceScale{
SpecReplicasPath: ".spec.replicas",
StatusReplicasPath: ".status.replicas",
},
},
},
},
Expand Down Expand Up @@ -96,6 +104,11 @@ properties:
default: "v1beta1"
v1beta2:
type: string
replicas:
default: 1
format: int32
minimum: 0
type: integer
status:
type: object
properties:
Expand All @@ -112,6 +125,11 @@ properties:
default: "v1beta1"
v1beta2:
type: string
replicas:
default: 0
format: int32
minimum: 0
type: integer
`

const defaultingFooV1beta2Schema = `
Expand All @@ -133,6 +151,11 @@ properties:
v1beta2:
type: string
default: "v1beta2"
replicas:
default: 1
format: int32
minimum: 0
type: integer
status:
type: object
properties:
Expand All @@ -149,6 +172,11 @@ properties:
v1beta2:
type: string
default: "v1beta2"
replicas:
default: 0
format: int32
minimum: 0
type: integer
`

const defaultingFooInstance = `
Expand Down Expand Up @@ -278,15 +306,15 @@ func testDefaulting(t *testing.T, watchCache bool) {
// spec.a and spec.b are defaulted in both versions
// spec.v1beta1 is defaulted when reading the incoming request
// spec.v1beta2 is defaulted when reading the storage response
mustExist(foo.Object, [][]string{{"spec", "a"}, {"spec", "b"}, {"spec", "v1beta1"}, {"spec", "v1beta2"}})
mustExist(foo.Object, [][]string{{"spec", "a"}, {"spec", "b"}, {"spec", "v1beta1"}, {"spec", "v1beta2"}, {"spec", "replicas"}})
mustNotExist(foo.Object, [][]string{{"status"}})

t.Logf("Updating status and expecting 'a' and 'b' to show up.")
unstructured.SetNestedField(foo.Object, map[string]interface{}{}, "status")
if foo, err = fooClient.UpdateStatus(foo, metav1.UpdateOptions{}); err != nil {
t.Fatal(err)
}
mustExist(foo.Object, [][]string{{"spec", "a"}, {"spec", "b"}, {"status", "a"}, {"status", "b"}})
mustExist(foo.Object, [][]string{{"spec", "a"}, {"spec", "b"}, {"status", "a"}, {"status", "b"}, {"status", "replicas"}})

t.Logf("Add 'c' default to the storage version and wait until GET sees it in both status and spec")
addDefault("v1beta2", "c", "C")
Expand Down
2 changes: 1 addition & 1 deletion staging/src/k8s.io/apimachinery/go.mod
View file
Open in desktop
Expand Up @@ -32,7 +32,7 @@ require (
gopkg.in/inf.v0 v0.9.1
gopkg.in/yaml.v2 v2.2.8
k8s.io/klog v1.0.0
k8s.io/kube-openapi v0.0.0-20200316234421-82d701f24f9d
k8s.io/kube-openapi v0.0.0-20200410145947-bcb3869e6f29 // release-1.17
sigs.k8s.io/yaml v1.1.0
)

Expand Down
4 changes: 2 additions & 2 deletions staging/src/k8s.io/apimachinery/go.sum
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

25 changes: 25 additions & 0 deletions staging/src/k8s.io/apimachinery/pkg/util/json/json.go
View file
Open in desktop
Expand Up @@ -66,11 +66,36 @@ func Unmarshal(data []byte, v interface{}) error {
// If the decode succeeds, post-process the map to convert json.Number objects to int64 or float64
return convertSliceNumbers(*v, 0)

case *interface{}:
// Build a decoder from the given data
decoder := json.NewDecoder(bytes.NewBuffer(data))
// Preserve numbers, rather than casting to float64 automatically
decoder.UseNumber()
// Run the decode
if err := decoder.Decode(v); err != nil {
return err
}
// If the decode succeeds, post-process the map to convert json.Number objects to int64 or float64
return convertInterfaceNumbers(v, 0)

default:
return json.Unmarshal(data, v)
}
}

func convertInterfaceNumbers(v *interface{}, depth int) error {
var err error
switch v2 := (*v).(type) {
case json.Number:
*v, err = convertNumber(v2)
case map[string]interface{}:
err = convertMapNumbers(v2, depth+1)
case []interface{}:
err = convertSliceNumbers(v2, depth+1)
}
return err
}

// convertMapNumbers traverses the map, converting any json.Number values to int64 or float64.
// values which are map[string]interface{} or []interface{} are recursively visited
func convertMapNumbers(m map[string]interface{}, depth int) error {
Expand Down