Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JsonSyntaxException while parsing response for listPersistentVolume, and listPVC #324

Closed
anuthamb opened this issue Jul 24, 2018 · 13 comments

Comments

@anuthamb
Copy link

Kubernetes client version: v1.10.4
Kubernetes server version: v1.10.4
kubernetes java client version: 2.0.0

Encountering JsonSyntaxException while parsing response of GET PersistentVolumes and PersistentVolumeClaims while trying to parse storage capacity -> "...."storage":"1Gi"},....." as a JSONObject.
The capacity is of type Map<String, Quantity> whereas "1Gi" is the value.

Looking forward for suggestions on how to fix this.

Exception in thread "main" com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 735
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:176)
	at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.read(TypeAdapterRuntimeTypeWrapper.java:40)
	at com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.read(MapTypeAdapterFactory.java:187)
	at com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.read(MapTypeAdapterFactory.java:145)
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
	at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.read(TypeAdapterRuntimeTypeWrapper.java:40)
	at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.read(CollectionTypeAdapterFactory.java:81)
	at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.read(CollectionTypeAdapterFactory.java:60)
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
	at com.google.gson.Gson.fromJson(Gson.java:803)
	at com.google.gson.Gson.fromJson(Gson.java:768)
	at com.google.gson.Gson.fromJson(Gson.java:717)
	at io.kubernetes.client.JSON.deserialize(JSON.java:110)
	at io.kubernetes.client.ApiClient.deserialize(ApiClient.java:668)
	at io.kubernetes.client.ApiClient.handleResponse(ApiClient.java:871)
	at io.kubernetes.client.ApiClient.execute(ApiClient.java:798)
	at io.kubernetes.client.apis.CoreV1Api.listNamespacedPersistentVolumeClaimWithHttpInfo(CoreV1Api.java:16537)
	at io.kubernetes.client.apis.CoreV1Api.listNamespacedPersistentVolumeClaim(CoreV1Api.java:16514)
	at com.vmware.systest.lib.stlib.helper.k8s.KubernetesPersistentVolumesClaimHelper.getVolumes(KubernetesPersistentVolumesClaimHelper.java:41)
	at com.vmware.systest.lib.stlib.helper.k8s.KubernetesPersistentVolumesClaimHelper.main(KubernetesPersistentVolumesClaimHelper.java:72)
Caused by: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 735
	at com.google.gson.stream.JsonReader.beginObject(JsonReader.java:374)
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:165)
	... 25 more

API response JSON:
{"kind":"PersistentVolumeList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/persistentvolumes","resourceVersion":"340886"},"items":[{"metadata":{"name":"pvc-a8bb7fbc-8ce2-11e8-a11a-005056acd71d","selfLink":"/api/v1/persistentvolumes/pvc-a8bb7fbc-8ce2-11e8-a11a-005056acd71d","uid":"aa0e4179-8ce2-11e8-a11a-005056acd71d","resourceVersion":"3157","creationTimestamp":"2018-07-21T12:36:17Z","annotations":{"kubernetes.io/createdby":"vsphere-volume-dynamic-provisioner","pv.kubernetes.io/bound-by-controller":"yes","pv.kubernetes.io/provisioned-by":"kubernetes.io/vsphere-volume"},"finalizers":["kubernetes.io/pv-protection"]},"spec":{"capacity":{"storage":"1Gi"},"vsphereVolume":{"volumePath":"[vsanDatastore] c928535b-25c3-b1fb-12d0-f8bc12068db0/kubernetes-dynamic-pvc-a8bb7fbc-8ce2-11e8-a11a-005056acd71d.vmdk","fsType":"ext4"},"accessModes":["ReadWriteOnce"],"claimRef":{"kind":"PersistentVolumeClaim","namespace":"default","name":"www-web-0","uid":"a8bb7fbc-8ce2-11e8-a11a-005056acd71d","apiVersion":"v1","resourceVersion":"3146"},"persistentVolumeReclaimPolicy":"Delete","storageClassName":"thin-disk"},"status":{"phase":"Bound"}},{"metadata":{"name":"pvc-c11986e8-8ce2-11e8-a11a-005056acd71d","selfLink":"/api/v1/persistentvolumes/pvc-c11986e8-8ce2-11e8-a11a-005056acd71d","uid":"c13b00ab-8ce2-11e8-a11a-005056acd71d","resourceVersion":"3234","creationTimestamp":"2018-07-21T12:36:56Z","annotations":{"kubernetes.io/createdby":"vsphere-volume-dynamic-provisioner","pv.kubernetes.io/bound-by-controller":"yes","pv.kubernetes.io/provisioned-by":"kubernetes.io/vsphere-volume"},"finalizers":["kubernetes.io/pv-protection"]},"spec":{"capacity":{"storage":"1Gi"},"vsphereVolume":{"volumePath":"[vsanDatastore] c928535b-25c3-b1fb-12d0-f8bc12068db0/kubernetes-dynamic-pvc-c11986e8-8ce2-11e8-a11a-005056acd71d.vmdk","fsType":"ext4"},"accessModes":["ReadWriteOnce"],"claimRef":{"kind":"PersistentVolumeClaim","namespace":"default","name":"www-web-1","uid":"c11986e8-8ce2-11e8-a11a-005056acd71d","apiVersion":"v1","resourceVersion":"3220"},"persistentVolumeReclaimPolicy":"Delete","storageClassName":"thin-disk"},"status":{"phase":"Bound"}}]}

@brendandburns
Copy link
Contributor

I can't reproduce this at head, can you retry at head?

I'm performing a 3.0.0-beta1 release soon.

@brendandburns
Copy link
Contributor

I just pushed the 3.0.0-beta1 release, can you try that and see if it fixes things?

@anuthamb
Copy link
Author

anuthamb commented Jul 30, 2018

Hi @brendandburns, I checked again with 3.0.0-beta1 and see the same issue. Do we have to regenerate the model package from Swagger codegen ?

@anuthamb
Copy link
Author

anuthamb commented Jul 31, 2018

Seeing error related to Storage capacity while attempting for StatefulSet app (with PVC) creation:

[Jul 31 2018 13:39:49.715 c.v.s.l.s.h.k.KubernetesStatefulSetHelper OkHttp https://10.172.96.30:6443/apis/apps/v1beta2/namespaces/default/statefulsets] ERROR -- ApiException::Response Body: {
	"kind": "Status",
	"apiVersion": "v1",
	"metadata": {},
	"status": "Failure",
	"message": "StatefulSet in version \"v1\" cannot be handled as a StatefulSet: v1.StatefulSet.Spec: v1.StatefulSetSpec.VolumeClaimTemplates: []v1.PersistentVolumeClaim: v1.PersistentVolumeClaim.Spec: v1.PersistentVolumeClaimSpec.Resources: v1.ResourceRequirements.Requests: unmarshalerDecoder: quantities must match the regular expression '^([+-]?[0-9.]+)([eEinumkKMGTP]*[-+]?[0-9]*)$', error found in #10 byte of ...|INARY_SI\"}}}}}]}}|..., bigger context ...|torage\":{\"number\":1073741824,\"format\":\"BINARY_SI\"}}}}}]}}|...",
	"reason": "BadRequest",
	"code": 400
}

@brendandburns
Copy link
Contributor

Are you sure you're using the latest version of the library? This looks like what would have happened before we introduced the Quantity type, but it's been in there for a while...

Do you have a cached dependency somewhere in your CLASSPATH?

Alternately, can you attach a gist with a reproduction of this and I will attempt to reproduce and debug?

@brendandburns
Copy link
Contributor

I'm going to close this as can not reproduce. Please re-open with code that reproduces it (a complete sample would be great) if it is still occuring.

@anuthamb
Copy link
Author

hello Brenden.. apologies for late reply.
I was able to reproduce the issue with a fresh K8s cluster setup and with master at head version for the java client. Earlier I had also cleared off the maven dependencies and also the Eclipse project build path.
Here is the gist to reproduce:
https://gist.github.com/anuthamb/432885d6fe63e6958cc0488fd3838107

@anuthamb
Copy link
Author

@brendandburns, Pls reopen the issue if you are able to repro. I do not have reopen permissions.

@anuthamb
Copy link
Author

@brendandburns , here is a easier repro:
The JSON generated by the java client contains storage capacity like:
capacity: { storage=Quantity{ number=5242880, format=BINARY_SI } }

kubectl create -f https://gist.github.com/anuthamb/432885d6fe63e6958cc0488fd3838107/raw/5d18a01a3d3c8050dda70e482315377a301c3eb7/pv_sample_with_quantity.json

error: error validating "https://gist.github.com/anuthamb/432885d6fe63e6958cc0488fd3838107/raw/5d18a01a3d3c8050dda70e482315377a301c3eb7/pv_sample_with_quantity.json": error validating data: ValidationError(PersistentVolume.spec.capacity.storage): invalid type for io.k8s.apimachinery.pkg.api.resource.Quantity: got "map", expected "string"; if you choose to ignore these errors, turn validation off with --validate=false

The JSON which works, is with storage specified as a String:
"storage": "5Gi"

Here is the Persistent Volume JSON which works fine:

kubectl create -f https://gist.github.com/anuthamb/432885d6fe63e6958cc0488fd3838107/raw/5d18a01a3d3c8050dda70e482315377a301c3eb7/pv_sample_storage_string.json

persistentvolume/task-pv-volume created

I tested the java client libraries after changing capacity in V1PersistentVolumeSpec from

  @SerializedName("capacity")
  private Map<String, Quantity> capacity = null;

to,

  @SerializedName("capacity")
  private Map<String, String> capacity = null;

@brendandburns, I am yet to figure out changing swagger.json and regenerate all the Models needing change. Could you please verify the change suggested ?

@brendandburns
Copy link
Contributor

I'll work on reproducing this. Quantity should be getting parsed via a GSON parser so replacing it with String isn't the right way to go, but something must be going wrong in the parser registration.

@brendandburns
Copy link
Contributor

How are you generating the JSON? Are you calling toString() or are you calling Gson.toJson(...)?

When I call Gson.toJson(...) I get the Quantity formatted as I expect.

@brendandburns
Copy link
Contributor

I tried to reproduce this following your steps, your code doesn't compile because there are missing base classes, but I can't reproduce the error.

Here's what I'm trying:

kubectl create -f pv_sample_storage_string.json
/*
Copyright 2017 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
    http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package io.kubernetes.client.examples;

import io.kubernetes.client.ApiClient;
import io.kubernetes.client.ApiException;
import io.kubernetes.client.Configuration;
import io.kubernetes.client.apis.CoreV1Api;
import io.kubernetes.client.models.V1PersistentVolume;
import io.kubernetes.client.models.V1PersistentVolumeList;
import io.kubernetes.client.util.Config;
import java.io.IOException;

/**
 * A simple example of how to use the Java API
 *
 * <p>Easiest way to run this: mvn exec:java
 * -Dexec.mainClass="io.kubernetes.client.examples.Example"
 *
 * <p>From inside $REPO_DIR/examples
 */
public class Example {
  public static void main(String[] args) throws IOException, ApiException {
    ApiClient client = Config.defaultClient();
    Configuration.setDefaultApiClient(client);

    CoreV1Api api = new CoreV1Api();
    V1PersistentVolumeList list =
        api.listPersistentVolume(null, null, null, null, null, null, null, null, null);
    for (V1PersistentVolume item : list.getItems()) {
      System.out.println(item.getMetadata().getName());
    }
  }
}

This works correctly and as expected with no parse errors.

Somehow, you aren't loading the contents of the 'custom' directory here:

https://github.com/kubernetes-client/java/tree/master/kubernetes/src/main/java/io/kubernetes/client/custom

Into your java classpath, specifically the annotation here registers the Quantity class with Gson.

Somehow that's not happening, you might try:

apiClient.getJSON().getGson().getAdapter(Quantity.class)

And see what the output is...

I'm going to closes this as cannot reproduce. Please re-open if you have a clean environment that I can compile and test against head.

@anuthamb
Copy link
Author

anuthamb commented Oct 1, 2018

@brendandburns thanks for adding the ParseExample. It helped narrow down the issue in my workspace. My classpath had gson v2.2.4 instead of the v2.6.2 used by kubernetes client. Fixing this maven dependency solved the Yaml parse errors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants