Getting exception from deleteNamespacedStatefulSet() call #86
Comments
|
I'll investigate this, I suspect that it is a problem with the Swagger/OpenAPI spec indicating the wrong code for Delete... But we'll need to investigate to be sure. |
|
@brendanburns thanks. |
|
Ok, I figured out the issue here... When you delete a StatefulSet (and probably a ReplicaSet too) it initially returns a This means that the Swagger should be But the Kubernetes Swagger here: https://raw.githubusercontent.com/kubernetes/kubernetes/master/api/openapi-spec/swagger.json Only has the Thus, the generated Java code barfs when it tries to parse a Unfortunately None of these are great options, and none of them will happen very quickly. So for now, I think the best you can do is catch this Exception and move on... Sorry! |
|
@brendandburns ok. I can handle it in my code. Thanks for debugging this issue. Should we keep this issue for tracking purpose or close it for now? |
|
Yes, let's definitely keep this open. |
|
`oneof` is not supported in OpeAPI v2. It is possible to fix generator when
we move to OpenAPI v3 (both our kube-openapi and swagger-codegen should
support it).
…On Oct 8, 2017 6:47 AM, "Brendan Burns" ***@***.***> wrote:
Yes, let's definitely keep this open.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#86 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABic4ONQvsUMsx4rLtWydW2eSPyywU6Rks5sqNJrgaJpZM4Pt_sW>
.
|
|
Bug also exists when trying to delete a namespaced Job. JSON output is here {
"kind": "Job",
"apiVersion": "batch/v1",
"metadata": {
"name": "pi-s44bw",
"generateName": "pi-",
"namespace": "default",
"selfLink": "/apis/batch/v1/namespaces/default/jobs/pi-s44bw",
"uid": "a48d258d-faea-11e7-bdc1-42010a84018c",
"resourceVersion": "11082999",
"creationTimestamp": "2018-01-16T18:25:34Z",
"deletionTimestamp": "2018-01-16T18:25:55Z",
"deletionGracePeriodSeconds": 0,
"labels": {
"controller-uid": "a48d258d-faea-11e7-bdc1-42010a84018c",
"job-name": "pi-s44bw"
},
"finalizers": [
"orphan"
]
},
"spec": {
"parallelism": 1,
"completions": 1,
"backoffLimit": 4,
"selector": {
"matchLabels": {
"controller-uid": "a48d258d-faea-11e7-bdc1-42010a84018c"
}
},
"template": {
"metadata": {
"creationTimestamp": null,
"labels": {
"controller-uid": "a48d258d-faea-11e7-bdc1-42010a84018c",
"job-name": "pi-s44bw"
}
},
"spec": {
"containers": [
{
"name": "perl",
"image": "perl",
"command": [
"perl",
"-Mbignum=bpi",
"-wle",
"print bpi(2000)"
],
"resources": {},
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"imagePullPolicy": "Always"
}
],
"restartPolicy": "Never",
"terminationGracePeriodSeconds": 30,
"dnsPolicy": "ClusterFirst",
"securityContext": {},
"schedulerName": "default-scheduler"
}
}
},
"status": {
"conditions": [
{
"type": "Complete",
"status": "True",
"lastProbeTime": "2018-01-16T18:25:41Z",
"lastTransitionTime": "2018-01-16T18:25:41Z"
}
],
"startTime": "2018-01-16T18:25:34Z",
"completionTime": "2018-01-16T18:25:41Z",
"succeeded": 1
}
}However API says status is a string... com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected a string but was BEGIN_OBJECT at line 1 column 1152 path $.status
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:220)
at com.google.gson.Gson.fromJson(Gson.java:879)
at com.google.gson.Gson.fromJson(Gson.java:844)
at com.google.gson.Gson.fromJson(Gson.java:793)
at io.kubernetes.client.JSON.deserialize(JSON.java:106)
at io.kubernetes.client.ApiClient.deserialize(ApiClient.java:668)
at io.kubernetes.client.ApiClient.handleResponse(ApiClient.java:871)
at io.kubernetes.client.ApiClient.execute(ApiClient.java:798)
at io.kubernetes.client.apis.BatchV1Api.deleteNamespacedJobWithHttpInfo(BatchV1Api.java:511)
at io.kubernetes.client.apis.BatchV1Api.deleteNamespacedJob(BatchV1Api.java:491)
at com.oscaro.kubernetes.ApiHelper._clinit__closure5$_closure7(ApiHelper.groovy:80)
at com.oscaro.kubernetes.ApiHelper._clinit__closure5(ApiHelper.groovy:73)
at com.oscaro.kubernetes.JobExecutorTest.it should return the expected output from the job "#ns/#prefix-*"(JobExecutorTest.groovy:53)
Caused by: java.lang.IllegalStateException: Expected a string but was BEGIN_OBJECT at line 1 column 1152 path $.status
at com.google.gson.stream.JsonReader.nextString(JsonReader.java:831)
at com.google.gson.internal.bind.TypeAdapters$16.read(TypeAdapters.java:422)
at com.google.gson.internal.bind.TypeAdapters$16.read(TypeAdapters.java:410)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:116)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:216)
... 12 more |
|
Yeah, this is may apply to all object deletes, we could try to wrap if we really wanted to... |
|
Since we're using the async API variants, there isn't a good way to wrap this in our code. So, depending on how far away the OpenAPI v3 fix is I'd be grateful if this was wrapped -- at least so that it doesn't gum up the logs. |
|
Reporting that I am also running into this issue. I guess it's pretty clear already, but here's my client debug log and stacktrace: --> POST https://192.168.1.129:8443/api/v1/namespaces?pretty=false HTTP/1.1 {"apiVersion":"v1","kind":"Namespace","metadata":{"labels":{"name":"7700e076-b66f-4715-a716-d67764ffc946"},"name":"7700e076-b66f-4715-a716-d67764ffc946"},"spec":{}} {"kind":"Namespace","apiVersion":"v1","metadata":{"name":"7700e076-b66f-4715-a716-d67764ffc946","selfLink":"/api/v1/namespaces/7700e076-b66f-4715-a716-d67764ffc946","uid":"cd231c15-485c-11e8-b607-00155d380109","resourceVersion":"101976","creationTimestamp":"2018-04-25T07:46:45Z","labels":{"name":"7700e076-b66f-4715-a716-d67764ffc946"}},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}} <-- END HTTP (404-byte body) {"apiVersion":"v1","kind":"DeleteOptions","propagationPolicy":"Background"} {"kind":"Namespace","apiVersion":"v1","metadata":{"name":"7700e076-b66f-4715-a716-d67764ffc946","selfLink":"/api/v1/namespaces/7700e076-b66f-4715-a716-d67764ffc946","uid":"cd231c15-485c-11e8-b607-00155d380109","resourceVersion":"101981","creationTimestamp":"2018-04-25T07:46:45Z","deletionTimestamp":"2018-04-25T07:46:46Z","labels":{"name":"7700e076-b66f-4715-a716-d67764ffc946"}},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Terminating"}} <-- END HTTP (452-byte body) |
|
So is this something that the Kubernetes server team will fix when improving Swagger documentation generation, or something you will (eventually) fix by using a different generator? Just some example deal-with-it code here for future readers. It's not pretty but meh: |
|
+1 |
|
This issue is getting dangerously close to its first birthday—have there been any developments that might lead to a fix? Or at least an "official" recommendation for a workaround. |
|
@patricklucas unfortunately, there is really very little that we can do here. Until OpenAPI supports multi-type return (or inheritance) the generated client is always going to only be able to return one Java type, while the actual Kubernetes API server will always return two different types, resulting in this exception. The only 'workaround' is to catch the exception and move on. It's difficult to turn that into a utility method, since it could be any delete call (we'd have to code-generate the exception-catching code) I'm open to suggestions to improve this behavior, but I have a hard time figuring out the right way to address the issue, without fixing the root cause, which is currently blocked on moving to OpenAPI 3 (kubernetes/kubernetes#51163) and having a codegen that supports OpenAPI 3 (swagger-api/swagger-codegen#6598) If you have suggestions for temporary workarounds until then, please feel free to add them to the issue, I'm happy to implement them. |
k8s-ci-robot
added
the
lifecycle/frozen
|
@brendanburns can you add CRUD operations to the Examples, please? It can spare a lot of time and frustration 😃. Thanks. |
|
A workaround for this issue will be using GenericKubernetesApi. plz refer to the example by #902 |
|
@brendanburns is this also related to the same issue? Yaml.addModelMap("v1", "PersistentVolumeClaim", V1PersistentVolumeClaim.class);
V1PersistentVolumeClaim v1ClaimYaml = (V1PersistentVolumeClaim)Yaml.load(reader);
log.debug("toString-v1ClaimYaml[{}]", v1ClaimYaml.toString());
V1PersistentVolumeClaim v1Claim = api.replaceNamespacedPersistentVolumeClaim(claimName, K8Config.getNamespace(), v1ClaimYaml, null, null, null);apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: test-pvc
spec:
accessModes:
- ReadWriteOnce
storageClassName: default
resources:
requests:
storage: 1Gi
|
You can write like this, you can see the detailed error information |
|
Guys,
The exception is as below, |
|
I debugged into this using GCP Kubernetes v1.16.8-gke.15. It seems that the "status" of the "V1Pod" response in the "deleteNamespacedPod" case is parsed as "V1Status" (as implemented in "deleteNamespacedPodWithHttpInfo") instead of "V1PodStatus" as defined in "V1Pod" which obviously fails. Anyway it looks like the actual response "V1Pod" does not match the expected response "V1Status" in the client. IMO this is a severe error in the client which should at least be documented. Since this is known for over two year now... A possible workaround is (iff your Kubernetes cluster is ALWAYS returning V1Pod instead of V1Status) to reimplement some of the client's convenience methods with the "correct" type, e.g. like this: Please note that this implementation is not 100% equivalent to the original "deleteNamespacedPod" method as it is missing some of its validation. |
|
@markusheiden Kubernetes returns both V1Status and the specific object depending on if the delete finished in time. Delete is a semi-asynchronous operation and what is returned depends on the time the operation takes. You can't rely on it always returning a Unfortunately, the Kubernetes Swagger spec here only specifies one of the types ( |
|
@brendandburns Thanks for the details. A possible workaround is to reimplement some of the client's convenience methods with auto detecting the type, e.g. like this: Please note that this implementation is not 100% equivalent to the original "deleteNamespacedPod" method as it is missing some of its validation. |
|
Hi, is this issue resoved? or is there any way to avoid this error? |
Why: During testing of the new feature - cancelling a running or submitted data job execution I stumbled upon an error in the K8S api. A sucessfull call would throw a JsonSyntaxException. This is a known issue on the K8S API - kubernetes-client/java#86 . We already handle this in other methods. What: Mimicked the way we handle this exception in our codebase and modified the cancelRunningCronJob method to handle JsonSyntaxExceptions. Type: Bug fix. Testing: ci/cd and ran tests concerning Execution cancellation locally. Signed-off-by: mrMoZ1 <mzhivkov@vmware.com>
) Why: During testing of the new feature - cancelling a running or submitted data job execution I stumbled upon an error in the K8S api. A sucessfull call would throw a JsonSyntaxException. This is a known issue on the K8S API - kubernetes-client/java#86 . We already handle this in other methods. What: Mimicked the way we handle this exception in our codebase and modified the cancelRunningCronJob method to handle JsonSyntaxExceptions. Type: Bug fix. Testing: ci/cd and ran tests concerning Execution cancellation locally. Signed-off-by: mrMoZ1 <mzhivkov@vmware.com>
and others
Hi,
I was getting exception when I issued deleteNamespacedStatefulSet() call. The call did delete statefulset, but it throw the exception back to caller.
Exception:
java.lang.IllegalStateException: Expected a string but was BEGIN_OBJECT at line 1 column 2004 path $.status
Here is my code,
V1DeleteOptions deleteOptions = new V1DeleteOptions()
Boolean orphanDependents = true
apiInstance.deleteNamespacedStatefulSet(name, namespace, deleteOptions, null, 60, orphanDependents, null)
Thanks.
The text was updated successfully, but these errors were encountered: