Can't create pods from yaml template

[josselin-c]

With apiserver HEAD, creating pod from this yml template fails:

id: rqueue_test_cluster
apiVersion: v1beta1
kind: ReplicationController
desiredState:
  replicas: 1
  replicaSelector:
    name: rqueue_test
    env: dev
    test-machine: a1
  podTemplate:
    desiredState:
      manifest:
        version: v1beta1
        id: rqueue_test
        containers:
          - name: test
            image: b8.dev.g8teway.com/test
            imagePullPolicy: PullAlways
            env:
              - name: TEST_SERVICE_TARGET
                value: 11.111.1.111:11111
              - name: TEST_SERVICE_LISTEN_PORT
                value: 1234
              - name: TEST_SERVICE_NAME
                value: test-a1.dev.g8teway.com

    labels:
      name: rqueue_test
      env: dev
      test-machine: a1

[joss@~/projects/kubernetes_cluster/kubernetes/rqueue-test]$ kubectl --server="https://b7.dev.g8teway.com:6443" create -f rqueue-test-a1.yml
F1205 11:37:09.060948   21864 create.go:59] error unmarshaling JSON: json: cannot unmarshal number into Go value of type string

[root@b7 ~]# kube-apiserver -version=true
Kubernetes v0.6.0-94-g39bf3d2950e394-dirty

[bgrant0607]

/cc @ghodss

[ghodss]

I believe if you wrap 1234 in quotes it will work, but this is definitely a
bug with the new yaml library. I will take a look at it tonight.

On Friday, December 5, 2014, bgrant0607 notifications@github.com wrote:

/cc @ghodss https://github.com/ghodss

—
Reply to this email directly or view it on GitHub
#2763 (comment)
.

Sam

[ghodss]

Unfortunately this is a real bug in the new way YAML is being decoded. If you recall from #2600, the way we handle YAML now is to convert it to JSON then unmarshal using JSON. The problem is that YAML is less specific than JSON so if you have a YAML value of 1234, go-yaml assumes it's an int and encodes it as such, then when it's converted to JSON it's kept as an int, then when it's attempted to be unmarshaled into a string, Go errors. I am working on a fix right now and will have something by Sunday. If this is urgent in the mean time, one workaround is to wrap the string value (in this case the env var value) in quotes, and the other is to rollback the YAML change (#2676). I am fine with a rollback if this is not an acceptable bug for the next 48 hours. There may just be some merge conflicts because of all the YAML tag changes but those should be easy to resolve. Let me know if there's anything else I can do in the mean time.

[ghodss]

Okay I have this fixed locally, but need to write a few more tests. I will have a PR ready by midday today.

[jasonkuhrt]

Thanks for working on this guys. Just ran into this myself.

[pencilcheck]

Still happening, boolean will also not work if it is not quoted.

[bgrant0607]

@pencilcheck Got example?

[pencilcheck]

An example:

apiVersion: v1
kind: Service
metadata:
  name: backend
spec:
  ports:
    -
      port: 8080
      targetPort: 8080
  selector:
    name: backend-pod
    version: "1"
    tier: middleware

The version has to be quoted in order for it to pass, however I don't need for port and targetPort.

[liggitt]

It's all about the underlying struct types. port/targetPort are of type "IntOrString", which can deserialize from strings or ints.

"selector" is a map of strings to strings, so the values have to be strings (and therefore quoted if all numeric, per yaml rules)

[ghodss]

In fact @pencilcheck is right, version: 1 should work fine without quotes because in YAML an integer without quotes should be considered a valid string if the unmarshaling target is of string type. I just opened #13702 which fixes the issue and should allow you to use version: 1 without the quotes.

[et304383]

This is still an issue and should be resolved by the Kubernetes team. I cannot specify environment variables with integer values. Integer values are perfectly valid YAML and JSON, so the issue is Kubernetes. I should not have to quote integer values in a YAML file.

What's worse is the inconsistency. This seems to work for port values, replicas, etc, but still not in environment variables.

This should be allowed as it is perfectly valid YAML AND JSON (when converted):

        env:
        - name: MONGO_PORT
          value: 27017

Yet, apply complains:

unable to decode "deployment_example.yaml": [pos 413]: json: expect char '"' but got char '2'

[rbuck]

A basic bug, four years in the running, still not fixed in 1.6+. Any word on when this will be fixed?

[liggitt]

it is not likely to be fixed. you must specify your types correctly in the yaml. the client is converting to json and sending to the API without knowledge of the struct types, and sending an int value for a string field is incorrect.

[rbuck]

But the format is legal YAML, where numerics are entirely legal. It's likely the JSON conversion is flawed.

[liggitt]

It is ambiguous whether the value is a string or a number.

[et304383]

How is it ambiguous when most processors handle this?

Why is it inconsistent across other fields where an integer value is allowed?

[rbuck]

Yeah, I don't understand this. As a Golang developer myself, the YAML processors out of the box handle numerics flawlessly, and preserve them as numerics. And when I use a standard JSON processor to write a JSON file from the hash, it works flawlessly. I don't get why this is being debated. It's a bug.

[liggitt]

Because the client has to send JSON to the API, and is converting YAML to JSON generically without the schema that tells it whether the field is a string or an int. It would be incorrect to assume the destination field was a string. Quoting the YAML types the data

[et304383]

https://www.json2yaml.com/

It's amazing how easy this seems to handle integers going from yaml to json.

It seems to me that Google is just not interested in spending the effort to figure out something is a number.

[liggitt]

JSON to YAML is easy. YAML to JSON is ambiguous

[et304383]

So again, you figure it out. This converter seems to handle it flawlessly, even with exponent notation.

There's only so many scenarios needed to determine something is a number.

Hell, if you don't want to figure it out, why don't you just take whatever value we've given and pop it into a string behind the scenes? It's just being used to set an environment variable so at the end of the day it DOESN'T MATTER.

[liggitt]

This converter seems to handle it flawlessly, even with exponent notation.

Kubectl does exactly what this converter does, preserving numerics. If you paste the snippet in question into the convertor, you get this JSON:

  "env": [
    {
      "name": "foo",
      "value": 1
    }
  ],

That's exactly what kubectl is converting to and submitting to the API, and what the API is rejecting, because value must be a string

[et304383]

Then either:
A) API is broken. Fix it to accept non string values.
B) What's sending to the API is broken and should simply quote all value values in the env array.

[manikantag]

I'm able to use numbers without quoting by prefixing values with !!integer. But !!int is not working though. (Using Kubernetes 1.8)

Ex:

apiVersion: v1
kind: ConfigMap
metadata:
 name: identity-config
data:
  TC_HOST: 192.168.1.10
  TC_PORT: !!integer 8080