Ansible operator: integer values of keys in objects in custom resources are strings in the ansible role

[gearoidibm]

Bug Report

What did you do?
Created an integer value of a key in an object in the custom resource and refer to it in the ansible task.

What did you expect to see?
The variable is an integer in the task so that it can be used as an integer in ansible modules.
For example as the number of replicas for a pod.

What did you see instead? Under which circumstances?

The variable is a sting in the task and causes ansible modules that require an integer to fail.
For example using the K8s module.

Filtering the value using | int also results in a string value.

Environment

• operator-sdk version: 0.9.0

• go version:

N/A

• Kubernetes version information:

Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-20T04:49:16Z", GoVersion:"go1.12.6", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:32:14Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}

• Kubernetes cluster kind:
  minikube
• Are you writing your operator in ansible, helm, or go?
  ansible
  Possible Solution

Additional context
top level integer variables are correctly available as integers in the task.

Example CRD, CR, and role

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: inttests.test.io
spec:
  group: test.io
  names:
    kind: IntTest
    listKind: IntTestList
    plural: inttests
    singular: inttest
  scope: Namespaced
  subresources:
    status: {}
  version: v1
  versions:
  - name: v1
    served: true
    storage: true

CR

apiVersion: test.io/v1
kind: IntTest
metadata:
  name: example-inttest
spec:
  # Add fields here
  top_level_int: 3
  my_object:
    object_int: 5

Role

---
# tasks file for inttest
- name:  debug top level int
  debug:
    msg:  "{{ top_level_int }}"

- name: debug object int
  debug:
    msg: "{{ my_object.object_int }}"

- name: debug object int cast to int
  debug:
    msg: "{{ my_object.object_int | int }}"

Output from ansible container log:
NOTE top_level_int is an int ,there's no ""s, where as my_object.object_int is a string, it's surrounded by ""s.

LAYBOOK: e92e8abb77f04d5891fe54caf9c0d270 *************************************
1 plays in /tmp/ansible-operator/runner/test.io/v1/IntTest/default/example-inttest/project/e92e8abb77f04d5891fe54caf9c0d270

PLAY [localhost] ***************************************************************

TASK [Gathering Facts] *********************************************************
ok: [localhost]
META: ran handlers

TASK [inttest : debug top level int] *******************************************
task path: /opt/ansible/roles/inttest/tasks/main.yml:3
ok: [localhost] => {
    "msg": 3
}

TASK [inttest : debug object int] **********************************************
task path: /opt/ansible/roles/inttest/tasks/main.yml:7
ok: [localhost] => {
    "msg": "5"
}

TASK [inttest : debug object int cast to int] **********************************
task path: /opt/ansible/roles/inttest/tasks/main.yml:11
ok: [localhost] => {
    "msg": "5"
}
META: ran handlers
META: ran handlers

PLAY RECAP *********************************************************************
localhost                  : ok=4    changed=0    unreachable=0    failed=0

[deweya]

I was able to reproduce with just plain Ansible (without the operator-sdk).

Notice though how I was still able to add to {{ my_object.object_int }} and treat it like an integer, even though it looks like a string in the debug output.

ansible --version:

ansible 2.8.2
  config file = /etc/ansible/ansible.cfg
  ...
  python version = 3.7.3 (default, May 11 2019, 00:38:04) [GCC 9.1.1 20190503 (Red Hat 9.1.1-1)]

playbook.yaml:

- name: Test play
  hosts: localhost
  tasks:
    - import_role:
        name: test-role

roles/test-role/defaults/main.yaml:

top_level_int: 3
my_object:
  object_int: 5

roles/test-role/tasks/main.yaml:

- name: debug top_level_int
  debug:
    msg: "{{ top_level_int }}"

- name: debug object_int
  debug:
    msg: "{{ my_object.object_int }}"

- name: debug object_int cast to int
  debug:
    msg: "{{ my_object.object_int | int }}"

- name: debug object_int cast to int plus 1
  debug:
    msg: "{{ my_object.object_int | int + 1 }}"

ansible-playbook playbook.yaml:

PLAY [Test play] ***************************************************************************************************************************************************************************************************************************

TASK [Gathering Facts] *********************************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [test-role : debug top_level_int] *****************************************************************************************************************************************************************************************************
ok: [localhost] => {
    "msg": 3
}

TASK [test-role : debug object_int] ********************************************************************************************************************************************************************************************************
ok: [localhost] => {
    "msg": "5"
}

TASK [test-role : debug object_int cast to int] ********************************************************************************************************************************************************************************************
ok: [localhost] => {
    "msg": "5"
}

TASK [test-role : debug object_int cast to int plus 1] *************************************************************************************************************************************************************************************
ok: [localhost] => {
    "msg": "6"
}

PLAY RECAP *********************************************************************************************************************************************************************************************************************************
localhost                  : ok=5    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

[deweya]

Found this issue here, which does a good job of explaining why those are strings in the debug output ansible/ansible#30366. Basically, it's a limitation of jinja2 in which the values can only be returned as strings (although it's interesting that the top_level_int actually looks like an integer)

[gearoidibm]

This is a problem of old versions of Jinja2 and Ansible. Later versions of Jinja2 along with Ansible configuration now allow all this to work as expected and have integers be returned. What's interesting here is it works correctly for the top level key but not for keys that part of objects.

[gearoidibm]

I have been able to patch this by updating the operator image Jinja2 to latest version ( 2.10.1) and by setting jinja2_native = True in its /etc/ansible/ansible.cfg

Can we get jinja2 updated in the base image as well as setting jinja_native2 = true by default ?

I think most custom resource types are going to have specifications that have integers with the intention that they are used as integers in the Ansible roles.

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[openshift-bot]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

[camilamacedo86]

Hi @gearoidibm,

The ansible and its deps were updated. Are you able to face the same issue with a project upgraded and/or created using the SDK 0.12?

[openshift-bot]

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

[openshift-ci-robot]

@openshift-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[StevenMercurio]

/remove-lifecycle rotten

[StevenMercurio]

/reopen

[openshift-ci-robot]

@StevenMercurio: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[camilamacedo86]

@StevenMercurio it was re-opened as you requested.

[StevenMercurio]

There seems to be no way I can figure out to stop an int that I assign to a var from being turned into a string when I use the variable to try and put the data where I need it. :(

[fabianvf]

Also raised in #2609 , and in ansible ansible/ansible#40185 , and discussed on slack https://kubernetes.slack.com/archives/CAW0GV7A5/p1582893529107700

Basically the issue here is that when you have an inline template, jinja templated fields always end up as strings (because they are wrapped in quotes and then yaml loaded). In order to get these fields to be loaded as non-strings, you need to load the template from a file, where the key: {{ value }} syntax is allowed (vs key: "{{ value }}" when inline).

Unfortunately I don't think there is a possible resolution on our end, though extra docs would make sense.

[miguelrdp]

I found a solution that seems to work and it doesn't require the ANSIBLE_JINJA2_NATIVE=True env variable

If instead of

initialDelaySeconds: "{{ _initialDelaySeconds }}"

you do

initialDelaySeconds: |
    {{ _initialDelaySeconds }}

you can omit the " in the jinja template and the value is not converted to string!

[jagibson]

Yeah I just hit this one hard until I saw this.
FYI I couldn't get @miguelrdp 's solution to work, but @gearoidibm 's did.

@fabianvf said:

though extra docs would make sense

Can we get this re-opened for updating the docs? Or does that need to be a separate issue?