Nodes has no role

[michaelcourcy]

Description

Provide a brief description of your issue here. For example:

After a fresh install of origin 3.9 the node that should have a compute role has actually no roles. Master is ok.

[vagrant@workstation ~]$ oc get nodes
NAME                     STATUS    ROLES     AGE       VERSION
master.lab.example.com   Ready     master    1h        v1.9.1+a0ce1bc657
node1.lab.example.com    Ready     <none>    1h        v1.9.1+a0ce1bc657
node2.lab.example.com    Ready     <none>    1h        v1.9.1+a0ce1bc657

Version

Please put the following version information in the code block
indicated below.

• Your ansible version per ansible --version : 2.5.2

If you're operating from a git clone:

• The output of git describe : openshift-ansible-3.9.27-1-43-g81cd65e

Steps To Reproduce

Install the cluster following advanced install

Expected Results

[vagrant@workstation ~]$ oc get nodes
NAME                     STATUS    ROLES     AGE       VERSION
master.lab.example.com   Ready     master    1h        v1.9.1+a0ce1bc657
node1.lab.example.com    Ready     compute    1h        v1.9.1+a0ce1bc657
node2.lab.example.com    Ready     compute    1h        v1.9.1+a0ce1bc657

Observed Results

Describe what is actually happening.
Roles of the node are instead of compute

[vagrant@workstation ~]$ oc get nodes
NAME                     STATUS    ROLES     AGE       VERSION
master.lab.example.com   Ready     master    1h        v1.9.1+a0ce1bc657
node1.lab.example.com    Ready     <none>    1h        v1.9.1+a0ce1bc657
node2.lab.example.com    Ready     <none>    1h        v1.9.1+a0ce1bc657

Additional Information

I suspect this to be kubernetes issue with nat interface but I'm really not sure :

kubernetes/kubernetes#58609
kubernetes/kubernetes#50010

A workaround is to add this label to the node

oc label node node1.lab.example.com node-role.kubernetes.io/compute=true

[vrutkovs]

openshift_manage_node role should have done that, what's in your inventory and what's the output of ansible-playbook -vv ..?

[michaelcourcy]

Thanks for your hint it helps me.

In my inventory I have 3 nodes 1 master and 2 nodes which are labeled infra.

# host group for masters
[masters]
master.lab.example.com openshift_ip=172.25.250.10

# host group for etcd
[etcd]
master.lab.example.com openshift_ip=172.25.250.10

# host group for nodes, includes region info
[nodes]
master.lab.example.com openshift_ip=172.25.250.10
node1.lab.example.com openshift_node_labels="{'region': 'infra', 'zone': 'east', 'type': 'router'}" openshift_ip=172.25.250.11
node2.lab.example.com openshift_node_labels="{'region': 'infra', 'zone': 'west'}" openshift_ip=172.25.250.12

And reading openshift_manage_node/set_default_node_role.yml

   - name: label non-master non-infra nodes compute
      oc_label:
        name: '{{ item }}'
        kind: node
        state: add
        labels:
          - key: node-role.kubernetes.io/compute
            value: 'true'
      with_items: "{{ non_master_non_infra_nodes_result.results.results.0['items'] | map(attribute='metadata') | map(attribute='name') | list }}"

I can see that only non infra and non master node will be automatically labelled compute.

An improvement would be to decide if no nodes at all were found then label the infra node also compute but I'm not sure that's smart : for instance if someone decide to add lately the compute node.

The workaround is to force the label in the inventory

node1.lab.example.com openshift_node_labels="{'region': 'infra', 'zone': 'east', 'type': 'router', 'node-role.kubernetes.io/compute': 'true'}" openshift_ip=172.25.250.11
node2.lab.example.com openshift_node_labels="{'region': 'infra', 'zone': 'west', 'node-role.kubernetes.io/compute': 'true'}" openshift_ip=172.25.250.12

[vrutkovs]

'region': 'infra' is deprecated, now node-role.kubernetes.io/infra: true is used instead. Its a bit weird that the code didn't mark all the other nodes as compute but it depends on the inventory settings

[michaelcourcy]

in the hosts.example we still find

[nodes]
ose3-master[1:3].test.example.com
ose3-infra[1:2].test.example.com openshift_node_labels="{'region': 'infra', 'zone': 'default'}"
ose3-node[1:2].test.example.com openshift_node_labels="{'region': 'primary', 'zone': 'default'}"

I guess you don't update it each time you release.

Anyway you would recommand this setting ?

node1.lab.example.com openshift_node_labels="{'zone': 'east', 'type': 'router', 'node-role.kubernetes.io/compute': 'infra'}" openshift_ip=172.25.250.11
node2.lab.example.com openshift_node_labels="{'zone': 'west', 'node-role.kubernetes.io/compute': 'infra'}" openshift_ip=172.25.250.12 

Is it a way to say that the node is both infra and compute ?

[vrutkovs]

I guess you don't update it each time you release.

Its considered deprecated, IIRC openshift-ansible would automatically mark these with node-role.kubernetes.io/infra: true.

Sorry, there was a typo in my previous comment (s/compute: infra/infra: true).

Is it a way to say that the node is both infra and compute ?

Its the other way around - by default any non-master non-infra node would be marked as compute, so in order to make the node both infra and compute it should have both labels.

[michaelcourcy]

ok thanks. So to finish my declaration became :

node1.lab.example.com openshift_node_labels="{'zone': 'east', 'node-role.kubernetes.io/compute': 'true', 'node-role.kubernetes.io/infra': 'true'}" openshift_ip=172.25.250.11
node2.lab.example.com openshift_node_labels="{'zone': 'west', 'node-role.kubernetes.io/compute': 'true', 'node-role.kubernetes.io/infra': 'true'}" openshift_ip=172.25.250.12