Restore code to set IPs on master Machines. #714

russellb · 2019-07-31T19:02:45Z

Setting master Machine IPs is still required. We don't have a way to do that automatically, and the UI relies on this so the NodeRef gets set on the Machine and the Machine-Node relationship can be shown.

Related to issue #260

This reverts commit a991cca. Some of the removed code is still needed. In particular, we still need to set the IPs on master Machines, as we have no other mechanism to do that. Without it, the master Machines and Nodes will not get linked, which the UI depends on. Related to issue openshift-metal3#260

We should no longer set IPs on the worker Machine objects, as cluster-api-provider-baremetal should be doing that automatically. Since we're not setting IPs, there's no requirement for the script to wait for the worker to come up, either. This is still a useful verification, so move it to run_ci.sh, instead. Related to issue openshift-metal3#260

russellb · 2019-07-31T19:05:17Z

Was testing something else locally, so this is untested, need to let CI run

metal3ci · 2019-07-31T22:53:59Z

Build SUCCESS, see build http://10.8.144.11:8080/job/dev-tools/979/

e-minguez · 2019-08-02T10:19:18Z

it seems the link-machine-and-node.sh script doesn't add the IP to the machine object:

$ oc get nodes
NAME                                                 STATUS                     ROLES    AGE   VERSION
openshift-master-0.myenv.mydomain.example.com   Ready                      master   47h   v1.14.0+8321bc3f1
openshift-master-1.myenv.mydomain.example.com   Ready                      master   47h   v1.14.0+8321bc3f1
openshift-master-2.myenv.mydomain.example.com   Ready,SchedulingDisabled   master   47h   v1.14.0+8321bc3f1

$ oc get machines -A
NAMESPACE               NAME            INSTANCE   STATE   TYPE   REGION   ZONE   AGE
openshift-machine-api   dev5-master-0                                             47h
openshift-machine-api   dev5-master-1                                             47h
openshift-machine-api   dev5-master-2                                             47h

$ oc get baremetalhosts -A
NAMESPACE               NAME                 STATUS   PROVISIONING STATUS      CONSUMER        BMC                   HARDWARE PROFILE   ONLINE   ERROR
openshift-machine-api   openshift-master-0   OK       externally provisioned   dev5-master-0   ipmi://10.10.100.29                      true     
openshift-machine-api   openshift-master-1   OK       externally provisioned   dev5-master-1   ipmi://10.10.100.30                      true     
openshift-machine-api   openshift-master-2   OK       externally provisioned   dev5-master-2   ipmi://10.10.100.31                      true     

$ oc get machine -n openshift-machine-api   dev5-master-0 -o yaml
apiVersion: machine.openshift.io/v1beta1
kind: Machine
metadata:
  annotations:
    metal3.io/BareMetalHost: openshift-machine-api/openshift-master-0
  creationTimestamp: "2019-07-31T10:38:24Z"
  finalizers:
  - machine.machine.openshift.io
  generation: 1
  labels:
    machine.openshift.io/cluster-api-cluster: dev5
    machine.openshift.io/cluster-api-machine-role: master
    machine.openshift.io/cluster-api-machine-type: master
  name: dev5-master-0
  namespace: openshift-machine-api
  resourceVersion: "1803750"
  selfLink: /apis/machine.openshift.io/v1beta1/namespaces/openshift-machine-api/machines/dev5-master-0
  uid: 52aa1af7-b37f-11e9-be13-5254004eba53
spec:
  metadata:
    creationTimestamp: null
  providerSpec:
    value:
      hostSelector: {}
      image:
        checksum: ca70feb55f490f241b1e6d1d1fa86ce7
        url: http://172.22.0.1/images/rhcos-ootpa-latest.qcow2
      metadata:
        creationTimestamp: null
      userData:
        name: master-user-data
status:
  lastUpdated: "2019-08-02T10:08:39Z"
  nodeRef:
    kind: Node
    name: openshift-master-0.myenv.mydomain.example.com
    uid: openshift-master-0.myenv.mydomain.example.com



$ ./link-machine-and-node.sh dev5-master-0 openshift-master-0.myenv.mydomain.example.com
+ set -e
+ machine=dev5-master-0
+ node=openshift-master-0.myenv.mydomain.example.com
+ '[' -z dev5-master-0 -o -z openshift-master-0.myenv.mydomain.example.com ']'
++ echo openshift-master-0.myenv.mydomain.example.com
++ cut -f1 -d:
+ uid=openshift-master-0.myenv.mydomain.example.com
++ echo openshift-master-0.myenv.mydomain.example.com
++ cut -f2 -d:
+ node_name=openshift-master-0.myenv.mydomain.example.com
+ proxy_pid=15449
+ PROXY_API_PATH=http://localhost:8001/apis/machine.openshift.io/v1beta1/namespaces/openshift-machine-api/machines
+ wait_for_json oc_proxy http://localhost:8001/apis/machine.openshift.io/v1beta1/namespaces/openshift-machine-api/machines 10 -H 'Accept: application/json' -H 'Content-Type: application/json'
+ local name
+ local url
+ local curl_opts
+ local timeout
+ local start_time
+ local curr_time
+ local time_diff
+ name=oc_proxy
+ url=http://localhost:8001/apis/machine.openshift.io/v1beta1/namespaces/openshift-machine-api/machines
+ timeout=10
+ shift 3
+ curl_opts='-H Accept: application/json -H Content-Type: application/json'
+ echo -n 'Waiting for oc_proxy to respond'
Waiting for oc_proxy to respond+ oc --config ocp/auth/kubeconfig proxy
++ date +%s
+ start_time=1564740828
+ curl -g -X GET http://localhost:8001/apis/machine.openshift.io/v1beta1/namespaces/openshift-machine-api/machines '-H Accept: application/json -H Content-Type: application/json'
+ jq .
+ echo -n .
.++ date +%s
+ curr_time=1564740828
+ time_diff=0
+ [[ 0 -gt 10 ]]
+ sleep 5
Starting to serve on 127.0.0.1:8001
+ jq .
+ curl -g -X GET http://localhost:8001/apis/machine.openshift.io/v1beta1/namespaces/openshift-machine-api/machines '-H Accept: application/json -H Content-Type: application/json'
+ echo ' Success!'
 Success!
+ return 0
++ jq -c .status.addresses
++ oc --config ocp/auth/kubeconfig get node openshift-master-0.myenv.mydomain.example.com -o json
+ addresses='[{"address":"10.10.101.105","type":"InternalIP"},{"address":"openshift-master-0.myenv.mydomain.example.com","type":"Hostname"}]'
+ curl -X PATCH http://localhost:8001/apis/machine.openshift.io/v1beta1/namespaces/openshift-machine-api/machines/dev5-master-0/status -H 'Content-type: application/merge-patch+json' -d '{"status":{"addresses":[{"address":"10.10.101.105","type":"InternalIP"},{"address":"openshift-master-0.myenv.mydomain.example.com","type":"Hostname"}],"nodeRef":{"kind":"Node","name":"openshift-master-0.myenv.mydomain.example.com","uid":"openshift-master-0.myenv.mydomain.example.com"}}}'
{"apiVersion":"machine.openshift.io/v1beta1","kind":"Machine","metadata":{"annotations":{"metal3.io/BareMetalHost":"openshift-machine-api/openshift-master-0"},"creationTimestamp":"2019-07-31T10:38:24Z","finalizers":["machine.machine.openshift.io"],"generation":1,"labels":{"machine.openshift.io/cluster-api-cluster":"dev5","machine.openshift.io/cluster-api-machine-role":"master","machine.openshift.io/cluster-api-machine-type":"master"},"name":"dev5-master-0","namespace":"openshift-machine-api","resourceVersion":"1807142","selfLink":"/apis/machine.openshift.io/v1beta1/namespaces/openshift-machine-api/machines/dev5-master-0/status","uid":"52aa1af7-b37f-11e9-be13-5254004eba53"},"spec":{"metadata":{"creationTimestamp":null},"providerSpec":{"value":{"hostSelector":{},"image":{"checksum":"ca70feb55f490f241b1e6d1d1fa86ce7","url":"http://172.22.0.1/images/rhcos-ootpa-latest.qcow2"},"metadata":{"creationTimestamp":null},"userData":{"name":"master-user-data"}}}},"status":{"addresses":[{"address":"10.10.101.105","type":"InternalIP"},{"address":"openshift-master-0.myenv.mydomain.example.com","type":"Hostname"}],"lastUpdated":"2019-08-02T10:08:39Z","nodeRef":{"kind":"Node","name":"openshift-master-0.myenv.mydomain.example.com","uid":"openshift-master-0.myenv.mydomain.example.com"}}}
+ kill 15449



$ oc get machine -n openshift-machine-api   dev5-master-0 -o yaml
apiVersion: machine.openshift.io/v1beta1
kind: Machine
metadata:
  annotations:
    metal3.io/BareMetalHost: openshift-machine-api/openshift-master-0
  creationTimestamp: "2019-07-31T10:38:24Z"
  finalizers:
  - machine.machine.openshift.io
  generation: 1
  labels:
    machine.openshift.io/cluster-api-cluster: dev5
    machine.openshift.io/cluster-api-machine-role: master
    machine.openshift.io/cluster-api-machine-type: master
  name: dev5-master-0
  namespace: openshift-machine-api
  resourceVersion: "1807143"
  selfLink: /apis/machine.openshift.io/v1beta1/namespaces/openshift-machine-api/machines/dev5-master-0
  uid: 52aa1af7-b37f-11e9-be13-5254004eba53
spec:
  metadata:
    creationTimestamp: null
  providerSpec:
    value:
      hostSelector: {}
      image:
        checksum: ca70feb55f490f241b1e6d1d1fa86ce7
        url: http://172.22.0.1/images/rhcos-ootpa-latest.qcow2
      metadata:
        creationTimestamp: null
      userData:
        name: master-user-data
status:
  lastUpdated: "2019-08-02T10:13:53Z"
  nodeRef:
    kind: Node
    name: openshift-master-0.myenv.mydomain.example.com
    uid: openshift-master-0.myenv.mydomain.example.com

e-minguez · 2019-08-07T12:36:42Z

Also, it seems the machine.openshift.io/machine annotation is only added to a single node. I don't know if this is this script issue or a baremetaloperator issue.

$ oc get nodes -o yaml | grep -i machine.openshift
$
$ for i in 0 1 2; do ./link-machine-and-node.sh dev5-master-${i} openshift-master-${i}.example.com; done
...
$ oc get nodes -o yaml | grep -i machine.openshift
      machine.openshift.io/machine: openshift-machine-api/dev5-master-1

As a workaround, use oc annotate

$ for i in 0 1 2; do oc annotate --overwrite node/openshift-master-${i}.example.com machine.openshift.io/machine=openshift-machine-api/dev5-master-${i}; done
$ oc get nodes -o yaml | grep -i machine.openshift
      machine.openshift.io/machine: openshift-machine-api/dev5-master-0
      machine.openshift.io/machine: openshift-machine-api/dev5-master-1
      machine.openshift.io/machine: openshift-machine-api/dev5-master-2

dhellmann · 2019-08-07T19:35:30Z

It looks like the actuator is removing the address settings as soon as we add them. I'll update the script to patch the host with addresses, which should allow the data to flow through the system in the normal way.

dhellmann · 2019-08-07T20:42:01Z

I can't update @russellb 's PR so I opened #730 with the updated version of this script.

russellb added the CI check this PR with CI label Jul 31, 2019

russellb added 2 commits July 31, 2019 15:03

russellb force-pushed the master-machine-ips branch from ca62d61 to fa10b0b Compare July 31, 2019 19:04

markmc mentioned this pull request Aug 7, 2019

Machine/Node linkage required for machine healthcheck #725

Closed

dhellmann closed this Aug 7, 2019

dhellmann mentioned this pull request Aug 7, 2019

Restore script to set address information to link machines and nodes #730

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Restore code to set IPs on master Machines. #714

Restore code to set IPs on master Machines. #714

russellb commented Jul 31, 2019

russellb commented Jul 31, 2019

metal3ci commented Jul 31, 2019

e-minguez commented Aug 2, 2019

e-minguez commented Aug 7, 2019 •

edited

dhellmann commented Aug 7, 2019

dhellmann commented Aug 7, 2019

Restore code to set IPs on master Machines. #714

Restore code to set IPs on master Machines. #714

Conversation

russellb commented Jul 31, 2019

russellb commented Jul 31, 2019

metal3ci commented Jul 31, 2019

e-minguez commented Aug 2, 2019

e-minguez commented Aug 7, 2019 • edited

dhellmann commented Aug 7, 2019

dhellmann commented Aug 7, 2019

e-minguez commented Aug 7, 2019 •

edited