cannot create node via kubectl

[caesarxuchao]

I have a VM instance on GCE. I try to create a node based on the following json file:

{
  "kind": "Node",
  "apiVersion": "v1beta3",
  "metadata": {
    "name": "10.240.79.157",
    "labels": {
      "name": "my-first-k8s-node"
    }
  }
}

10.240.79.157 is the internal IP of the VM instance, and I have set up firewall rules to allow http/https traffic.

kubectl get nodes shows the node is created but it's in the status "unknown" and then it is deleted. Did I miss something in my json file? Thanks.
@yujuhong @dchen1107

[yujuhong]

Was kubelet running on your node?

[caesarxuchao]

No. So it requires kubelet running? Then it's different from the [doc]: "If the node is valid, i.e. all necessary services are running ..... Note that Kubernetes will keep invalid node unless explicitly deleted by client, and it will keep checking to see if it becomes valid."

[yujuhong]

It requires you to provision the node yourself. All necessary services include kubelet and everything a k8s node needs.

[caesarxuchao]

Yes. But it says "Note that Kubernetes will keep invalid node", but my invalid node is deleted.

[yujuhong]

It should not have. The only place where node controller deletes pod is when it finds out that the node is not ready and the node is not in the list of nodes that the cloud provider reports.
https://github.com/GoogleCloudPlatform/kubernetes/blob/master/pkg/cloudprovider/nodecontroller/nodecontroller.go#L441

Were your VM still running when the node object got deleted? You can also check the controller manager log to see whether it deleted the node.

[caesarxuchao]

Yes, my VM is still running and I will check the log. Does status "unknown" gives any clue?

[yujuhong]

Does the node name from the output of gcloud compute instances list match the name of the node object you created?

[caesarxuchao]

I used the instances internal IP as the name. So I should use the name? I will let you know if that works. Thanks!

[caesarxuchao]

Still no luck. Here is the gce instance (not provisioned yet) I want to add into the kubernetes cluster:

$ gcloud compute instances list
NAME                   ZONE          MACHINE_TYPE  PREEMPTIBLE INTERNAL_IP    EXTERNAL_IP     STATUS
glusterfs-1            us-central1-b n1-standard-1             10.240.79.157  104.197.36.231  RUNNING

The json file:

 {
   "kind": "Node",
   "apiVersion": "v1beta3",
   "metadata": {
     "name": "glusterfs-1",
     "labels": {
       "name": "my-first-k8s-node"
     }
   }
 }

After its creation, the node status is "unknown" and is then deleted.

[yujuhong]

There is a bunch of custom stuff (setting up network, route, firewalls, etc) that you need to do to configure a node. If something goes wrong when checking the node, the node controller would go ahead and delete it. I am pretty sure that your node did not meet the requirements.

On the other hand, there is ongoing work to allow nodes to register themselves (#6087). I am not sure if we still want users to create a node object by themselves. +@roberthbailey, who is more qualified to answer the questions.

[roberthbailey]

The node registration PR (#6949) was merged today so if you are running from head and you launch a kubelet with the correct command line flags it should register itself with the master. Note that this doesn't guarantee a functioning cluster, because you still need to have the proper networking (routes, firewalls) configured and kube-proxy running and configured. The startup guides for various platforms help walk you through this (and in many cases automate it for you).

[kostko]

I am not sure if we still want users to create a node object by themselves.

What if you want to assign labels to nodes before they are ready?

[yujuhong]

What if you want to assign labels to nodes before they are ready?

@kostko, you are right that user may want to create nodes due to reasons like this. This is doable today with or without node self-registration. As long as your node is configured correctly, the node controller can see the node so that it wouldn't attempt to delete it.

[kostko]

As long as your node is configured correctly, the node controller can see the node so that it wouldn't attempt to delete it.

What are the conditions for the node controller to delete an existing node?

[roberthbailey]

What are the conditions for the node controller to delete an existing node?

This likely changed with the recent changes for nodes to self register. The new logic is that if the nodecontroller stops getting status updates for nodes, it will reach out to the cloud provider (if one exists) to verify that the node still exists. If not, then it will delete the node. The relevant code implementing this logic is here.

[kostko]

Ok, so it seems that if there is no cloud provider (nc.cloud is null), then the node is not removed. Good.

[yujuhong]

@caesarxuchao, @roberthbailey, what do we want from this issue? Better documentation for user creating nodes themselves?

[caesarxuchao]

@yujuhong I opened this issue because I can't create a node manually with 'kubectl', on which I still stuck...
Updating the documentation sounds good and you can close the issue at your discretion. Thank you all.

[0xmichalis]

/sig node
/sig docs

[warmchang]

👍

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten
/remove-lifecycle stale

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close