Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal for KubeEdge Component Config #1172

Merged
merged 1 commit into from Feb 7, 2020

Conversation

@kadisi
Copy link
Member

kadisi commented Oct 2, 2019

Signed-off-by: zhangjie iamkadisi@163.com

What type of PR is this?

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespaces from that line:

/kind api-change
/kind bug
/kind cleanup
/kind design
/kind documentation
/kind test
/kind failing-test
/kind feature

/kind design
What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:


@kadisi

This comment has been minimized.

Copy link
Member Author

kadisi commented Oct 2, 2019

@kadisi kadisi force-pushed the kadisi:proposal_change_config_format branch from 6a0ca4a to 0ff133c Oct 2, 2019
@sids-b sids-b self-requested a review Oct 4, 2019
@kadisi kadisi force-pushed the kadisi:proposal_change_config_format branch from 0ff133c to e2e9c6a Oct 11, 2019
docs/proposals/configuration.md Outdated Show resolved Hide resolved
docs/proposals/configuration.md Outdated Show resolved Hide resolved
docs/proposals/configuration.md Outdated Show resolved Hide resolved
@kadisi kadisi force-pushed the kadisi:proposal_change_config_format branch from bd9d200 to b8c0ff0 Oct 12, 2019
Copy link
Member

chendave left a comment

my instant feeling on the subject is that the proposal is going to use another type of file (instead of yaml) for the configuration file.

docs/proposals/configuration.md Outdated Show resolved Hide resolved
docs/proposals/configuration.md Outdated Show resolved Hide resolved
docs/proposals/configuration.md Outdated Show resolved Hide resolved
@kadisi kadisi force-pushed the kadisi:proposal_change_config_format branch from b8c0ff0 to 06d60a4 Oct 12, 2019
@kadisi kadisi force-pushed the kadisi:proposal_change_config_format branch 5 times, most recently from c9465fb to ce66e3a Oct 12, 2019
Copy link
Member

kevin-wangzefeng left a comment

A few highlevel comments:

  1. How to implement backward compatibility for existing config schema? Would like to see it covered by the proposal.
  2. How to offer forward compatibility, e.g. addition/deprecattion/movement of some fields, etc. A version field might be needed.
  3. The overall development plan and graduation criteria - What's in alpha, when to release alpha, and what about beta and GA?
@kubeedge-bot kubeedge-bot added size/XL and removed size/L labels Oct 20, 2019
@kadisi

This comment has been minimized.

Copy link
Member Author

kadisi commented Oct 20, 2019

@kadisi kadisi force-pushed the kadisi:proposal_change_config_format branch 2 times, most recently from 04b3739 to 04896d0 Oct 21, 2019
docs/proposals/configuration.md Outdated Show resolved Hide resolved
docs/proposals/configuration.md Outdated Show resolved Hide resolved
Copy link
Member

kevin-wangzefeng left a comment

Please checkout the inline comments

docs/proposals/configuration.md Outdated Show resolved Hide resolved
docs/proposals/configuration.md Outdated Show resolved Hide resolved
docs/proposals/configuration.md Show resolved Hide resolved
docs/proposals/configuration.md Outdated Show resolved Hide resolved
docs/proposals/configuration.md Outdated Show resolved Hide resolved
docs/proposals/configuration.md Outdated Show resolved Hide resolved
docs/proposals/configuration.md Outdated Show resolved Hide resolved
docs/proposals/configuration.md Outdated Show resolved Hide resolved
docs/proposals/configuration.md Outdated Show resolved Hide resolved
docs/proposals/configuration.md Outdated Show resolved Hide resolved
@kadisi kadisi force-pushed the kadisi:proposal_change_config_format branch 4 times, most recently from 2c5c0b2 to b0c6dd1 Nov 27, 2019
@kadisi

This comment has been minimized.

Copy link
Member Author

kadisi commented Dec 10, 2019

@kadisi

This comment has been minimized.

Copy link
Member Author

kadisi commented Jan 3, 2020

After discussing with @kevin-wangzefeng and @fisherxu , we decided to use the following configuration format.

**
Note:
1 comment represent the original configuration options
2 Top-level configuration includes modules and common configuration
**

for cloudcore:

kind: cloudcore
apiVersion: cloudcore.config.kubeedge.io/v1alpha1

kubeAPIConfig:
  master: "" # controller.kube.master
  contentType: "application/vnd.kubernetes.protobuf" # controller.kube.content_type
  qps: 5 # controller.kube.qps
  burst: 10 # controller.kube.burst
  kubeconfig: "/root/.kube/config"   # controller.kube.

#edgeSite:
#  enable: false # metamanager.edgesite  注释: 需要删除, 只存在 edgesite 中
#  nodeName: "" # controller.kube.node-name 注释: 需要删除, 考虑参数列表 或者 结构体成员中

modules:
  cloudhub:
    enable: true # 是否开启此模块 default true, if false ,no need check other config
    keepaliveInterval: 30  # cloudhub.keepalive-interval
    nodeLimit: 10  # cloudhub.node-limit
    tlsCAFile: /etc/kubeedge/ca/rootCA.crt # cloudhub.ca
    tlsCertFile: /etc/kubeedge/certs/edge.crt # cloudhub.cert
    tlsPrivateKeyFile: /etc/kubeedge/certs/edge.key # cloudhub.key
    writeTimeout: 30 # cloudhub.write-timeout
    quic:
      enable: true # cloudhub.protocol_quic
      address: 0.0.0.0 # cloudhub.address
      port: 10001 # cloudhub.quic_port
      maxIncomingStreams: 10000 # cloudhub.max_incomingstreams
    unixsocket:
      enable: true # cloudhub.enable_uds
      unixSocketAddress: unix:///var/lib/kubeedge/kubeedge.sock # cloudhub.uds_address
    websocket:
      enable: true # cloudhub.protocol_websocket
      address: 0.0.0.0 # cloudhub.address
      port: 10000 # cloudhub.port

  edgecontroller:
    enable: true # 是否开启此模块 default true, if false ,no need check other config
    nodeUpdateFrequency: 10 #controller.kube.node_update_frequency
    buffer:
      updatePodStatus: 1024 # controller.buffer.update-pod-status
      updateNodeStatus: 1024 # controller.buffer.update-node-status
      queryConfigmap: 1024 # controller.buffer.query-configmap
      querySecret: 1024 # controller.buffer.query-secret
      queryService: 1024 # controller.buffer.query-service
      queryEndpoints: 1024 # controller.buffer.query-endpoints
      podEvent: 1 # controller.buffer.pod-event
      configmapEvent: 1 # controller.buffer.configmap-event
      secretEvent: 1 # controller.buffer.secret-event
      serviceEvent: 1 # controller.buffer.service-event
      endpointsEvent: 1 # controller.buffer.endpoints-event
      queryPersistentvolume: 1024 # controller.buffer.query-persistentvolume
      queryPersistentvolumeclaim: 1024 # controller.buffer.query-persistentvolumeclaim
      queryVolumeattachment: 1024 # controller.buffer.query-volumeattachment
      queryNode: 1024 # controller.buffer.query-node
      updateNode: 1024 # controller.buffer.update-node
    context:
      sendModule: cloudhub # controller.context.send-module
      receiveModule: edgecontroller # controller.context.receive-module
      responseModule: cloudhub # controller.context.response-module
    load:
      updatePodStatusWorkers: 1 # controller.load.update-pod-status-workers
      updateNodeStatusWorkers: 1 # controller.load.update-node-status-workers
      queryConfigmapWorkers: 1 # controller.load.query-configmap-workers
      querySecretWorkers: 4 # controller.load.query-secret-workers
      queryServiceWorkers: 4 # controller.load.query-service-workers
      queryEndpointsWorkers: 4 # controller.load.query-endpoints-workers
      queryPersistentvolumeWorkers: 4 # controller.load.query-persistentvolume-workers
      queryPersistentvolumeclaimWorkers: 4 # controller.load.query-persistentvolumeclaim-workers
      queryVolumeattachmentWorkers: 4 # controller.load.query-volumeattachment-workers
      queryNodeWorkers: 4 # controller.load.query-node-workers
      updateNodeWorkers: 4 # controller.load.update-node-workers

  devicecontroller:
    enable: true # 是否开启此模块 default true, if false ,no need check other config
    context:
      receiveModule: devicecontroller # devicecontroller.context.receive-module
      responseModule: cloudhub # devicecontroller.context.response-module
      sendModule: cloudhub # devicecontroller.context.send-module
    buffer:
      updateDeviceStatus: 1024 # devicecontroller.buffer.update-device-status
      deviceEvent: 1 # devicecontroller.buffer.device-event
      deviceModelEvent: 1 # devicecontroller.buffer.device-model-event
    load:
      updateDeviceStatusWorkers: 1 # devicecontroller.load.update-device-status-workers



for edgecore:

kind: edgecore
apiVersion: edgecore.config.kubeedge.io/v1alpha1

database:
  driver: "sqlite3" # database.driver
  name: "default" # database.name
  source: "/var/lib/kubeedge/edgecore.db" # database.source # 直接覆盖, 不读取

#edgeSite:
#  enable: false # metamanager.edgesite  注释: 需要删除, 只存在 edgesite 中
#  nodeName: "" # controller.kube.node-name 注释: 需要删除, 考虑参数列表 或者 结构体成员中

modules:
  edged:
    nodeStatusUpdateFrequency: 10 # edged.node-status-update-frequency
    dockerAddress: unix:///var/run/docker.sock # edged.docker-address
    runtimeType: docker # edged.runtime-type
    nodeIP: "" # edged.node-ip
    clusterDNS: "" # edged.cluster-dns
    clusterDomain: "" # edged.cluster-domain
    edgedMemoryCapacityBytes: 7852396000 # edged.edged-memory-capacity-bytes
    remoteRuntimeEndpoint: unix:///var/run/dockershim.sock # edged.remote-runtime-endpoint
    remoteImageEndpoint: unix:///var/run/dockershim.sock # edged.remote-image-endpoint
    podsandboxImage: kubeedge/pause:3.1 # edged.podsandbox-image
    imagePullProgressDeadline: 60 # edged.image-pull-progress-deadline:
    runtimeRequestTimeout: 2 # edged.runtime-request-timeout
    hostnameOverride: edge-node # edged.hostname-override
    registerNodeNamespace: default # edged.register-node-namespace
    interfaceName: eth0 # edged.interface-name
    devicePluginEnabled: false # edged.device-plugin-enabled
    gpuPluginEnabled: false # edged.gpu-plugin-enabled
    imageGCHighThreshold: 80 # edged.image-gc-high-threshold
    imageGCLowThreshold: 40 # edged.image-gc-low-threshold
    maximumDeadContainersPerPod: 1 # edged.maximum-dead-containers-per-container
    cgroupDriver: cgroupfs # edged.cgroup-driver
  edgehub:
    heartbeat: 15 # edgehub.controller.heartbeat
    projectID: e632aba927ea4ac2b575ec1603d56f10 # edgehub.controller.project-id
    tlsCaFile: /etc/kubeedge/ca/rootCA.crt # edgehub.quic.cafile
    tlsCertFile: /etc/kubeedge/certs/edge.crt # edgehub.websocket.certfile
    tlsPrivateKeyFile: /etc/kubeedge/certs/edge.key # edgehub.websocket.keyfile
    quic:
      enable: false # edgehub.controller.protocol (websocket, quic)
      handshakeTimeout: 30 # edgehub.quic.handshake-timeout
      readDeadline: 15 # edgehub.quic.read-deadline
      server: 127.0.0.1:10001 # edgehub.quic.url
      writeDeadline: 15 # edgehub.quic.write-deadline
    webSocket:
      enable: true # edgehub.controller.protocol (websocket, quic)
      handshakeTimeout: 30 # edgehub.websocket.handshake-timeout
      readDeadline: 15 # edgehub.websocket.read-deadline
      server: 127.0.0.1:10000 # edgehub.websocket.url (need change wss://0.0.0.0:10000/e632aba927ea4ac2b575ec1603d5
6f10/edge-node/events formart)
      writeDeadline: 15 # edgehub.websocket.write-deadline
  eventbus:
    mqttQOS: 0 # mqtt.qos
    mqttRetain: false  # mqtt.retain
    mqttSessionQueueSize: 100  # mqtt.session-queue-size
    mqttServerInternal: tcp://127.0.0.1:1884  # mqtt.internal-server
    mqttServerExternal: tcp://127.0.0.1:1883  # mqtt.server
    mqttMode: 0  # mqtt.mode
  metamanager:
    contextSendGroup: hub # metamanager.context-send-group
    contextSendModule: websocket # metamanager.context-send-module
    podStatusSyncInterval: 60 # meta.sync.podstatus.interval

  edgemesh:
    lbStrategy: RoundRobin # mesh.loadbalance.strategy-name

for edgesite:

kind: edgesite
apiVersion: edgesite.config.kubeedge.io/v1alpha1

database:
  driver: "sqlite3" # database.driver
  name: "default" # database.name
  source: "/var/lib/kubeedge/edgesite.db" # database.source # 直接覆盖, 不读取

edgeSite:
  enable: false # metamanager.edgesite
  nodeName: "" # controller.kube.node-name

kubeAPIConfig:
  master: "" # controller.kube.master
  contentType: "application/vnd.kubernetes.protobuf" # controller.kube.content_type
  qps: 5 # controller.kube.qps
  burst: 10 # controller.kube.burst
  kubeconfig: "/root/.kube/config"   # controller.kube.kubeconfig

modules:
  eventbus:
    mqttQOS: 0 # mqtt.qos
    mqttRetain: false # mqtt.retain
    mqttSessionQueueSize: 100 # mqtt.session-queue-size
    mqttServerInternal: tcp://127.0.0.1:1884 # mqtt.internal-server
    mqttServerExternal: tcp://127.0.0.1:1883 # mqtt.server
    mqttMode: 0 # mqtt.mode

  edgecontroller:
    enable: true # 是否开启此模块 default true, if false ,no need check other config
    nodeUpdateFrequency: 10 #controller.kube.node_update_frequency
    buffer:
      updatePodStatus: 1024 # controller.buffer.update-pod-status
      updateNodeStatus: 1024 # controller.buffer.update-node-status
      queryConfigmap: 1024 # controller.buffer.query-configmap
      querySecret: 1024 # controller.buffer.query-secret
      queryService: 1024 # controller.buffer.query-service
      queryEndpoints: 1024 # controller.buffer.query-endpoints
      podEvent: 1 # controller.buffer.pod-event
      configmapEvent: 1 # controller.buffer.configmap-event
      secretEvent: 1 # controller.buffer.secret-event
      serviceEvent: 1 # controller.buffer.service-event
      endpointsEvent: 1 # controller.buffer.endpoints-event
      queryPersistentvolume: 1024 # controller.buffer.query-persistentvolume
      queryPersistentvolumeclaim: 1024 # controller.buffer.query-persistentvolumeclaim
      queryVolumeattachment: 1024 # controller.buffer.query-volumeattachment
      queryNode: 1024 # controller.buffer.query-node
      updateNode: 1024 # controller.buffer.update-node
    context:
      sendModule: metaManager # controller.context.send-module
      receiveModule: edgecontroller # controller.context.receive-module
      responseModule: metaManager # controller.context.response-module
    load:
      updatePodStatusWorkers: 1 # controller.load.update-pod-status-workers
      updateNodeStatusWorkers: 1 # controller.load.update-node-status-workers
      queryConfigmapWorkers: 1 # controller.load.query-configmap-workers
      querySecretWorkers: 4 # controller.load.query-secret-workers
      queryServiceWorkers: 4 # controller.load.query-service-workers
      queryEndpointsWorkers: 4 # controller.load.query-endpoints-workers
      queryPersistentvolumeWorkers: 4 # controller.load.query-persistentvolume-workers
      queryPersistentvolumeclaimWorkers: 4 # controller.load.query-persistentvolumeclaim-workers
      queryVolumeattachmentWorkers: 4 # controller.load.query-volumeattachment-workers
      queryNodeWorkers: 4 # controller.load.query-node-workers
      updateNodeWorkers: 4 # controller.load.update-node-workers

  edged:
    nodeStatusUpdateFrequency: 10 # edged.node-status-update-frequency
    dockerAddress: unix:///var/run/docker.sock # edged.docker-address
    runtimeType: docker # edged.runtime-type
    nodeIP: "" # edged.node-ip
    clusterDNS: "" # edged.cluster-dns
    clusterDomain: "" # edged.cluster-domain
    edgedMemoryCapacityBytes: 7852396000 # edged.edged-memory-capacity-bytes
    remoteRuntimeEndpoint: unix:///var/run/dockershim.sock # edged.remote-runtime-endpoint
    remoteImageEndpoint: unix:///var/run/dockershim.sock # edged.remote-image-endpoint
    podsandboxImage: kubeedge/pause:3.1 # edged.podsandbox-image
    imagePullProgressDeadline: 60 # edged.image-pull-progress-deadline
    runtimeRequestTimeout: 2 # edged.runtime-request-timeout
    hostnameOverride: edge-node # edged.hostname-override
    registerNodeNamespace: default # edged.register-node-namespace
    interfaceName: eth0 # edged.interface-name
    devicePluginEnabled: false # edged.device-plugin-enabled
    gpuPluginEnabled: false # edged.gpu-plugin-enabled
    imageGCHighThreshold: 80 # edged.image-gc-high-threshold
    imageGCLowThreshold: 40 # edged.image-gc-low-threshold
    maximumDeadContainersPerPod: 1 # edged.maximum-dead-containers-per-container
    cgroupDriver: cgroupfs # edged.cgroup-driver

  metamanager:
    contextSendGroup: edgecontroller # metamanager.context-send-group
    #edgesite: false # 需要删除
    contextSendModule: edgecontroller # metamanager.context-send-module
    podStatusSyncInterval: 60 # meta.sync.podstatus.interval

@kadisi kadisi force-pushed the kadisi:proposal_change_config_format branch from 497656c to 599e521 Jan 4, 2020
@kubeedge-bot kubeedge-bot added size/XXL and removed size/XL labels Jan 4, 2020
@kadisi kadisi force-pushed the kadisi:proposal_change_config_format branch 2 times, most recently from ef02a3b to 92e2237 Jan 20, 2020
@kadisi kadisi force-pushed the kadisi:proposal_change_config_format branch 2 times, most recently from 9281826 to da673ab Feb 3, 2020

creation-date: 2019-10-02

status: provisional

This comment has been minimized.

Copy link
@fisherxu
… api and support --defaultconfig, --minconfig subcommand

Signed-off-by: zhangjie <iamkadisi@163.com>
Co-Authored-By: Kevin Wang <wangzefeng@huawei.com>
Signed-off-by: zhangjie <iamkadisi@163.com>
@kadisi kadisi force-pushed the kadisi:proposal_change_config_format branch from da673ab to 77e9ff1 Feb 7, 2020
@fisherxu

This comment has been minimized.

Copy link
Member

fisherxu commented Feb 7, 2020

/lgtm

@kubeedge-bot kubeedge-bot added the lgtm label Feb 7, 2020
@kevin-wangzefeng

This comment has been minimized.

Copy link
Member

kevin-wangzefeng commented Feb 7, 2020

/approve

@kubeedge-bot

This comment has been minimized.

Copy link
Collaborator

kubeedge-bot commented Feb 7, 2020

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kevin-wangzefeng

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubeedge-bot kubeedge-bot merged commit feb1536 into kubeedge:master Feb 7, 2020
1 of 2 checks passed
1 of 2 checks passed
tide Not mergeable. Needs approved label.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@kadisi kadisi deleted the kadisi:proposal_change_config_format branch Feb 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

6 participants
You can’t perform that action at this time.