Skip to content

Commit

Permalink
Merge pull request #3118 from Icarus9913/fix/wk/coordinator-panic
Browse files Browse the repository at this point in the history 
fix coordinator upgrade panic with CRD property empty
  • Loading branch information
@ty-dc
ty-dc committed Jan 24, 2024
2 parents 421bfb0 + ea50ffb commit ca8606e
Show file tree
Hide file tree
Showing 7 changed files with 45 additions and 15 deletions.
7 changes: 7 additions & 0 deletions charts/spiderpool/crds/spiderpool.spidernet.io_spidercoordinators.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -39,18 +39,23 @@ spec:
description: CoordinationSpec defines the desired state of SpiderCoordinator.
properties:
detectGateway:
default: false
type: boolean
detectIPConflict:
default: false
type: boolean
hijackCIDR:
items:
type: string
type: array
hostRPFilter:
default: 0
type: integer
hostRuleTable:
default: 500
type: integer
mode:
default: auto
enum:
- auto
- underlay
Expand All @@ -74,8 +79,10 @@ spec:
podMACPrefix:
type: string
tunePodRoutes:
default: true
type: boolean
txQueueLen:
default: 0
type: integer
type: object
status:
Expand Down
7 changes: 7 additions & 0 deletions charts/spiderpool/crds/spiderpool.spidernet.io_spidermultusconfigs.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -52,18 +52,23 @@ spec:
description: CoordinationSpec defines the desired state of SpiderCoordinator.
properties:
detectGateway:
default: false
type: boolean
detectIPConflict:
default: false
type: boolean
hijackCIDR:
items:
type: string
type: array
hostRPFilter:
default: 0
type: integer
hostRuleTable:
default: 500
type: integer
mode:
default: auto
enum:
- auto
- underlay
Expand All @@ -87,8 +92,10 @@ spec:
podMACPrefix:
type: string
tunePodRoutes:
default: true
type: boolean
txQueueLen:
default: 0
type: integer
type: object
customCNI:
Expand Down
7 changes: 4 additions & 3 deletions cmd/spiderpool-init/cmd/root.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,13 @@ import (
"context"
"time"

"github.com/spidernet-io/spiderpool/pkg/constant"
spiderpoolv2beta1 "github.com/spidernet-io/spiderpool/pkg/k8s/apis/spiderpool.spidernet.io/v2beta1"
"github.com/spidernet-io/spiderpool/pkg/logutils"
"go.uber.org/zap"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/utils/pointer"

"github.com/spidernet-io/spiderpool/pkg/constant"
spiderpoolv2beta1 "github.com/spidernet-io/spiderpool/pkg/k8s/apis/spiderpool.spidernet.io/v2beta1"
"github.com/spidernet-io/spiderpool/pkg/logutils"
)

var logger *zap.Logger
Expand Down
24 changes: 12 additions & 12 deletions docs/reference/crd-spidercoordinator.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -44,18 +44,18 @@ status:

This is the Spidercoordinators spec for users to configure.

| Field | Description | Schema | Validation | Values | Default |
|--------------------|--------------------------------------------------------------|----------------------|------------|------------------------------|------------------------------|
| mode | The mode in which the coordinator. auto: automatically determine if it's overlay or underlay. underlay: coordinator creates veth devices to solve the problem that CNIs such as macvlan cannot communicate with clusterIP. overlay: fix the problem that CNIs such as Macvlan cannot access ClusterIP through the Calico network card attached to the pod,coordinate policy route between interfaces to ensure consistence data path of request and reply packets | string | require | auto,underlay,overlay | auto |
| podCIDRType | The ways to fetch the CIDR of the cluster. auto(default), This means that it will automatically switch podCIDRType to cluster or calico or cilium. based on cluster CNI. calico: auto fetch the subnet of the pod from the ip pools of calico, This only works if the cluster CNI is calico; cilium: Auto fetch the pod's subnet from cilium's configMap or ip pools. Supported IPAM modes: ["cluster-pool","kubernetes","multi-pool"]; cluster: auto fetch the subnet of the pod from the kubeadm-config configmap, This is useful if there is only a globally unique default pod's subnet; none: don't get the subnet of the pod, which is useful for some special cases. In this case,you can manually configure the hijackCIDR field | string | require | auto,cluster,calico,cilium,none | auto |
| tunePodRoutes | tune pod's route while the pod is attached to multiple NICs | bool | optional | true,false | true |
| podDefaultRouteNIC | The NIC where the pod's default route resides | string | optional | "",eth0,net1... | underlay: eth0,overlay: net1 |
| detectGateway | enable detect gateway while launching pod, If the gateway is unreachable, pod will be failed to created; Note: We use ARP probes to detect if the gateway is reachable, and some gateway routers may warn about this | boolean | optional | true,false | false |
| detectIPConflict | enable the pod's ip if is conflicting while launching pod. If an IP conflict of the pod is detected, pod will be failed to created | boolean | optional | true,false | false |
| podMACPrefix | fix the pod's mac address with this prefix + 4 bytes IP | string | optional | a invalid mac address prefix | "" |
| hostRPFilter | sysctls: rp_filter in host | int | required | 0,1,2;suggest to be 0 | 0 |
| hostRuleTable | The directly routing table of the host accessing the pod's underlay IP will be placed in this policy routing table | int | required | int | 500 |
| txQueueLen | The Transmit Queue Length (txqueuelen) is a TCP/IP stack network interface value that sets the number of packets allowed per kernel transmit queue of a network interface device | int | optional | >= 0, default to 0, it's mean to don't set it |
| Field | Description | Schema | Validation | Values | Default |
|--------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------|------------|-----------------------------------------------|------------------------------|
| mode | The mode in which the coordinator. auto: automatically determine if it's overlay or underlay. underlay: coordinator creates veth devices to solve the problem that CNIs such as macvlan cannot communicate with clusterIP. overlay: fix the problem that CNIs such as Macvlan cannot access ClusterIP through the Calico network card attached to the pod,coordinate policy route between interfaces to ensure consistence data path of request and reply packets | string | require | auto,underlay,overlay | auto |
| podCIDRType | The ways to fetch the CIDR of the cluster. auto(default), This means that it will automatically switch podCIDRType to cluster or calico or cilium. based on cluster CNI. calico: auto fetch the subnet of the pod from the ip pools of calico, This only works if the cluster CNI is calico; cilium: Auto fetch the pod's subnet from cilium's configMap or ip pools. Supported IPAM modes: ["cluster-pool","kubernetes","multi-pool"]; cluster: auto fetch the subnet of the pod from the kubeadm-config configmap, This is useful if there is only a globally unique default pod's subnet; none: don't get the subnet of the pod, which is useful for some special cases. In this case,you can manually configure the hijackCIDR field | string | require | auto,cluster,calico,cilium,none | auto |
| tunePodRoutes | tune pod's route while the pod is attached to multiple NICs | bool | optional | true,false | true |
| podDefaultRouteNIC | The NIC where the pod's default route resides | string | optional | "",eth0,net1... | underlay: eth0,overlay: net1 |
| detectGateway | enable detect gateway while launching pod, If the gateway is unreachable, pod will be failed to created; Note: We use ARP probes to detect if the gateway is reachable, and some gateway routers may warn about this | boolean | optional | true,false | false |
| detectIPConflict | enable the pod's ip if is conflicting while launching pod. If an IP conflict of the pod is detected, pod will be failed to created | boolean | optional | true,false | false |
| podMACPrefix | fix the pod's mac address with this prefix + 4 bytes IP | string | optional | a invalid mac address prefix | "" |
| hostRPFilter | sysctls: rp_filter in host | int | required | 0,1,2;suggest to be 0 | 0 |
| hostRuleTable | The directly routing table of the host accessing the pod's underlay IP will be placed in this policy routing table | int | required | int | 500 |
| txQueueLen | The Transmit Queue Length (txqueuelen) is a TCP/IP stack network interface value that sets the number of packets allowed per kernel transmit queue of a network interface device | int | optional | >= 0, default to 0, it's mean to don't set it |

### Status (subresource)

Expand Down
4 changes: 4 additions & 0 deletions docs/usage/install/upgrade-zh_CN.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,10 @@ kubectl patch sp ${auto-pool} --type merge --patch '{"metadata": {"labels": {"ip
~# ls | grep '\.yaml$' | xargs -I {} kubectl apply -f {}
```

### 低于 0.9.0 (不包含 0.9.0) 升级到最高版本的注意事项

由于在 0.9.0 的版本中，我们给 [SpiderCoordinator CRD](./../../reference/crd-spidercoordinator.md) 补充了 `txQueueLen` 字段，但由于执行升级时 Helm 不支持升级或删除 CRD，因此在升级前需要你手动更新一下 CRD。(建议越过 0.9.0 直接升级至 0.9.1 版本)

### 更多版本升级的注意事项

*TODO.*
4 changes: 4 additions & 0 deletions docs/usage/install/upgrade.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,10 @@ In versions higher than 0.5.0, the [SpiderMultusConfig](../spider-multus-config.
~# ls | grep '\.yaml$' | xargs -I {} kubectl apply -f {}
```

### Upgrading from a version below 0.9.0 (Excludes 0.9.0) to a higher version

Due to the addition of the `txQueueLen` field to the [SpiderCoordinator CRD](./../../reference/crd-spidercoordinator.md) in version 0.9.0, you need to manually update the CRD before upgrading as Helm does not support upgrading or deleting CRDs during the upgrade process.(We suggest skipping version 0.9.0 and upgrading directly to version 0.9.1)

### More notes on version upgrades

*TODO.*
7 changes: 7 additions & 0 deletions pkg/k8s/apis/spiderpool.spidernet.io/v2beta1/spidercoordinator_types.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import (
type CoordinatorSpec struct {
// +kubebuilder:validation:Enum=auto;underlay;overlay;disabled
// +kubebuilder:validation:Optional
// +kubebuilder:default=auto
Mode *string `json:"mode,omitempty"`

// CoordinatorSpec is used by SpiderCoordinator and SpiderMultusConfig
Expand All @@ -27,24 +28,30 @@ type CoordinatorSpec struct {
PodMACPrefix *string `json:"podMACPrefix,omitempty"`

// +kubebuilder:validation:Optional
// +kubebuilder:default=true
TunePodRoutes *bool `json:"tunePodRoutes,omitempty"`

// +kubebuilder:validation:Optional
PodDefaultRouteNIC *string `json:"podDefaultRouteNIC,omitempty"`

// +kubebuilder:validation:Optional
// +kubebuilder:default=500
HostRuleTable *int `json:"hostRuleTable,omitempty"`

// +kubebuilder:validation:Optional
// +kubebuilder:default=0
HostRPFilter *int `json:"hostRPFilter,omitempty"`

// +kubebuilder:validation:Optional
// +kubebuilder:default=0
TxQueueLen *int `json:"txQueueLen,omitempty"`

// +kubebuilder:validation:Optional
// +kubebuilder:default=false
DetectIPConflict *bool `json:"detectIPConflict,omitempty"`

// +kubebuilder:validation:Optional
// +kubebuilder:default=false
DetectGateway *bool `json:"detectGateway,omitempty"`
}

Expand Down

0 comments on commit ca8606e

Please sign in to comment.