Skip to content
This repository has been archived by the owner on Jun 29, 2022. It is now read-only.

Add Packet CCM to Packet Lokomotive clusters #548

Closed
iaguis opened this issue Jun 3, 2020 · 13 comments · Fixed by #1155
Closed

Add Packet CCM to Packet Lokomotive clusters #548

iaguis opened this issue Jun 3, 2020 · 13 comments · Fixed by #1155
Assignees
@invidian @knrt10
Labels
kind/roadmap Roadmap issues size/l Issues which likely require many work days (but not weeks)
Projects
Roadmap
Milestone
v0.6.0

Comments

@iaguis
Copy link
Contributor

iaguis commented Jun 3, 2020

We should add the Packet CCM to Lokomotive clusters deployed to Packet.

The CCM handles, for example, node changes and is gonna be needed to use the peer autodiscovery feature of MetalLB once it gets merged (we're currently using a patched version of MetalLB to have the feature).
@iaguis iaguis created this issue from a note in Roadmap (Queued) Jun 3, 2020
@iaguis iaguis added the kind/roadmap Roadmap issues label Jun 3, 2020
@invidian
Copy link
Member

invidian commented Jun 3, 2020

When implemented, it will also hit #478 😞

@iaguis
Copy link
Contributor Author

iaguis commented Jun 3, 2020

Keep in mind this comment about things needed to bootstrap the CCM, since it'll probably apply to the Packet CCM too.

@knrt10
Copy link
Member

knrt10 commented Sep 17, 2020

Looking into this

@iaguis iaguis mentioned this issue Sep 17, 2020
Open
@johananl
Copy link
Member

https://github.com/packethost/packet-ccm#bgp

The Packet CCM enables BGP for the project and enables it on all nodes as they come up.

@knrt10 FYI ^. We should verify that users can still decide not to enable BGP for a worker pool.

@knrt10 knrt10 added the proposed/next-sprint Issues proposed for next sprint label Sep 23, 2020
@knrt10 knrt10 self-assigned this Sep 23, 2020
@surajssd surajssd removed the proposed/next-sprint Issues proposed for next sprint label Sep 23, 2020
@knrt10 knrt10 added the proposed/next-sprint Issues proposed for next sprint label Oct 14, 2020
@iaguis iaguis added the size/l Issues which likely require many work days (but not weeks) label Oct 14, 2020
@johananl johananl mentioned this issue Oct 26, 2020
Closed
@knrt10
Copy link
Member

knrt10 commented Nov 2, 2020 •
edited
Loading

Current status:

I tried manually deploying ccm with current master. We had to deploy Lokomotive's MetalLb component for ccm to work. ccm watches and picks up the configMap config from metallb-system namespace. The problems that I faced are:

  • Once ccm deployment is deployed on cluster, you have to apply MetalLb component. If component is re-applied, ccm does not pick configMap correctly and throws error that configMap is not correct, which should not happen. You have to delete the pod manually to get it to work again.
failed to update and sync nodes: failed to get metallb config map: could not parse config: yaml: unmarshal errors:
  line 1: field peer-autodiscovery not found in type metallb.ConfigFile
  • BGP is automatically enabled for all nodes. We have to find a way for users it disable it.
  • Once we deploy Lokomotive's MetalLb component, you have to manually apply Elastic IP to worker nodes on Packet console.

These things should be fixed, once packet-ccm fix in their patch release

knrt10 added a commit that referenced this issue Nov 3, 2020
@knrt10
Add packet-ccm
9cb58e0 
- Add helm chart for packet-ccm.
- Add varibales api_token and project_id to bootkube.
- Run packet-ccm while bootstrapping.

closes: #548
Signed-off-by: knrt10 <kautilya@kinvolk.io>
@knrt10 knrt10 mentioned this issue Nov 3, 2020
Merged
knrt10 added a commit that referenced this issue Nov 3, 2020
@knrt10
Add packet-ccm
de92890 
- Add helm chart for packet-ccm.
- Add varibales api_token and project_id to bootkube.
- Run packet-ccm while bootstrapping.

closes: #548
Signed-off-by: knrt10 <kautilya@kinvolk.io>
knrt10 added a commit that referenced this issue Nov 3, 2020
@knrt10
Add packet-ccm
f08e7f4 
- Add helm chart for packet-ccm.
- Add varibales api_token and project_id to bootkube.
- Run packet-ccm while bootstrapping.

closes: #548
Signed-off-by: knrt10 <kautilya@kinvolk.io>
knrt10 added a commit that referenced this issue Nov 3, 2020
@knrt10
Add packet-ccm
0c0e277 
- Add helm chart for packet-ccm.
- Add varibales api_key and project_id to bootkube.
- Run packet-ccm while bootstrapping.

closes: #548
Signed-off-by: knrt10 <kautilya@kinvolk.io>
knrt10 added a commit that referenced this issue Nov 3, 2020
@knrt10
Add packet-ccm
0b2b305 
- Add helm chart for packet-ccm.
- Add varibales auth_token and project_id to bootkube.
- Run packet-ccm while bootstrapping.

closes: #548
Signed-off-by: knrt10 <kautilya@kinvolk.io>
knrt10 added a commit that referenced this issue Nov 3, 2020
@knrt10
Add packet-ccm
4620b33 
- Add helm chart for packet-ccm.
- Add varibales auth_token and project_id to bootkube.
- Run packet-ccm while bootstrapping.

closes: #548
Signed-off-by: knrt10 <kautilya@kinvolk.io>
knrt10 added a commit that referenced this issue Nov 3, 2020
@knrt10
Add packet-ccm
06d7c4b 
- Add helm chart for packet-ccm.
- Add varibales auth_token and project_id to bootkube.
- Run packet-ccm while bootstrapping.

closes: #548
Signed-off-by: knrt10 <kautilya@kinvolk.io>
knrt10 added a commit that referenced this issue Nov 3, 2020
@knrt10
Add packet-ccm
3c02500 
- Add helm chart for packet-ccm.
- Add varibales auth_token and project_id to bootkube.
- Run packet-ccm while bootstrapping.

closes: #548
Signed-off-by: knrt10 <kautilya@kinvolk.io>
knrt10 added a commit that referenced this issue Nov 3, 2020
@knrt10
Add packet-ccm
050dd14 
- Add helm chart for packet-ccm.
- Add varibales auth_token and project_id to bootkube.
- Run packet-ccm while bootstrapping.

closes: #548
Signed-off-by: knrt10 <kautilya@kinvolk.io>
knrt10 added a commit that referenced this issue Nov 4, 2020
@knrt10
Add packet-ccm
9c27968 
- Add helm chart for packet-ccm.
- Run packet-ccm while bootstrapping.

closes: #548
Signed-off-by: knrt10 <kautilya@kinvolk.io>
@iaguis iaguis removed the proposed/next-sprint Issues proposed for next sprint label Nov 4, 2020
knrt10 added a commit that referenced this issue Nov 5, 2020
@knrt10
Add packet-ccm
4ff2879 
- Add helm chart for packet-ccm.
- Run packet-ccm while bootstrapping.

closes: #548
Signed-off-by: knrt10 <kautilya@kinvolk.io>
knrt10 added a commit that referenced this issue Nov 5, 2020
@knrt10
Add packet-ccm
6e014b9 
- Add helm chart for packet-ccm.
- Run packet-ccm while bootstrapping.

closes: #548
Signed-off-by: knrt10 <kautilya@kinvolk.io>
knrt10 added a commit that referenced this issue Nov 6, 2020
@knrt10
Add packet-ccm
7264c50 
- Add helm chart for packet-ccm.
- Run packet-ccm while bootstrapping.
- Export field auth_token to packet.

This is required for packet-ccm. packet-ccm needs secret to be
configured before it is applied. In that secret 2 values are required,
one is packet project_id and other is api_key. We were already getting
project_id and auth_token from user.

closes: #548
Signed-off-by: knrt10 <kautilya@kinvolk.io>
knrt10 added a commit that referenced this issue Nov 6, 2020
@knrt10
Add packet-ccm
b213bb2 
- Add helm chart for packet-ccm.
- Run packet-ccm while bootstrapping.
- Export field auth_token to packet.

This is required for packet-ccm. packet-ccm needs secret to be
configured before it is applied. In that secret 2 values are required,
one is packet project_id and other is api_key. We were already getting
project_id and auth_token from user.

closes: #548
Signed-off-by: knrt10 <kautilya@kinvolk.io>
knrt10 pushed a commit that referenced this issue Nov 23, 2020
@invidian @knrt10
pkg/components/metallb: read peer config also from annotations
dd4f967 
Which are set by Packet CCM right now.

Refs #548

Signed-off-by: Mateusz Gozdek <mateusz@kinvolk.io>
knrt10 pushed a commit that referenced this issue Nov 23, 2020
@invidian @knrt10
packet: enable CCM
3f58bef 
This commit enables CCM for Packet platform to allow dropping hacky way
of labeling Node objects on Packet via Ignition script for MetalLB
autodiscovery to work, which we are doing right now.

Closes #548

Signed-off-by: Mateusz Gozdek <mateusz@kinvolk.io>
knrt10 added a commit that referenced this issue Nov 23, 2020
@knrt10
kubelet Helm chart: add support for setting --cloud-provider flag
a50ca11 
Using CCM on Kubernetes requires --cloud-provider to be set to
"external".

This commit enables this customization. Default value remains empty
string, so no flag will be set for compatibility with other platforms.

Refs #548

Signed-off-by: knrt10 <kautilya@kinvolk.io>
knrt10 pushed a commit that referenced this issue Nov 23, 2020
@invidian @knrt10
assets/charts/control-plane: add packet-ccm Helm chart
14f6590 
This commit add packet-ccm Helm chart based on manifests from
https://github.com/packethost/packet-ccm/tree/master/deploy/template,
but with Lokomotive-specific modifications like using image with BGP
Node Selector support, enabled leader election for HA, removed
unnecessary RBAC grants etc.

This chart will be used to enable CCM on Packet platform.

Refs #548

Signed-off-by: Mateusz Gozdek <mateusz@kinvolk.io>
knrt10 pushed a commit that referenced this issue Nov 23, 2020
@invidian @knrt10
test/platform/packet: add test for enabling BGP
c5a7b20 
So we don't hit a regression when we move the responsibility of enabling
BGP for nodes from Terraform to Packet CCM.

Refs #548

Signed-off-by: Mateusz Gozdek <mateusz@kinvolk.io>
knrt10 pushed a commit that referenced this issue Nov 23, 2020
@invidian @knrt10
pkg/components/metallb: read peer config also from annotations
05f0a24 
Which are set by Packet CCM right now.

Refs #548

Signed-off-by: Mateusz Gozdek <mateusz@kinvolk.io>
knrt10 pushed a commit that referenced this issue Nov 23, 2020
@invidian @knrt10
packet: enable CCM
a60c650 
This commit enables CCM for Packet platform to allow dropping hacky way
of labeling Node objects on Packet via Ignition script for MetalLB
autodiscovery to work, which we are doing right now.

Closes #548

Signed-off-by: Mateusz Gozdek <mateusz@kinvolk.io>
invidian pushed a commit that referenced this issue Nov 23, 2020
@knrt10 @invidian
kubelet Helm chart: add support for setting --cloud-provider flag
6013792 
Using CCM on Kubernetes requires --cloud-provider to be set to
"external".

This commit enables this customization. Default value remains empty
string, so no flag will be set for compatibility with other platforms.

Refs #548

Signed-off-by: knrt10 <kautilya@kinvolk.io>
invidian added a commit that referenced this issue Nov 23, 2020
@invidian
assets/charts/control-plane: add packet-ccm Helm chart
5b864eb 
This commit add packet-ccm Helm chart based on manifests from
https://github.com/packethost/packet-ccm/tree/master/deploy/template,
but with Lokomotive-specific modifications like using image with BGP
Node Selector support, enabled leader election for HA, removed
unnecessary RBAC grants etc.

This chart will be used to enable CCM on Packet platform.

Refs #548

Signed-off-by: Mateusz Gozdek <mateusz@kinvolk.io>
invidian added a commit that referenced this issue Nov 23, 2020
@invidian
test/platform/packet: add test for enabling BGP
8421216 
So we don't hit a regression when we move the responsibility of enabling
BGP for nodes from Terraform to Packet CCM.

Refs #548

Signed-off-by: Mateusz Gozdek <mateusz@kinvolk.io>
invidian added a commit that referenced this issue Nov 23, 2020
@invidian
pkg/components/metallb: read peer config also from annotations
f7cb8a8 
Which are set by Packet CCM right now.

Refs #548

Signed-off-by: Mateusz Gozdek <mateusz@kinvolk.io>
invidian added a commit that referenced this issue Nov 23, 2020
@invidian
packet: enable CCM
5922ff7 
This commit enables CCM for Packet platform to allow dropping hacky way
of labeling Node objects on Packet via Ignition script for MetalLB
autodiscovery to work, which we are doing right now.

Closes #548

Signed-off-by: Mateusz Gozdek <mateusz@kinvolk.io>
@invidian invidian added this to the v0.6.0 milestone Nov 25, 2020
invidian pushed a commit that referenced this issue Dec 2, 2020
@knrt10 @invidian
kubelet Helm chart: add support for setting --cloud-provider flag
e389bbe 
Using CCM on Kubernetes requires --cloud-provider to be set to
"external".

This commit enables this customization. Default value remains empty
string, so no flag will be set for compatibility with other platforms.

Refs #548

Signed-off-by: knrt10 <kautilya@kinvolk.io>
invidian added a commit that referenced this issue Dec 2, 2020
@invidian
assets/charts/control-plane: add packet-ccm Helm chart
3540d0f 
This commit add packet-ccm Helm chart based on manifests from
https://github.com/packethost/packet-ccm/tree/master/deploy/template,
but with Lokomotive-specific modifications like using image with BGP
Node Selector support, enabled leader election for HA, removed
unnecessary RBAC grants etc.

This chart will be used to enable CCM on Packet platform.

Refs #548

Signed-off-by: Mateusz Gozdek <mateusz@kinvolk.io>
invidian added a commit that referenced this issue Dec 2, 2020
@invidian
test/platform/packet: add test for enabling BGP
d96fb69 
So we don't hit a regression when we move the responsibility of enabling
BGP for nodes from Terraform to Packet CCM.

Refs #548

Signed-off-by: Mateusz Gozdek <mateusz@kinvolk.io>
invidian added a commit that referenced this issue Dec 2, 2020
@invidian
pkg/components/metallb: read peer config also from annotations
a23cee2 
Which are set by Packet CCM right now.

Refs #548

Signed-off-by: Mateusz Gozdek <mateusz@kinvolk.io>
invidian added a commit that referenced this issue Dec 2, 2020
@invidian
packet: enable CCM
1cee197 
This commit enables CCM for Packet platform to allow dropping hacky way
of labeling Node objects on Packet via Ignition script for MetalLB
autodiscovery to work, which we are doing right now.

Closes #548

Signed-off-by: Mateusz Gozdek <mateusz@kinvolk.io>
invidian added a commit that referenced this issue Dec 2, 2020
@invidian
packet: enable CCM
c7022ef 
This commit enables CCM for Packet platform to allow dropping hacky way
of labeling Node objects on Packet via Ignition script for MetalLB
autodiscovery to work, which we are doing right now.

Closes #548

Signed-off-by: Mateusz Gozdek <mateusz@kinvolk.io>
invidian pushed a commit that referenced this issue Dec 3, 2020
@knrt10 @invidian
kubelet Helm chart: add support for setting --cloud-provider flag
e36e643 
Using CCM on Kubernetes requires --cloud-provider to be set to
"external".

This commit enables this customization. Default value remains empty
string, so no flag will be set for compatibility with other platforms.

Refs #548

Signed-off-by: knrt10 <kautilya@kinvolk.io>
invidian added a commit that referenced this issue Dec 3, 2020
@invidian
assets/charts/control-plane: add packet-ccm Helm chart
5b43534 
This commit add packet-ccm Helm chart based on manifests from
https://github.com/packethost/packet-ccm/tree/master/deploy/template,
but with Lokomotive-specific modifications like using image with BGP
Node Selector support, enabled leader election for HA, removed
unnecessary RBAC grants etc.

This chart will be used to enable CCM on Packet platform.

Refs #548

Signed-off-by: Mateusz Gozdek <mateusz@kinvolk.io>
invidian added a commit that referenced this issue Dec 3, 2020
@invidian
test/platform/packet: add test for enabling BGP
7ca8f36 
So we don't hit a regression when we move the responsibility of enabling
BGP for nodes from Terraform to Packet CCM.

Refs #548

Signed-off-by: Mateusz Gozdek <mateusz@kinvolk.io>
invidian added a commit that referenced this issue Dec 3, 2020
@invidian
pkg/components/metallb: read peer config also from annotations
0d0a8e5 
Which are set by Packet CCM right now.

Refs #548

Signed-off-by: Mateusz Gozdek <mateusz@kinvolk.io>
invidian added a commit that referenced this issue Dec 3, 2020
@invidian
packet: enable CCM
67d5710 
This commit enables CCM for Packet platform to allow dropping hacky way
of labeling Node objects on Packet via Ignition script for MetalLB
autodiscovery to work, which we are doing right now.

Closes #548

Signed-off-by: Mateusz Gozdek <mateusz@kinvolk.io>
invidian added a commit that referenced this issue Dec 3, 2020
@invidian
assets/charts/control-plane: add packet-ccm Helm chart
2e7f696 
This commit add packet-ccm Helm chart based on manifests from
https://github.com/packethost/packet-ccm/tree/master/deploy/template,
but with Lokomotive-specific modifications like using image with BGP
Node Selector support, enabled leader election for HA, removed
unnecessary RBAC grants etc.

This chart will be used to enable CCM on Packet platform.

Refs #548

Signed-off-by: Mateusz Gozdek <mateusz@kinvolk.io>
invidian added a commit that referenced this issue Dec 3, 2020
@invidian
test/platform/packet: add test for enabling BGP
ed52211 
So we don't hit a regression when we move the responsibility of enabling
BGP for nodes from Terraform to Packet CCM.

Refs #548

Signed-off-by: Mateusz Gozdek <mateusz@kinvolk.io>
invidian added a commit that referenced this issue Dec 3, 2020
@invidian
pkg/components/metallb: read peer config also from annotations
a7217c4 
Which are set by Packet CCM right now.

Refs #548

Signed-off-by: Mateusz Gozdek <mateusz@kinvolk.io>
invidian added a commit that referenced this issue Dec 3, 2020
@invidian
packet: enable CCM
02b1ef6 
This commit enables CCM for Packet platform to allow dropping hacky way
of labeling Node objects on Packet via Ignition script for MetalLB
autodiscovery to work, which we are doing right now.

Closes #548

Signed-off-by: Mateusz Gozdek <mateusz@kinvolk.io>
Roadmap automation moved this from Queued to Done Dec 3, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@invidian invidian

@knrt10 knrt10

Labels
kind/roadmap Roadmap issues size/l Issues which likely require many work days (but not weeks)
Projects
Roadmap
  
Done
Milestone
v0.6.0
Development

Successfully merging a pull request may close this issue.

Add Packet CCM to Packet platform kinvolk/lokomotive
5 participants
@iaguis @surajssd @johananl @invidian @knrt10