Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CFP: Operator Manages Cilium Identities #27752

Closed
dlapcevic opened this issue Aug 28, 2023 · 5 comments
Closed

CFP: Operator Manages Cilium Identities #27752

dlapcevic opened this issue Aug 28, 2023 · 5 comments
Labels
kind/feature This introduces new functionality. sig/agent Cilium agent related. stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale.

Comments

@dlapcevic
Copy link
Contributor

dlapcevic commented Aug 28, 2023

Cilium Feature Proposal

Thanks for taking time to make a feature proposal for Cilium! If you have usage questions, please try the slack channel and see the FAQ first.

Is your proposed feature related to a problem?

CID - Cilium Identity

CIDs are presenting issues with reliability and scalability, and are causing network policies to be in a broken state. Most notable cases:

  • Reaching maximum number (65k) of CIDs in the cluster.
  • eBPF policy maps overflowing with over 16k entries.
  • CID garbage collection malfunction.
  • Network policy incorrectly dropping connections on KCP upgrade, when cilium-agent restarts. Related to CID duplication
  • Unhealthy cluster state that is difficult (or impossible) to recover from - related to numbers of IDs (and other Cilium custom resources) and CID duplication.

Describe the feature you'd like

Centralize CID management to a single pod (Operator), instead of distributed management, done by all Agents. The goal is to improve security, reliability, performance and scalability of CID, and enable more advanced optimizations.

(Optional) Describe your proposed solution

CFP document

Phase 1: Operator manages Cilium Identities.
Move CID creation from cilium-agent to cilium-operator.

Phase 2: CID lazy creation
Create IDs only for labels used in network policies to greatly reduce the number of CIDs. Only pod labels used in the peer pod label selector of network policies will be relevant for ID creation.

A separate CFP is required for the Phase 2.


Please complete this section if you have ideas / suggestions on how to implement the feature. We strongly recommend discussing your approach with Cilium committers before spending lots of time implementing a change.

For longer proposals, you are welcome to link to an external doc (e.g. a Google doc). We have a Cilium Feature Proposal template to help you structure your proposal - if you would like to use it, please make a copy and ensure it's publicly visible, and then add the link here.

Once the CFP is close to being finalized, please add it as a PR to the design-cfps repo for final approval.

@dlapcevic dlapcevic added the kind/feature This introduces new functionality. label Aug 28, 2023
@dlapcevic dlapcevic changed the title CFP: CFP: Operator Manages Cilium Identities Aug 28, 2023
@aanm aanm added the sig/agent Cilium agent related. label Aug 28, 2023
Copy link

github-actions bot commented Dec 5, 2023

This issue has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.

@github-actions github-actions bot added the stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale. label Dec 5, 2023
@dlapcevic
Copy link
Contributor Author

The implementation is in progress.

@github-actions github-actions bot removed the stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale. label Dec 19, 2023
@linsun
Copy link
Contributor

linsun commented Jan 12, 2024

Hi there! I wanted to mention Rob proposed xDS adapter for Cilium which may be relevant here, PTAL: https://docs.google.com/document/d/1U4pO_dTaHERKOtrneNA8njW19HSVbq3sBM3x8an4878/edit#heading=h.y3v1ksm0ev6r

Copy link

This issue has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.

@github-actions github-actions bot added the stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale. label Mar 13, 2024
dlapcevic pushed a commit to dlapcevic/cilium that referenced this issue Mar 22, 2024
**Feature: Operator Manages Cilium Identities**

CFP: cilium#27752

**Description**

A new feature hidden behind a flag. Disabled by default.

Besides the new flag that enables the feature, there are no other user visible changes.

cilium-operator is going to manage (create and delete) Cilium Identity API objects instead of cilium-agent.

cilium-operator calculates the desired state for Cilium Identities based on watched events for Cilium Identities, Pods, Namespaces and Cilium Endpoint Slices (if enabled).

cilium-operator creates Cilium Identities on pod updates for a unique label set based on pod and namespace labels.

cilium-operator deletes Cilium Identities when their labels are not used by any pods.

cilium-agent no longer writes to Cilium Identities.

cilium-agent only watches Cilium Identities.

In case when there is no Cilium Identity in the watcher store for a newly created or updated pod, a temporary security identity (temp id) will be created and used locally by the agent, until it's replaced by a global identity (Cilium Identity).

```release-note
Feature: Operator Manages Cilium Identities
A new feature hidden behind a flag. Disabled by default.
cilium-operator is going to manage (create and delete) Cilium Identity API objects instead of cilium-agent.
```

kind/feature

Signed-off-by: Dorde Lapcevic <dordel@google.com>
Copy link

This issue has not seen any activity since it was marked stale.
Closing.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Mar 27, 2024
dlapcevic pushed a commit to dlapcevic/cilium that referenced this issue Apr 11, 2024
**Feature: Operator Manages Cilium Identities**

CFP: cilium#27752

**Description**

A new feature hidden behind a flag. Disabled by default.

Besides the new flag that enables the feature, there are no other user visible changes.

cilium-operator is going to manage (create and delete) Cilium Identity API objects instead of cilium-agent.

cilium-operator calculates the desired state for Cilium Identities based on watched events for Cilium Identities, Pods, Namespaces and Cilium Endpoint Slices (if enabled).

cilium-operator creates Cilium Identities on pod updates for a unique label set based on pod and namespace labels.

cilium-operator deletes Cilium Identities when their labels are not used by any pods.

cilium-agent no longer writes to Cilium Identities.

cilium-agent only watches Cilium Identities.

In case when there is no Cilium Identity in the watcher store for a newly created or updated pod, a temporary security identity (temp id) will be created and used locally by the agent, until it's replaced by a global identity (Cilium Identity).

```release-note
Feature: Operator Manages Cilium Identities
A new feature hidden behind a flag. Disabled by default.
cilium-operator is going to manage (create and delete) Cilium Identity API objects instead of cilium-agent.
```

kind/feature

Signed-off-by: Dorde Lapcevic <dordel@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature This introduces new functionality. sig/agent Cilium agent related. stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale.
Projects
None yet
Development

No branches or pull requests

3 participants