Skip to content

Stuck figuring out how to expose a Kubernetes Service internally using NetBird Operator. #182

Description

@Archisman-Mridha

I have a Hetzner Network with CIDR 10.0.0.0/16. And a workload cluster is running inside that private network. The cluster has default Kubernetes Service CIDR :

kubectl get servicecidr
NAME         CIDRS          AGE
kubernetes   10.96.0.0/12   7d9h

I have deployed NetBird Operator in that workload cluster, with the following Helm values :

managementURL: "...."

netbirdAPI:
  keyFromSecret:
    name: "netbird-management-api-key"
    key: "NB_API_KEY"

cluster:
  name: qa-cluster

ingress:
  enabled: true

  # Allow creating policies through Service annotations
  allowAutomaticPolicyCreation: true

  # Kubernetes services is a common way to route traffic to your application pods. With the NetBird
  # operator ingress you can expose services to your NetBird network as resources by using
  # annotations in your services. The operator will create networks, resources, and add routing
  # peers to your NetBird configuration.
  router:
    enabled: true

    tolerations:
      - operator: Exists
        key: node-role.kubernetes.io/control-plane
    nodeSelector:
      node-role.kubernetes.io/control-plane: ""

# To access your Kubernetes control plane from a NetBird network, you can expose your Kubernetes
# control plane as a NetBird resource.
kubernetesAPI:
  enabled: true

operator:
  tolerations:
    - operator: Exists
      key: node-role.kubernetes.io/control-plane
  nodeSelector:
    node-role.kubernetes.io/control-plane: ""

webhook:
  failurePolicy: Ignore

Now, I have deployed NGINX. Here is the Kubernetes Service for that NGINX deployment :

apiVersion: v1
kind: Service
metadata:
  annotations:
    netbird.io/expose: "true"
    netbird.io/policy: default
    netbird.io/resource-name: nginx
    netbird.io/policy-source-groups: Obmondo
    netbird.io/groups: QA cluster nodes
  labels:
    app: nginx
  name: nginx
  namespace: netbird
spec:
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: 80
  selector:
    app: nginx
  type: ClusterIP

I can see that the NetBird Operator has created the nginx resource in the qa-cluster network in NetBird :

Image

In my machine :

netbird networks list
Available Networks:

  - ID: nginx
    Domains: nginx.netbird.svc.cluster.local
    Status: Not Selected
    Resolved IPs: -

  - ID: qa-cluster
    Network: 10.0.0.0/16
    Status: Selected

If I do dig +short nginx.netbird.svc.cluster.local, it doesn't resolve to any IP.

Now, if I disconnect from the qa-cluster network and connect to nginx network, dig +short nginx.netbird.svc.cluster.local resolves to 10.107.184.110 : which is the IP of the Kubernetes Service corresponding to the NGINX deployment. But ofcourse, it's not reachable from my local machine, since I have disconnected from the qa-cluster network.

I need some help with how to expose this NGINX deployment internally using the NetBird Operator 🙏.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions