Skip to content

Add ExternalServicePorts helper — public-facing port set (NodePortsForMode minus scrape-only ports) #7

@bdchatham

Description

@bdchatham

Summary

Add a helper alongside NodePortsForMode(mode) that returns the subset of ports appropriate for public-facing / client-traffic services, as opposed to the full per-pod port set.

Proposed signature:

// ExternalServicePorts returns the subset of NodePortsForMode that should
// be exposed on a public-facing (L4 aggregate) Service. Excludes
// scrape-only and gossip-only ports (e.g. metrics).
func ExternalServicePorts(mode NodeMode) []NodePort

Motivation

In sei-protocol/sei-k8s-controller, the controller creates three kinds of Services per SeiNodeDeployment pod:

  • <sndname>-<ordinal> — per-pod headless (scrape target, internal use)
  • <sndname>-internal — aggregate ClusterIP (internal client traffic)
  • <sndname>-external — aggregate ClusterIP (public-facing, backs HTTPRoutes when networking: {} is set)

Today, generateExternalService uses portsForMode(...) which calls NodePortsForMode(...) — same set as the per-pod service — so the external Service ends up with a metrics:26660 port it shouldn't have. ServiceMonitor selectors then match both the per-pod and external services, double-scraping every pod (observed as 2×pods series in Prometheus for k8s-managed chains).

I'm shipping a workaround in the controller that drops metrics inline (sei-protocol/sei-k8s-controller PR TBD after this issue). That's the wrong place for the rule — the definition of "what ports belong on an external Service" is a sei-config concern since sei-config already owns NodePortsForMode.

Scope

  • Add ExternalServicePorts(mode) returning NodePortsForMode(mode) minus ports tagged as non-public. Minimally: drop metrics.
  • Consider also dropping p2p — it's gossip and today gets exposed publicly, which may be intentional for operator-run external validators but isn't for an RPC fleet. Happy to discuss.
  • Semantic tags on ports (e.g. Public bool, ScrapeOnly bool) could back this cleanly and let callers compose their own filters, if you want a more durable API. The one-off ExternalServicePorts helper is the minimum.

Why not just fix it in the controller

Two reasons:

  1. The "which ports are public" fact belongs with the port definitions, not scattered across consumers.
  2. Other consumers (Alloy config, future side-projects) may need the same filter and will replicate the logic ad-hoc.

Referenced by: sei-protocol/sei-k8s-controller (controller ships a tactical inline fix pending this helper).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions