proposal: Pomerium Chart simplification

[travisgroth]

Background

The Pomerium helm chart has evolved with both the capabilities of the core project as well as the needs of the community over the years. As such, there are a number of options for new users to navigate and for maintainers to reason about when making changes. The result is complex conditionals and template rendering which can wind up breaking in unexpected ways and lead folks down the path of options that only exist for older installs. Broadly, we've kept an implicit backward compatibility guarantee in the chart such that whichever deployment model you used 3 years ago is still supported and probably the default.

Problem

With these older deployment models still being implicitly supported and encouraged in the chart, support requests have become difficult to field as many factors go into how a component might be configured, work, and interact with surrounding systems.

Here's a rough list of the deployment models in the chart in date order.

1. Ingress based proxy. Applications were configured under a single wildcard domain to simplify the Ingress resource and all routes had to also be defined manually in the config block.
2. Ingress based proxy with forward auth. Same as (1) but a third party ingress controller was responsible for handling the data plane. This required manual manipulation of the config block for each route definition, but allowed for use of more than one Ingress for disparate applications.
3. Forward auth with Operator. This is (2) but with some automation built to turn annotations on a third party Ingress resource into route/policy definitions in the configuration.
4. LoadBalancer. Pomerium receives an external IP and terminates TLS with any of a number of certs, requiring only DNS to be pointed at the IP. Route definitions and policies are maintained in the config block or Pomerium Enterprise.
5. Ingress Controller. Built on top of (4) but, like (3), automatically turns Ingress resources into concrete Pomerium route and policy definitions. This replaces the Operator and promotes Pomerium into the category of first class Ingress.

To this day, (1) is still the default. Anecdotally, it would appear (3) is the most widely used. (5) is what we would consider best supported by Pomerium (both org and software) and the current recommended best practice. At this stage, (5) is the likely terminal model, though there will be additional internal improvements as time goes on.

Proposal

Given the background and current state, it is time to reduce the number of ways a user may interact with the official chart. The goal would be to standardize on a single model (5), and remove support for older models from the helm chart itself.

This would provide a cleaner experience for new users, reduce maintenance overhead, reduce support burden, improve the ability for the community to contribute, and lead to much greater stability in the chart. It also paves the way for future changes which will be simpler/safer if the deployment model is singular.

It is worth making explicit note of what this proposal means for users of deployment models 1/2/3.

• Pomerium itself is not being changed to remove support. Existing manifests should continue to work, unless breaking changes are made in Pomerium itself.
• The older chart versions will not be removed. However, they will enter the realm of community support.
• Forward auth will continue to be a function supported by Pomerium core, but will not be a model the helm chart provides going forward.
• The Ingress Controller's operator-like run mode will continue to exist but will not be directly exposed by the chart.

The resulting chart would effectively be like running the current chart with the following values:

ingress:
  enabled: false
ingressController:
  enabled: true
forwardAuth:
  enabled: false

However, this would not be configurable.

Comments

We're opening this proposal in public before working on the changes for a few reasons: First, we want the community to have an opportunity to understand our reasoning and to contribute to the discussion. Second, we want such a move to be communicated clearly in advance rather than show up as a breaking change already baked.

This proposal is exactly that - a proposal - but one which we do have an interest in seeing through. We'd welcome comments and discussion ahead of the change and this issue is the place to do so.

[desimone]

related:

• pomerium: add support all-in-one pomerium service #285 (comment)