design: Add a pattern for adding extensible status information to HTTPProxy

[youngnick]

This issue covers the work to add more information to HTTPProxy's status stanza.

Features we need from HTTPProxy's status stanza:

• Ability to see the way to reach the HTTPProxy (the equivalent to the Ingress status.loadBalancer stanza). Set load balancer address on HTTPProxy objects #2371 covers the work to implement this.
• Ability to set multiple statuses on a HTTPProxy. Although the guidance for their design is somewhat muddy at the moment (see Update Condition guidance kubernetes/community#4521), Conditions are the API-standard way to do this.
• A way for HTTPProxy resources to understand what routing conditions are delegated to them from parent HTTPProxies, and possibly a way for parent HTTPProxies to see what is visible underneath them.

Blocks #2325, through this blocks #370, #432, and #1691, and #399
Blocks #2021
Blocks #2371
Blocks #1598
Blocks #2141

Previous work was in #1868, although that was before I knew about Conditions.

We may also need to consider changing the status stanza to be an actual subresource, although that may be a large enough change that it should be a new document version We did need to update to a subresource to make handling the status updates easier, and to allow separate RBAC of them.

[bgagnon]

This would be a much welcome addition for us:

• we auto-provision DNS from HTTPProxy, but we're lacking a way to communicate the status of that back to the user
• we also auto-provision TLS certificates -- Contour's status field says valid when the Secret exists and invalid before that, but there's not a lot of nuance

Our solution for now is to emit detailed events (success/error) attached to the HTTPProxy for the above. If the status field can reflect the Envoy configuration status, plus some extensible fields, then we'd have a comprehensive status.

The only thing missing for a full CD pipeline would be for kubectl to grow the ability to wait on these status values (kubernetes/kubernetes#83094).

[youngnick]

That's a good point @bgagnon - As I start experimenting with adding Conditions to HTTPProxy, I'll make sure I try out the support kubectl wait has for Conditions already.

In addition, after talking to @jpeach and @stevesloka today, I'm going to go ahead with #2371, and add the loadBalancer stanza to HTTPProxy. This will work in the same way that Ingress status updating works today - either pulled from the Envoy service, or passed directly to Contour.

[stevesloka]

@youngnick what were the thoughts around the basic info bits? I was looking to add things like fqdn, path, headers, etc to all proxies giving users a way to see what's been included.

Poking around a bit we need to add an Info section to the DetailedCondition.

Something like:

// Infos contains a slice of relevant info subconditions for this object.
// +optional
Infos []SubCondition `json:"infos,omitempty"`

Then if we re-use the same SubConcondition struct, we'd get something like this:

status:
  conditions:
  - infos:
    - message: baremetal.slokalabs.io.  # <--------------- NEW!
      reason: info
      status: "True"
      type: fqdn
    lastTransitionTime: "2020-11-16T21:29:19Z"
    message: Valid HTTPProxy
    observedGeneration: 2
    reason: Valid
    status: "True"
    type: Valid
  currentStatus: valid
  description: Valid HTTPProxy
  loadBalancer: {}

I'd like to agree on what we thing we should call all this bits. In this example I named it Infos since we have Errors and Warnings and couldn't think of a better name.

(@skriss )

[jpeach]

@youngnick what were the thoughts around the basic info bits? I was looking to add things like fqdn, path, headers, etc to all proxies giving users a way to see what's been included.

Why would you not just add these fields directly to status?

[stevesloka]

Makes sense to me @jpeach, was just trying to follow the idea of the conditions spec idea.

Following the thread of just adding to the status struct could have something like this:

// HTTPProxyStatus reports the current state of the HTTPProxy.
type HTTPProxyStatus struct {
  ...
  <existing> 
   ...
  
  FQDN []string
  Routes []RouteStatus 
}

// RouteStatus represents the status of fully qualified 
// conditions that might be included to this HTTPProxy
type RouteStatus struct {
  MatchConditions []MatchCondition
}

[stevesloka]

Other bits of information like who my parent is, might be interesting to follow the chain of inclusion around.

[youngnick]

We'll talk about this more, but my initial thoughts were:

• whatever we do, it has to be namespaced by root fqdn, and then show the full set of conditions, since a HTTPProxy can be included multiple times, either under different roots or the same root.
• I think that Conditions is actually not a good fit for this information - it's better as a field of some sort.
• Showing who the parent is would also be useful.

[youngnick]

Closed by mistake, my bad.

[github-actions]

The Contour project currently lacks enough contributors to adequately respond to all Issues.

This bot triages Issues according to the following rules:

• After 60d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, the Issue is closed

You can:

• Mark this Issue as fresh by commenting
• Close this Issue
• Offer to help out with triage

Please send feedback to the #contour channel in the Kubernetes Slack

[github-actions]

The Contour project currently lacks enough contributors to adequately respond to all Issues.

This bot triages Issues according to the following rules:

• After 60d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, the Issue is closed

You can:

• Mark this Issue as fresh by commenting
• Close this Issue
• Offer to help out with triage

Please send feedback to the #contour channel in the Kubernetes Slack