Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Metadata discovery service #43937

Closed
kyessenov opened this issue Mar 15, 2023 · 36 comments
Closed

Metadata discovery service #43937

kyessenov opened this issue Mar 15, 2023 · 36 comments
Assignees
@kyessenov
Labels
Ambient Beta Must have for Beta of Ambient Mesh area/extensions and telemetry

Comments

@kyessenov
Copy link
Contributor

This is a proposal to use the Workload Discovery Service in the Istio mesh as the source of the peer metadata for the telemetry.
This effectively deprecates the requirement to supply the baggage header in the request/response pair in the HBONE protocol, and provides an alternative design for the existing metadata exchange protocols in the sidecar mesh.

Requirements

R1: Ambient mesh produces Istio peer telemetry istio_requests_total for backwards compatibility with the sidecar mode. Additionally, any new telemetry proposals using Otel are supported.
R2: Pay-as-you-go for the telemetry - users should not carry costs unless they choose to get the value of the telemetry.
R3: Minimal requirements on the mesh members to join the mesh.
R4: Trust in the peer metadata.

Problems with the status quo

The current design for the telemetry production relies on several disjoint mechanisms to produce the standard telemetry in the Istio mesh:

1. HTTP metadata exchange

HTTP sidecar telemetry relies on the special headers x-envoy-peer-metadata to announce the source metadata per each request. This design is very flexible, but unfortunately it suffers from several issues: high cost and exclusive headers. First, the high per-request cost of the telemetry: each request gets up to 4K overhead on the wire, and up to 10% CPU overhead decoding the header. This is counter to the goal R2 - any Istio telemetry forces enablement of HTTP metadata exchange, which incurs significant data plane overhead. Second, special headers are counter to goal R3. To produce the telemetry on the server, clients have to synthesize the non-standard Istio header. Third, there is no way to establish the integrity of the metadata for R4, since the peer metadata is not signed by a trusted authority.

2. TCP ALPN metadata exchange

TCP traffic uses a different protocol that relies on the bytes prefix in TCP connections, guarded by a special TLS ALPN string. This proposal violates R3 - custom ALPN and custom wire encoding is incompatible with any client except the Istio sidecar. Similar to the above, R4 is not satisfied, and the metadata is untrusted.

3. Baggage: HTTP CONNECT metadata exchange

This is the same as the first design, but instead works on the longer living tunnel HTTP CONNECT streams. It suffers from the same problems: extra cost on the wire, extra cost for the interoperation, lack of integrity. The main benefit is better protocol encapsulation, since the applications no longer see the metadata header in-flight. Current implementation only supports the server-side reporting, with the client telemetry reporting depending on EDS design below.

4. CDS and EDS

This is a fallback mechanism to use the dynamic metadata from CDS and EDS in case the metadata exchange fails. This is generally useful for the failure scenarios and for destinations outside the mesh. However, this clearly violates R2 - any load balancing server has to provide the peer metadata as part of the response, which is traditionally outside the domain of the global load balancing. A load balancing client also receives a much larger xDS even when not using the telemetry included with it. Interoperation in R3 is improved since there is no special data protocol, and the integrity R4 can be implemented by the control plane.

Proposal

We propose to replace all of the above with a dedicated workload metadata service, with the following payload and IP address as the resource key:

message Workload {
  // Name represents the name for the workload.
  // For Kubernetes, this is the pod name.
  // This is just for debugging and may be elided as an optimization.
  string name = 1;
  // Namespace represents the namespace for the workload.
  // This is just for debugging and may be elided as an optimization.
  string namespace = 2;
  // IP address of the workload. Serves as the lookup key.
  bytes address = 3;
  // The SPIFFE identity of the workload. The identity is joined to form spiffe://<trust_domain>/ns/<namespace>/sa/<service_account>.
  // TrustDomain of the workload. May be elided if this is the mesh wide default (typically cluster.local)
  string trust_domain = 6;
  // ServiceAccount of the workload. May be elided if this is "default"
  string service_account = 7;
  // CanonicalName for the workload. Used for telemetry.
  string canonical_name = 10;
  // CanonicalRevision for the workload. Used for telemetry.
  string canonical_revision = 11;
  // WorkloadType represents the type of the workload. Used for telemetry.
  WorkloadType workload_type = 12;
  // WorkloadName represents the name for the workload (of type WorkloadType). Used for telemetry.
  string workload_name = 13;
}
enum WorkloadType {
  DEPLOYMENT = 0;
  CRONJOB = 1;
  POD = 2;
  JOB = 3;
}

This metadata service is xDS-based, supports on-demand lookup, and allows querying metadata by network IP directly. The proposal satisfies all the requirements:

R1: The metadata content in the service subsumes the existing peer metadata.
R2: The service is optional and has zero wire and per-request CPU costs. The memory cost is O(workloads) with SotW xDS and is identical to the existing HTTP metadata exchange with on-demand delta xDS.
R3: Since this proposal eliminates the requirement for special headers or protocols, it’s simpler for endpoints to interoperate with the mesh.
R4: The integrity of the metadata can be established by using trusted lookup keys in the service, e.g. by using IPs when they are not easily spoofed, or by using extra attributes in the TLS certificates.

It is worth noting that a dedicated service is a better alternative to the CDS/EDS design above since it allows decoupling the metadata provider from the load balancing concern. It permits metadata to be keyed by something other than the endpoint addresses, maximizes sharing of data (same endpoints in two clusters). A dedicated client to the metadata service can be embedded in the user applications, or in the telemetry processing pipeline (e.g OTel collector) for backfilling data, instead of shoehorning CDS/EDS for the same purpose.

Risks and Mitigations

1. Control plane load

As for any new xDS, there is a risk with a new operational load on the control plane. It’s worth noting that EDS already includes the majority of the information needed, and is consumed by all endpoints, therefore imposing the load in the sidecar model. To minimize the risk, we propose to share the xDS pipeline with ztunnel, and re-purpose the existing WDS as-is. We intentionally deleted the PTR parts of the proto related to the authorization, since that is outside the scope of the proposal.

2. Data plane load

There is additional memory overhead required to hold the peer metadata in the proxies. To minimize the risk, we propose to couple metadata discovery with HBONE. In other words, HBONE would require metadata discovery to produce Istio telemetry in all clients of HBONE. This would leave sidecars without HBONE safe, and allow us to gradually gain experience with the service as HBONE matures. A longer term approach to minimize the memory overhead is to switch to the on-demand model. This would require modifying the telemetry pipelines in Envoy to be asynchronous and await for xDS response before flushing the telemetry reports. This can be delayed until there’s a strong need for it, since xDS protocol fully supports this mode of operation.
@howardjohn
Copy link
Member

One thing I wasn't sure about - if I get a request from 1.2.3.4, that IP may live in other networks. How do we handle this? (sorry if its discussed already, wanted to comment befor I forgot)

@costinm
Copy link
Contributor

costinm commented Mar 16, 2023

Related comment: please add a section on how we get the IP and network.

Default is simple - from the network layer.

As a server - x-forwarder-for or forwarded headers I assume, but only if the peer is auth and is a gateway or waypoint.

As client - don't remember if xff is propagated on return, need to check.

Would be worth adding a section on security ( can a client forge his telemetry ?).

I still believe we should include podname and clustername in all responses and allow hostname in addition to network+ip - I would use clustername instead of network, since Istiod may need to do on-demand lookup too in very large meshes.

@kyessenov
Copy link
Contributor Author

@howardjohn For multi-network, the proposal is to aggregate metadata in a federated way. That means that the metadata provider can access metadata from any mesh endpoint. Specifically, for multi-cluster endpoints we'd expose k8s metadata from all clusters to the provider.

@costinm Those are good points, and I don't have a good answer for all of them and deliberately left them underspecified.
There are several options:

  1. Client or server lookup using mTLS property. ztunnel presents a client pod certificate with client pod name in it, and we look up based on that.

  2. Client lookup using x-forwarded-for. ztunnel appends the header during forwarding and we use the first hop IP address as the key.

  3. Client lookup using pod name and pod cluster. ztunnel injects or propagates the header identifying the "real client" as pod ID and pod cluster ID, and we that as the key.

  4. Server lookup using destination IP. A client uses the destination IP and/or endpoint metadata as the lookup key.

  5. Server lookup using destination pod name and pod cluster. We'd require special response headers to identify the workload in the server gateway/ztunnel.

  6. All of the above.

I think there's a need to standardize on the "workload identification headers" in HBONE protocol. Baggage is an indirect way to deliver this information, but I think we need a first class representation for it, not assume only telemetry usage.

For background information, OTEL k8s attribute processor fulfills the same design goal in a broader k8s context.

@lei-tang
Copy link
Contributor

lei-tang commented Apr 5, 2023

Hi Kuat, thanks for the proposal! Can you add a detailed workflow diagram to help readers better understand the proposal?

@kyessenov
Copy link
Contributor Author

@lei-tang This is a basic workflow for metadata discovery by a gateway:
Unnamed Diagram

CC @markdroth : FYI a proposal to drop "peer metadata" header from transport protocol requirements, and rely on a separate "back-fill" metadata flow. This aligns well with OTel processor pipeline architecture, instantiated as a custom xDS-based telemetry processor in Envoy.

@kyessenov kyessenov mentioned this issue Apr 5, 2023
Open
@howardjohn
Copy link
Member

One thing I think Baggage gives that this doesn't is the ability for the client to tell the server which Service it access it through

@costinm
Copy link
Contributor

costinm commented Apr 6, 2023 via email

@kyessenov
Copy link
Contributor Author

FYI destination service was never in scope for the metadata exchange. It's a property of a request, while the subject is a description of the peer. Many things break if you try to put per-request property onto peer metadata since the consumers assume peer metadata is immutable and it is aggressively cached.

Besides that, yes it could be useful. We had that before with a mixer attribute but it was not a popular design choice.

@costinm
Copy link
Contributor

costinm commented Apr 7, 2023 via email

@louiscryan
Copy link
Contributor

Trying to catch up on this a bit and it seems like it might be worth an explicit review.

I'm not totally sold that we can make federation work reliably for multi-network solutions. We already allow for connectivity without full multi-network knowledge via loose coupling. This might even be problematic within large networks with very high cluster counts.

As for keeping bandwidth down I don't see the option of simply POSTing the metadata on connection initiation which would outperform putting into CONNECT headers. The reserved endpoint for this can be versioned so we could actually treat this like a real API. I do agree that many of the alternative methods are pretty terrible.

Finally for verification we can make sure inlined baggage is signed by an authority like the control plane which is what was proposed above. (Aside - this feels a lot like putting things into SANs so might be worth talking about those two things together)

I generally agree with @costinm that we can do both and fallback to the control-plane if we can't resolve inline passing the caller identity etc.

@kdorosh
Copy link
Contributor

kdorosh commented May 15, 2023

For multi-network, the proposal is to aggregate metadata in a federated way. That means that the metadata provider can access metadata from any mesh endpoint. Specifically, for multi-cluster endpoints we'd expose k8s metadata from all clusters to the provider.

just reiterating what @louiscryan said above in a different way: beyond loose coupling, many isolated control plane architectures have strong requirements to preserve telemetry without allowing contact across boundaries (all communication is coming through network gateways, which is exactly what we want telemetry on. we cannot force users to make cross network requests to a federated metadata service for telemetry information)

@kyessenov
Copy link
Contributor Author

@kdorosh @louiscryan

If every externally facing gateway acts as a metadata discovery service for its network, would that address your concern about the centralized metadata discovery service?

Concretely, a server telemetry producer would call gRPC POST asynchronously to retrieve metadata for an endpoint behind a gateway on a well-known endpoint. The gateway address would be either a network address or a dedicated header in the CONNECT request. Every sidecar could also respond to the Metadata discovery to itself.

I agree that signing the header would work as a delegation mechanism, although that would be the first instance in Istio. However, signing doesn't address the other problems with inline headers:

  • coupling with the protocol (CONNECT)
  • per-request overhead (higher in fact with signing)
  • coupling with Istio proxies, a telemetry intermediary like Otel collector cannot participate in the telemetry production and off-load the proxies.

@kdorosh
Copy link
Contributor

kdorosh commented May 15, 2023

If every externally facing gateway acts as a metadata discovery service for its network, would that address your concern about the centralized metadata discovery service?

I think it could, provided we update HBONE baggage or something to include the source network per istio/ztunnel#515

Also in #43937 (comment)

Many things break if you try to put per-request property onto peer metadata since the consumers assume peer metadata is immutable and it is aggressively cached.

Can you elaborate on this caching? Who is caching, why, and for how long? I'm a bit concerned with all this async stuff since IPs can be recycled in k8s.

Also generally worth discussing.. I think this metadata service may have to handle a very high amount of requests (barring aggressive caching, concerns noted above) and that comes with its own operational and CPU/memory costs. Given that for request metrics (not peer metadata) we will still need HTTP headers or TCP metadata for things like originating client IP, originating network, etc.. it seems like we're already paying the cost on each request and that baggage is not as painful as it seems.

@kdorosh kdorosh mentioned this issue May 15, 2023
Open
@kyessenov
Copy link
Contributor Author

Many things break if you try to put per-request property onto peer metadata since the consumers assume peer metadata is immutable and it is aggressively cached.

Can you elaborate on this caching? Who is caching, why, and for how long? I'm a bit concerned with all this async stuff since IPs can be recycled in k8s.

It is cached in the MX extension because the CPU cost to decode the header is significant https://github.com/istio/proxy/blob/master/extensions/metadata_exchange/plugin.cc#L116. There's no expiration or verification - it's easy to confuse telemetry with the same key.

@costinm
Copy link
Contributor

costinm commented May 16, 2023 via email

@bleggett
Copy link
Contributor

bleggett commented May 16, 2023 •
edited
Loading

@kdorosh @louiscryan

If every externally facing gateway acts as a metadata discovery service for its network, would that address your concern about the centralized metadata discovery service?

Concretely, a server telemetry producer would call gRPC POST asynchronously to retrieve metadata for an endpoint behind a gateway on a well-known endpoint. The gateway address would be either a network address or a dedicated header in the CONNECT request. Every sidecar could also respond to the Metadata discovery to itself.

I agree that signing the header would work as a delegation mechanism, although that would be the first instance in Istio. However, signing doesn't address the other problems with inline headers:

* coupling with the protocol (CONNECT)

* per-request overhead (higher in fact with signing)

* coupling with Istio proxies, a telemetry intermediary like Otel collector cannot participate in the telemetry production and off-load the proxies.

  1. If every gateway acts as a metadata source for its network, how do you do control information leakage? Trusting remote proxies to be good consumers? Granted, information leakage is a concern already today with envoy-peer-metadata, but this seems messy.

  2. Signing headers feels gross, it adds a lot of overhead, and doesn't help with 1). If you need to sign a header, then you shouldn't be using a header - something involving out-of-band checks against a singular identity to establish authenticity of source versus relying on request header signatures to establish authenticity of source (e.g. how SPIRE does it) makes a whole lot more sense as a scalable long-term option to me here.

@costinm
Copy link
Contributor

costinm commented May 16, 2023 via email

@bleggett
Copy link
Contributor

bleggett commented May 23, 2023 •
edited
Loading

Where I think I'm at on this:

  1. If we have a metadata discovery service, it will have to be publicly exposed across clusters, and thus have basic authz controls at the very least. I don't think this means we shouldn't do it, but it's probably the largest risk to control for. We could do something where we expose an endpoint that takes a workload cert and returns a metadata blob if the cert was issued by our local CA and is still valid. At this point we're beginning to get pretty "NIH SPIRE"-y though and that means workload certs would have to be conveyed across clusters via XFCC or similar.

  2. If we don't have a metadata discovery service, we need signed headers. And we should probably only construct/send those when crossing boundaries.

  3. If we use signed headers, we should instead just use JWTs, as has been floated previously (which is effectively a purpose-built signed metadata blob header).

  4. We can use a metadata service locally, and send JWTs across borders to avoid 1)

tl;dr it's a metadata service or JWTs, or a hybrid of the two. The hybrid is currently more appealing IMO especially since it can be implemented in stages.

The other option is "lean more heavily on SPIRE workload identity attestation and SPIRE workload identity federation" which is probably not simpler than any of the above for us to do, though it would offer additional attestation and PoP capabilities that the above options do not.

@costinm
Copy link
Contributor

costinm commented May 23, 2023 via email

@bleggett
Copy link
Contributor

bleggett commented May 23, 2023 •
edited
Loading

In Istio we still have each Istiod watch all clusters and pods - so it can
generate EDS - which means it can generate MDS.
And the XDS federation model (which is partially support) is also based on
Istiod talking with other XDS servers. There is some auth and trust in both
cases, of course, but istiod needs to authenticate itself to k8s or XDS
servers.

Makes sense, and as you mention this is simple for single-cluster to start with. I'm not sure how scalable it is to use XDS federation for all cases, but it's not the end of the world to convey the same info differently across boundaries if we need to, so it feels deferrable.

Either way, we'll probably eventually need either JWTs or a workload analog to the OIDC /userinfo endpoint for non-federated workload metadata.

And we are moving into 'why not just use JWT plus TLS as alternative to client certs, and add meta to JWT instead of cert'. Which is not bad for peers outside of Istio MC or XDS federation.

Certs are good simple identity documents, and terrible metadata stores. It makes sense to me to keep certs for identifying (used for authZ) metadata, and use a JWT (or a metadata endpoint that accepts an identity document) for non-identifying metadata (if we really ever need to shuttle baggage around as a blob).

probably with audience because otherwise they're as good as regular headers.

Meh, if they're signed by the workload cert I don't think aud helps us much but I might be wrong.

@bleggett
Copy link
Contributor

bleggett commented May 25, 2023 •
edited
Loading

Agree we probably need multiple mechanisms here. Putting peer metadata in certs is wildly impractical beyond the very very basic identity-specific fields the x509 spec dictates (which will not be sufficient for Istio's needs)

  • A peer metadata query service for cluster-local lookup makes sense since most of that info is readily available locally.
  • As @kdorosh said - we cannot force users to make cross network requests to a federated metadata service for telemetry information) so we need an alternate mechanism to propagate that metadata across boundaries.

if we have a workload metadata authority (whether strictly local or not) I think it's going to need to be replacable/composable in an equivalent way to how workload identity authorities (e.g. CA) are.

  • If I want my workload identity authority to attest more things about the workload than the default istio CA is capable of attesting, before issuing that workload an identity, I can replace the Istio CA with my own CA that does this.
  • If I want my workload metadata authority to attest more metadata about the workload than the default Istio workload metadata authority cares about, there needs to be a way to compose/append (or replace) that.

This is much simpler if the workload identity document and the workload metadata document are the same (JWT), but that's infeasible for Istio so I think we have to contend with the alternative options.

It's a little less simple if the workload identity document and workload metadata document are disjoint (workload cert + potential baggage JWT).

@costinm
Copy link
Contributor

costinm commented May 26, 2023 via email

@keithmattix keithmattix added the Ambient Beta Must have for Beta of Ambient Mesh label Jul 10, 2023
@louiscryan
Copy link
Contributor

@kyessenov

I think I'm relatively convinced that using WDS (Workload Discovery API) is the right mechanism for the majority of use-cases. I chatted a bit with @howardjohn and @costinm about this as well as @bleggett

Some basic constraints and supporting information....

  • We are already distributing workload metadata for the entire fleet visible to a single istiod instance to every ztunnel already so we are not particularly concerned with availability issues for resolving client metadata on the server side. It does not seem like we need to use on-demand loading at this time and if we have to add it later for scale reasons our mitigation approaches are viable
  • For more federated control-plane use-cases where traffic transits E-W Gateways but the remote workload information is opaque to the client we should send baggage so there needs to be a property in WDS indicating to ztunnel that it should do this. See @stevenctl recent work on this for WorkloadEntry
  • Using OTEL baggage as the encoding form seems innappropriate for this use case. Its specification indicates that its not hop-by-hop and there is no verification mechanism for the authenticity of the data. Instead it seems appropriate to use a JWT with signed claims to convey origin information. Having signed claims has the added benefit of allowing trusted claims to be used in policy decisions and not just telemetry. Discussion of whether this was possible with x509 was extensive (and likely ongoing) but this was the consensus.
  • As noted above this solution has the best performance dynamic for the majority of traffic flows (intra-cluster)
  • This solution allows the WDS and dataplanes to largely evolve independently, making it easier to enhance capabilities in either over time.

We discussed what the JWT should look like and how it should be constructed and signed. While details need to be worked out as a strawman we thought that:

  • JWT should be issued by and signed by istiod and delivered to ztunnel inside the WDS API. Something must attest to the claims it contains and istiod is an authority here
  • The JWT must be channel-bound to the x509 credential of the same workload identity. I.e When receiving the JWT the receiver must validate that it's 'sub' matches the SPIFFE ID of the SAN for the negotiated channel.
  • Since x509 certificates are subject to re-issue it is not clear that the JWTs require re-issue and so do not require an expiry.
  • The issuer should be istiod's identity and not any other account, particularly not any other workload identity so they cannot be used in other contexts to impersonate workloads.

The above needs to be put into a doc and thrashed out. It is of course unfortunate that we cannot use the certificate issuance and attestation flow to achieve the above effect but as long as we are confident in the security relationship between istiod and the apiserver this seems acceptable to layer on top.

@bleggett
Copy link
Contributor

bleggett commented Jul 14, 2023 •
edited
Loading

The JWT must be channel-bound to the x509 credential of the same workload identity. I.e When receiving the JWT the receiver must validate that it's 'sub' matches the SPIFFE ID of the SAN for the negotiated channel.

if we do this, then I don't think it matters what is used to sign the JWT (istiod cert or workload cert), and this:

The issuer should be istiod's identity and not any other account, particularly not any other workload identity so they cannot be used in other contexts to impersonate workloads.

becomes largely moot.

Unless we're just saying that impersonation is less likely with istiod because the istiod certs are "more protected" than the workload certs and therefore less susceptible to exfil - which is pretty tenuous - it will be ztunnel doing all impersonation no matter what.

If ztunnel is trusted to impersonate the workload for the purposes of constructing the channel then it arguably doesn't matter whether istiod or ztunnel sign/mint the JWT - it's a net-nil difference in impersonation risk.

But that's an impl detail, approach SGTM.

@howardjohn
Copy link
Member

JWT should be issued by and signed by istiod and delivered to ztunnel inside the WDS API. Something must attest to the claims it contains and istiod is an authority here
The JWT must be channel-bound to the x509 credential of the same workload identity. I.e When receiving the JWT the receiver must validate that it's 'sub' matches the SPIFFE ID of the SAN for the negotiated channel.

Instead of having the CA issue a cert, and Istiod issue a JWT, then linking them by sub, is there a way to somehow directly sign the JWT with the workload certificate itself?

More or less agreeing with #43937 (comment) I think.

@kyessenov kyessenov mentioned this issue Jul 14, 2023
Merged
@louiscryan
Copy link
Contributor

It depends whether we think the workload is a sufficiently reliable asserter of its claims. I think the receiving side would like some confidence that claims are trustable if policy is going to be enforced against them.

ztunnel can impersonate workloads but it can't/shouldn't be able to impersonate the credential used to sign the claims ideally. I agree this boils down to itiod's (or some CA's) credentials being better protected, we're certainly reliant on that. being the case for a CA.

We don't stritcly have to use the workload identity of istiod as the signing key, that choice is more flexible. A common secret for instance would suffice.

@howardjohn
Copy link
Member

howardjohn commented Jul 14, 2023 via email

@louiscryan
Copy link
Contributor

Capturing some related work going on for reference

Standardizing some OIDs for K8S, primarily for machine identification in kubelet but seems like it could be leveraged to improve some of the information in x509 with some effort, particularly a workload identity in addition to SA identity in the SAN. Likely insufficient to obviate the need for a JWT to capture info but worth tracking...

kubernetes/k8s.io#1959

@costinm
Copy link
Contributor

costinm commented Jul 15, 2023 via email

@bleggett bleggett mentioned this issue Jul 21, 2023
Closed
@kyessenov kyessenov mentioned this issue Aug 22, 2023
Closed
@kyessenov
Copy link
Contributor Author

cc @whitneygriffith

@costinm
Copy link
Contributor

costinm commented Oct 5, 2023 via email

@kyessenov kyessenov mentioned this issue Oct 5, 2023
Merged
@kyessenov
Copy link
Contributor Author

Refs: #47205, #47584

@stevenctl stevenctl mentioned this issue Dec 14, 2023
Open
9 tasks
@linsun
Copy link
Member

linsun commented Dec 19, 2023

@kyessenov is this already implemented?

from our prior chat:

Metadata discovery requires "ambient WDS controller" on by default. Waypoint always uses it.

If I do enable the ambient PEER_METADATA_DISCOVERY, I'll be able to use the metadata discovery service for sidecar.

@kyessenov
Copy link
Contributor Author

Yes, it can be used on sidecar as a fallback. The traditional headers will take priority.

@kyessenov kyessenov mentioned this issue Dec 21, 2023
Open
@linsun
Copy link
Member

linsun commented Jan 26, 2024

Cool, anything remaining or should we close this out?

@kyessenov
Copy link
Contributor Author

It's done as opt-in and on waypoints. Changing the defaults for Istio is difficult due to compatibility concerns, we'll need a separate issue for that.

@howardjohn howardjohn mentioned this issue Apr 4, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kyessenov kyessenov

Labels
Ambient Beta Must have for Beta of Ambient Mesh area/extensions and telemetry
Projects
Ambient tracking
Status: Done
Milestone
No milestone
Development

No branches or pull requests
10 participants
@costinm @howardjohn @bleggett @keithmattix @linsun @zirain @louiscryan @kyessenov @kdorosh @lei-tang