Identify the server in audit webhook

[irfanurrehman]

Issue by crassirostris
Thursday Aug 03, 2017 at 09:14 GMT
Originally opened as kubernetes/kubernetes#50076

---

Spinning up a discussion from kubernetes/kubernetes#48561

It would be nice to identify the server that sent the audit event, i.e. the aggregator vs. an end-user apiserver.

It can be implemented right now by adding a parameter to the webhook address on the deployment stage and process this parameter later in the pipeline. However, it's (relatively) easy to add information to the API and we want to make sure that the information we currently have is in the right state before going to beta / GA.

One thought is that it probably doesn't make sense to have this information in the event object, because events from different sources rarely mix up (until the backend anyway) and it would just mean the same information copied all over the request to the webhook. WDYT about adding this information to the EventList object?

/cc @tallclair @sttts @CaoShuFeng @ericchiang

[irfanurrehman]

Comment by sttts
Thursday Aug 10, 2017 at 17:39 GMT

---

/cc @nikhiljindal

[irfanurrehman]

Comment by sttts
Thursday Aug 10, 2017 at 17:40 GMT

---

There were some discussions months ago around the idea to connect federation requests with requests "forwarded" to the kube-apiservers. If I understand correctly (@nikhiljindal knows more), this forwarding is done via a controller, i.e. asynchronously? This means our audit header does not help so much right now.

[irfanurrehman]

Comment by nikhiljindal
Wednesday Aug 16, 2017 at 19:08 GMT

---

Umm am sorry am a bit lost. Not sure which Audit header is being talked about here.

I can provide context on the discussion I had with @sttts:

When a federated kubernetes cluster receives a request from federation control plane today, the audit logs say that the request came from "federation-control-plane" service account. We do not get any information on which user triggered that request. (For ex: it was Bob that created a federation deployment, which resulted in federation control plane creating a deployment in federated clusters).

We have discussed a few options:

• Federation control plane should use Bob's identity to send the request to federated clusters (Federation per-user cluster authentication, authorization and audit logging kubernetes/kubernetes#35254)
• Federation request can include a "on behalf of" header that indicates that it is the federation control plane sending the request on behalf of Bob.
• Federation request can include the audit ID from federation-apiserver for Bob's request to create the deployment. So that anyone looking at kube-apiserver audit logs has enough information to track the request back to Bob's request to federation-apiserver.

And yes the "forwarding" is done via controllers i.e. asynchronously.

cc @kubernetes/sig-federation-misc

[irfanurrehman]

Comment by crassirostris
Thursday Aug 17, 2017 at 08:42 GMT

---

As far as I can see, there are actually two separate problems:

• Correlating audit events from the federation apiserver and the cluster apiserver
• Distinguishing the audit logs that came from the cluster apiserver, federation apiserver, kube-aggregator, etc.

If we're talking about the first one, in the original proposal it's suggested to have the same ID for the audit events that belong to the same requests and pass it around in the http header. But AFAIU, there's a problem with this approach and the way federated server does requests currently.

[irfanurrehman]

Comment by ericchiang
Thursday Aug 17, 2017 at 18:12 GMT

---

Correlating audit events from the federation apiserver and the cluster apiserver

I think this would be like trying to correlate a request to create a replicaset with the follow up request to create a pod. This seems out of scope for the audit log, and should probably be addressed by annotations or ownerRef equivalent, right?

[irfanurrehman]

Comment by crassirostris
Friday Aug 18, 2017 at 15:49 GMT

---

I think this would be like trying to correlate a request to create a replicaset with the follow up request to create a pod. This seems out of scope for the audit log, and should probably be addressed by annotations or ownerRef equivalent, right?

No, that's a different thing. Imagine we have N events in the backend with the same AuditID, for the same call essentially. Some of these events were ingested from the apiserver, some from the kube-aggregator, some from federated server. How to understand which server this particular event with the same AuditID came from?

[irfanurrehman]

Comment by ericchiang
Friday Aug 18, 2017 at 16:29 GMT

---

The federated API server doesn't behave like an aggregated API server where the request is proxied and the request will have the same AuditID. In federation, a user creates a object through the federated API server, the federated controllers observe the object, then reconciles objects on the underlying clusters.

An individual request to the federated API server doesn't directly translate to a individual request to the underlying clusters. There's no "same call." So I think some sort higher level method of correlation is probably better for that use case.

(also sorry for closing the issue accidentally)

[irfanurrehman]

Comment by crassirostris
Sunday Aug 20, 2017 at 20:38 GMT

---

@ericchiang OK, I see, thanks. I was thinking more about the kube-aggregator case and though that some calls in the federated server are synchronous

[irfanurrehman]

Comment by crassirostris
Wednesday Oct 25, 2017 at 10:38 GMT

---

WDYT about something like "audit context" -- object ref that would be populated by asynchronous servers like federation control plane that would specify, what object caused the request to be fired?

It could then be used in other controllers, e.g. DaemonSet controller could specify the DaemonSet object ref in the request to create/delete pods

[irfanurrehman]

Comment by fejta-bot
Tuesday Jan 23, 2018 at 11:12 GMT

---

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

[irfanurrehman]

Comment by sttts
Thursday Jan 25, 2018 at 16:17 GMT

---

/lifecycle frozen

[irfanurrehman]

cc @crassirostris