#2857: Add Request UID to JSON log enricher to correlate container and API Server Audit log

[ngopalak-redhat]

What type of PR is this?

/kind feature

What this PR does / why we need it:

This PR addresses an issue with JSON Log Enricher: the inability to correlate in logs generated from kubectl exec sessions with the API Server Audit Logs. This information is required to find the end-user who executed a command on the pod/container.

Problem
Kubernetes, by default, does not pass user authentication details and request details (request UID) from the API server into the container's environment when a user runs kubectl exec. This means JSON logs for exec sessions lack context about who initiated the command. When there are multiple users doing kubectl exec requests and the audit log for the API Server is enabled, its not possible to correlate the server side audit log and the container audit log.

Solution
This PR introduces a mutating admission webhook designed to inject the request ID directly into the environment variables of containers targeted by kubectl exec requests. Sufficient checks are placed to ensure that the user cannot override them. The request ID is also added as a Audit annotation of the API server audit log.

How it Works

When a user initiates an kubectl exec command, the request is intercepted by the new mutating webhook (execmetadata.spo.io).
The webhook adds environment variable (EXEC_REQUEST_UID) to the exec command. This environment variables is then available to the process running inside the container (e.g., JSON Log Enricher), accessible via /proc/pid/environ.
JSON Log Enricher is updated to read and incorporate this injected request ID into the log lines.

Which issue(s) this PR fixes:

Fixes #2857

Does this PR have test?

Yes. Added e2e and unit tests

Special notes for your reviewer:

Initial PR was raised to inject the User information. Since there are concerns of PII now only the request UID is injected. Also made the webhook optional.
@haircommander / @ccojocar Sorry for the change in the title. It was required to provide clarity on exact change.

Does this PR introduce a user-facing change?

Add Request UID to JSON log enricher to correlate container and API Server Audit log

[k8s-ci-robot]

Hi @ngopalak-redhat. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 57.71812% with 126 lines in your changes missing coverage. Please review.

Project coverage is 24.72%. Comparing base (11d77f4) to head (5befed7).
Report is 881 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #2878       +/-   ##
===========================================
- Coverage   45.50%   24.72%   -20.78%     
===========================================
  Files          79      120       +41     
  Lines        7782    20733    +12951     
===========================================
+ Hits         3541     5126     +1585     
- Misses       4099    15353    +11254     
- Partials      142      254      +112     

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ngopalak-redhat]

/assign @haircommander @ccojocar

[ccojocar]

/ok-to-test

[ccojocar]

In which context is this feature useful? Isn't the binary enough, and maybe container ID?

I am not sure if we should add an additional webhook into the deployment just to inject some additional user information into the logs.

[ngopalak-redhat]

Thanks for reworking this implementation. I just left a few minor comments. It would be nice if you could address them before we merge this.

Thanks for the review @ccojocar . I have addressed the comments.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ccojocar, ngopalak-redhat

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [ccojocar]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ccojocar]

This needs a rebase. I am not sure if it is ready to be merged? Thanks

[ngopalak-redhat]

This needs a rebase. I am not sure if it is ready to be merged? Thanks

Thanks @ccojocar I have done the rebase. I'm waiting for @haircommander to unhold to the merge.

[ccojocar]

/lgtm

[haircommander]

I think you need a rebase @ngopalak-redhat but
/unhold

[haircommander]

/lgtm