OCPBUGS-28637: Update cluster-wide proxy Risks and Mitigations for Windows nodes. #1610

jrvaldes · 2024-04-22T13:06:21Z

This change updates the Risks and Mitigations section in the cluster-wide proxy enhancements to set expectations on the scope of the operator's responsibilities alongside recommendations to enable traffic through proxy by default for Powershell sessions.

openshift-ci-robot · 2024-04-22T13:06:26Z

@jrvaldes: This pull request references Jira Issue OCPBUGS-28637, which is invalid:

expected the bug to target the "4.16.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

This change updates the Risks and Mitigations section in the cluster-wide proxy enhancements to set expectations on the scope of the operator's responsibilities alongside recommendations to enable traffic through proxy by default for Powershell sessions.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

jrvaldes · 2024-04-22T13:10:05Z

/jira refresh

openshift-ci-robot · 2024-04-22T13:10:12Z

@jrvaldes: This pull request references Jira Issue OCPBUGS-28637, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.16.0) matches configured target version for branch (4.16.0)
bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @rrasouli

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

sebsoto · 2024-04-22T13:51:03Z

enhancements/windows-containers/windows-node-egress-proxy.md

 * Maintain normal functionality in non-proxied clusters

 ### Non-Goals

 * First-class support/enablement of proxy utilization for user-provided applications
 * *ingress* and reverse proxy settings are out of scope
 * Monitor cert expiration dates or automatically replace expired CAs in the cluster's trust bundle
+* Windows workloads created in Windows nodes with cluster-wide proxy enabled do not inherit proxy settings from the 
+  node. This is default behavior on Linux Nodes side as well.


This is the default behavior for Linux Nodes as well.

sebsoto · 2024-04-22T13:51:42Z

enhancements/windows-containers/windows-node-egress-proxy.md

@@ -110,6 +116,24 @@ Although cluster infra resources already do a best effort validation on the user
 a user could provide non-functional proxy settings/certs. This would be propagated to their Windows nodes and workloads,
 taking down existing application connectivity and preventing new Windows nodes from being bootstrapped.

+To enable global HTTP proxy by default on Windows nodes with cluster-wide proxy for all PowerShell's sessions, 


This is doing a very good job at sharing the mitigation, but not the risk.
For what purpose would they do this? Please add if this has any impact on the functionality as a Node in the cluster.

Not following here.

This is a mitigation for a non-goal, and the impact is indeed the traffic of the Powershell session going via proxy. The latter does not introduce any additional risk.

@jrvaldes this section is for risk and mitigation, the risk needs to be explained first followed by its mitigation.

I think the mitigation implies the risk but it might be useful to call it out directly:

If users use Windows images with PowerShell as the default shell, then there is a risk that outbound traffic from the PowerShell CLI wouldn't go through the cluster-wide proxy by default. Although this does not affect WMCO/OpenShift's view of the Node, this is different on-instance behavior than an admin would see if they were using CMD Prompt. To mitigate this...

saifshaikh48 · 2024-04-22T16:10:36Z

enhancements/windows-containers/windows-node-egress-proxy.md

+Powershell sessions that reads the proxy environment variables maintained by WMCO and populate the `DefaultWebProxy` property:
+```powershell
+    Set-Content -Path $PROFILE.AllUsersCurrentHost -Value '$proxyValue=[Environment]::GetEnvironmentVariable("HTTP_PROXY", "Process")' -Force
+    Add-Content -Path $PROFILE.AllUsersCurrentHost -Value '[System.Net.WebRequest]::DefaultWebProxy = New-Object System.Net.Webproxy("$proxyValue")' -Force


if we also want to provide powershell guidance for HTTPS/custom certs, we can point to using this constructor when creating the System.Net.Webproxy object

mansikulkarni96 · 2024-04-22T16:43:33Z

enhancements/windows-containers/windows-node-egress-proxy.md

+where `PROXY_URL` is the URI of the cluster-wide proxy.
+
+Run the following commands in the Windows Node with cluster-wide proxy enabled to create a default profile for the 
+Powershell sessions that reads the proxy environment variables maintained by WMCO and populate the `DefaultWebProxy` property:


We should mention the example commands below are only for HTTP_PROXY and this needs to be done for all proxy variables.

mansikulkarni96 · 2024-04-22T16:46:26Z

enhancements/windows-containers/windows-node-egress-proxy.md

@@ -59,13 +59,19 @@ requests to off-cluster services, and those that use a
  + [Additional certificate authorities](https://docs.openshift.com/container-platform/4.12//rest_api/config_apis/proxy-config-openshift-io-v1.html#spec-trustedca) required to validate the proxy's certificate
 * Configure the proxy settings in WMCO-managed components on Windows nodes (kubelet, containerd runtime)
 * React to changes to the cluster-wide proxy settings during WMCO runtime
+* Synchronize environment variables `NO_PROXY`, `HTTP_PROXY`, and `HTTPS_PROXY` on Windows nodes with cluster-wide 
+  proxy settings in both shells _CMD_ and _PowerShell_


I feel that this point conflicts with the PowerShell non-goals: PowerShell's shell sessions do not inherit proxy settings by default on Windows nodes. Additionally, do we specifically do anything for it to be synchronized in both shells?

Re-worded it for more clarity, but it does not. This refers to the environment variables.

The PowerShell non-goals, cover enabling proxy by default in Powershell session, where the environment variables should exist, as per goals.

mansikulkarni96 · 2024-04-24T15:29:59Z

enhancements/windows-containers/windows-node-egress-proxy.md

+* Windows workloads created in Windows nodes with cluster-wide proxy enabled do not inherit proxy settings from the 
+  node. This is the default behavior on Linux Nodes side as well.
+* PowerShell's shell sessions do not inherit proxy settings by default on Windows nodes with cluster-wide proxy enabled 
+  (See [Risks and Mitigations](#risks-and-mitigations) section for recommendations) 


This link is broken for me, can you double-check?

@mansikulkarni96 this works fine locally and I also tested it in https://github.com/jrvaldes/openshift-enhancements/blob/wmco-proxy-update/enhancements/windows-containers/windows-node-egress-proxy.md

mansikulkarni96 · 2024-04-24T16:23:25Z

/approve

jrvaldes · 2024-04-24T16:29:17Z

/cc @mburke5678 @rrasouli

for docs and QE

jrvaldes · 2024-04-24T16:44:04Z

/cc @mandre

saifshaikh48

/lgtm

mburke5678 · 2024-04-24T17:04:18Z

/lgtm

sebsoto · 2024-04-24T18:24:28Z

enhancements/windows-containers/windows-node-egress-proxy.md

@@ -110,6 +116,29 @@ Although cluster infra resources already do a best effort validation on the user
 a user could provide non-functional proxy settings/certs. This would be propagated to their Windows nodes and workloads,
 taking down existing application connectivity and preventing new Windows nodes from being bootstrapped.

+In case users use Windows images with PowerShell as the default shell, then there is a risk that outbound traffic 


Please clarify if Windows images refers to a VM image, or container image

Good catch, I replaced images with nodes to avoid confusion as the proxy configuration for Windows Workloads is out of the scope for this.

sebsoto · 2024-04-24T18:25:26Z

LGTM other than the above comment

This change updates the Risks and Mitigations section in the cluster-wide proxy enhancements to set expectations on the scope of the operator's responsibilities alongside recommendations to enable traffic through proxy by default for Powershell sessions.

sebsoto · 2024-04-24T19:33:14Z

/lgtm

mandre · 2024-04-25T05:55:11Z

Sorry, I'm not the most qualified when it comes to windows containers. Would it be possible to add your team lead (@sebsoto ?) as an approver of this repo?
/unassign mandre

jrvaldes · 2024-04-25T12:01:37Z

Sorry, I'm not the most qualified when it comes to windows containers. Would it be possible to add your team lead (@sebsoto ?) as an approver of this repo? /unassign mandre

@mandre Thanks for the recommendation; I think that is a good idea.

In the interim, the team lead LGTM'ed already. Could that suffice for you to \approve?

jrvaldes · 2024-04-25T12:16:06Z

/cc @mrunalp

openshift-ci · 2024-04-25T20:30:54Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mansikulkarni96, mrunalp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [mrunalp]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci · 2024-04-25T20:44:42Z

@jrvaldes: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

openshift-ci-robot · 2024-04-25T20:49:24Z

@jrvaldes: Jira Issue OCPBUGS-28637: All pull requests linked via external trackers have merged:

openshift/enhancements#1610

Jira Issue OCPBUGS-28637 has been moved to the MODIFIED state.

In response to this:

This change updates the Risks and Mitigations section in the cluster-wide proxy enhancements to set expectations on the scope of the operator's responsibilities alongside recommendations to enable traffic through proxy by default for Powershell sessions.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Apr 22, 2024

openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Apr 22, 2024

openshift-ci bot requested review from rrasouli, jmguzik and LalatenduMohanty April 22, 2024 13:10

sebsoto suggested changes Apr 22, 2024

View reviewed changes

saifshaikh48 reviewed Apr 22, 2024

View reviewed changes

mansikulkarni96 reviewed Apr 22, 2024

View reviewed changes

jrvaldes force-pushed the wmco-proxy-update branch from b538811 to 1b5e9ab Compare April 22, 2024 17:16

jrvaldes requested review from sebsoto, mansikulkarni96 and saifshaikh48 April 23, 2024 02:06

mburke5678 mentioned this pull request Apr 23, 2024

Add details on WMCO and cluster-wide proxy openshift/openshift-docs#75056

Open

jrvaldes force-pushed the wmco-proxy-update branch from 1b5e9ab to 249dde6 Compare April 24, 2024 14:34

mansikulkarni96 reviewed Apr 24, 2024

View reviewed changes

openshift-ci bot requested a review from mandre April 24, 2024 16:44

saifshaikh48 reviewed Apr 24, 2024

View reviewed changes

openshift-ci bot assigned saifshaikh48 Apr 24, 2024

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Apr 24, 2024

openshift-ci bot assigned mburke5678 Apr 24, 2024

sebsoto reviewed Apr 24, 2024

View reviewed changes

jrvaldes force-pushed the wmco-proxy-update branch from 249dde6 to 3dad8c7 Compare April 24, 2024 18:55

openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label Apr 24, 2024

openshift-ci bot assigned sebsoto Apr 24, 2024

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Apr 24, 2024

openshift-ci bot requested a review from mrunalp April 25, 2024 12:16

mrunalp approved these changes Apr 25, 2024

View reviewed changes

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 25, 2024

openshift-merge-bot bot merged commit 70969ab into openshift:master Apr 25, 2024
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OCPBUGS-28637: Update cluster-wide proxy Risks and Mitigations for Windows nodes. #1610

OCPBUGS-28637: Update cluster-wide proxy Risks and Mitigations for Windows nodes. #1610

jrvaldes commented Apr 22, 2024

openshift-ci-robot commented Apr 22, 2024

jrvaldes commented Apr 22, 2024

openshift-ci-robot commented Apr 22, 2024

sebsoto Apr 22, 2024

jrvaldes Apr 22, 2024

sebsoto Apr 22, 2024

jrvaldes Apr 22, 2024

mansikulkarni96 Apr 23, 2024

saifshaikh48 Apr 23, 2024

jrvaldes Apr 24, 2024

saifshaikh48 Apr 22, 2024

jrvaldes Apr 22, 2024

mansikulkarni96 Apr 22, 2024

mansikulkarni96 Apr 22, 2024

jrvaldes Apr 22, 2024

mansikulkarni96 Apr 24, 2024

jrvaldes Apr 24, 2024

mansikulkarni96 commented Apr 24, 2024

jrvaldes commented Apr 24, 2024 •

edited

jrvaldes commented Apr 24, 2024

saifshaikh48 left a comment

mburke5678 commented Apr 24, 2024

sebsoto Apr 24, 2024

jrvaldes Apr 24, 2024

sebsoto commented Apr 24, 2024

sebsoto commented Apr 24, 2024

mandre commented Apr 25, 2024

jrvaldes commented Apr 25, 2024 •

edited

jrvaldes commented Apr 25, 2024

openshift-ci bot commented Apr 25, 2024

openshift-ci bot commented Apr 25, 2024

openshift-ci-robot commented Apr 25, 2024

OCPBUGS-28637: Update cluster-wide proxy Risks and Mitigations for Windows nodes. #1610

OCPBUGS-28637: Update cluster-wide proxy Risks and Mitigations for Windows nodes. #1610

Conversation

jrvaldes commented Apr 22, 2024

openshift-ci-robot commented Apr 22, 2024

jrvaldes commented Apr 22, 2024

openshift-ci-robot commented Apr 22, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mansikulkarni96 commented Apr 24, 2024

jrvaldes commented Apr 24, 2024 • edited

jrvaldes commented Apr 24, 2024

saifshaikh48 left a comment

Choose a reason for hiding this comment

mburke5678 commented Apr 24, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sebsoto commented Apr 24, 2024

sebsoto commented Apr 24, 2024

mandre commented Apr 25, 2024

jrvaldes commented Apr 25, 2024 • edited

jrvaldes commented Apr 25, 2024

openshift-ci bot commented Apr 25, 2024

openshift-ci bot commented Apr 25, 2024

openshift-ci-robot commented Apr 25, 2024

jrvaldes commented Apr 24, 2024 •

edited

jrvaldes commented Apr 25, 2024 •

edited