Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proxy configuration for Image Pull only #2052

Closed
thomasdanan opened this issue Nov 18, 2019 · 1 comment · Fixed by #2071 or #2201
Closed

Proxy configuration for Image Pull only #2052

thomasdanan opened this issue Nov 18, 2019 · 1 comment · Fixed by #2071 or #2201
Assignees
Labels
priority:urgent Any issue we should jump in as soon as possible topic:deployment Bugs in or enhancements to deployment stages topic:networking Networking-related issues topic:security Security-related issues

Comments

@thomasdanan
Copy link
Contributor

Component:

'containerd'

Why this is needed:

In some customer environments, cluster nodes are not connected to the internet. When it comes to download images from the internet, we usually use a proxy, which is external to the cluster. We suggested in our documentation to add proxy configuration in /etc/environment on all hosts so that 'containerd' pulls image through the proxy. The issue with that approach is that all pods running on the host will also try to access other services through this proxy and it will fail unless you configure every endpoint, which is not what we want.

What should be done:

We would like to configure the system so that a proxy can be used, only for pulling images but without affecting the traffic between pods.

Implementation proposal (strongly recommended):

Test plan:

@thomasdanan thomasdanan added topic:security Security-related issues topic:networking Networking-related issues topic:deployment Bugs in or enhancements to deployment stages moonshot labels Nov 18, 2019
@thomasdanan thomasdanan added this to the MetalK8s 2.4.2 milestone Nov 18, 2019
@thomasdanan thomasdanan added the priority:urgent Any issue we should jump in as soon as possible label Nov 19, 2019
@thomasdanan thomasdanan added this to To do in Week 47/2019 via automation Nov 19, 2019
@alexandre-allard alexandre-allard self-assigned this Nov 22, 2019
@thomasdanan thomasdanan moved this from To do to In progress in Week 47/2019 Nov 22, 2019
alexandre-allard added a commit that referenced this issue Nov 22, 2019
Set NO_PROXY environment variable with
control, workload plane and K8s internal
networks in containerd systemd unit file,
to avoid using system wide defined HTTP(S)
proxy, if any, when trying to pull resources
from metalk8s registry.

Refs: #2052
alexandre-allard added a commit that referenced this issue Nov 22, 2019
Set NO_PROXY environment variable with
control, workload plane and K8s internal
networks in containerd systemd unit file,
to avoid using system wide defined HTTP(S)
proxy, if any, when trying to pull resources
from metalk8s registry.

Refs: #2052
alexandre-allard added a commit that referenced this issue Nov 22, 2019
Set NO_PROXY environment variable with
control, workload plane and K8s internal
networks in containerd systemd unit file,
to avoid using system wide defined HTTP(S)
proxy, if any, when trying to pull resources
from metalk8s registry.

Refs: #2052
alexandre-allard added a commit that referenced this issue Nov 25, 2019
Set NO_PROXY environment variable with
control, workload plane and K8s internal
networks in containerd systemd unit file,
to avoid using system wide defined HTTP(S)
proxy, if any, when trying to pull resources
from metalk8s registry.

Refs: #2052
@thomasdanan thomasdanan added this to To do in Week 48/2019 via automation Nov 25, 2019
@thomasdanan thomasdanan moved this from To do to In progress in Week 48/2019 Nov 25, 2019
@gdemonet gdemonet added this to In progress in Week 49/2019 Dec 2, 2019
alexandre-allard added a commit that referenced this issue Dec 6, 2019
We need to disable proxy usage on metalk8s
internal repositories, otherwise we can't reach
them if any http(s)_proxy variable is set.

Refs: #2052
alexandre-allard added a commit that referenced this issue Dec 6, 2019
Use the /etc/environment file in containerd systemd
unit to inherit from proxy variables set inside

Refs: #2052
alexandre-allard added a commit that referenced this issue Dec 6, 2019
Set NO_PROXY environment variable with
control, workload plane and K8s internal
networks in containerd systemd unit file,
to avoid using system wide defined HTTP(S)
proxy, if any, when trying to pull resources
from metalk8s registry.

Refs: #2052
alexandre-allard added a commit that referenced this issue Dec 6, 2019
Set HTTP_PROXY, HTTPS_PROXY and NO_PROXY
environment variables in containerd systemd unit file,

Refs: #2052
alexandre-allard added a commit that referenced this issue Dec 9, 2019
Move the documentation for proxies to the bootstrap
configuration section and update it with the
new way to setup proxies.

Refs: #2052
alexandre-allard added a commit that referenced this issue Dec 9, 2019
Move the documentation for proxies to the bootstrap
configuration section and update it with the
new way to setup proxies.

Refs: #2052
alexandre-allard added a commit that referenced this issue Dec 9, 2019
Move the documentation for proxies to the bootstrap
configuration section and update it with the
new way to setup proxies.

Refs: #2052
@gdemonet gdemonet added this to In progress in Week 50/2019 Dec 10, 2019
alexandre-allard added a commit that referenced this issue Dec 10, 2019
We need to disable proxy usage on metalk8s
internal repositories, otherwise we can't reach
them if any http(s)_proxy variable is set.

Refs: #2052
alexandre-allard added a commit that referenced this issue Dec 10, 2019
Set HTTP_PROXY, HTTPS_PROXY and NO_PROXY
environment variables in containerd systemd unit file,

Refs: #2052
alexandre-allard added a commit that referenced this issue Dec 10, 2019
Move the documentation for proxies to the bootstrap
configuration section and update it with the
new way to setup proxies.

Refs: #2052
alexandre-allard added a commit that referenced this issue Dec 10, 2019
Set HTTP_PROXY, HTTPS_PROXY and NO_PROXY
environment variables in containerd systemd unit file,

Refs: #2052
alexandre-allard added a commit that referenced this issue Dec 10, 2019
Move the documentation for proxies to the bootstrap
configuration section and update it with the
new way to setup proxies.

Refs: #2052
@alexandre-allard
Copy link
Contributor

This issue can be workaround by appending the following in /etc/systemd/system/containerd.service.d/50-metalk8s.conf:

Environment="HTTP_PROXY=http://proxy-ip:proxy-port HTTPS_PROXY=https://proxy-ip:proxy-port NO_PROXY=<metalk8s networks>"

Where <metalk8s networks> is the comma-separated list of both kubernetes internal networks and hosts networks.
To retrieve this list, this command can be run, on one of the metalk8s host:

salt-call slsutil.renderer string=’{% from "metalk8s/map.jinja" import networks with context %}{{ networks.values() | join(",") }}’ saltenv=metalk8s-<metalk8s-version>

<metalk8s-version> must be replaced by the version currently installed (e.g. 2.4.1)

alexandre-allard added a commit that referenced this issue Dec 13, 2019
Set HTTP_PROXY, HTTPS_PROXY and NO_PROXY
environment variables in containerd systemd unit file,

Refs: #2052
alexandre-allard added a commit that referenced this issue Dec 13, 2019
Move the documentation for proxies to the bootstrap
configuration section and update it with the
new way to setup proxies.

Refs: #2052
alexandre-allard added a commit that referenced this issue Dec 17, 2019
We need to disable proxy usage on metalk8s
internal repositories, otherwise we can't reach
them if any http(s)_proxy variable is set.

Refs: #2052
alexandre-allard added a commit that referenced this issue Dec 17, 2019
Set HTTP_PROXY, HTTPS_PROXY and NO_PROXY
environment variables in containerd systemd unit file,

Refs: #2052
alexandre-allard added a commit that referenced this issue Dec 17, 2019
Move the documentation for proxies to the bootstrap
configuration section and update it with the
new way to setup proxies.

Refs: #2052
@thomasdanan thomasdanan added this to In progress in Week 51/2019 Dec 17, 2019
alexandre-allard added a commit that referenced this issue Dec 18, 2019
We need to disable proxy usage on metalk8s
internal repositories, otherwise we can't reach
them if any http(s)_proxy variable is set.

Refs: #2052
alexandre-allard added a commit that referenced this issue Dec 18, 2019
Set HTTP_PROXY, HTTPS_PROXY and NO_PROXY
environment variables in containerd systemd unit file,

Refs: #2052
alexandre-allard added a commit that referenced this issue Dec 18, 2019
Move the documentation for proxies to the bootstrap
configuration section and update it with the
new way to setup proxies.

Refs: #2052
alexandre-allard added a commit that referenced this issue Dec 18, 2019
Move the documentation for proxies to the bootstrap
configuration section and update it with the
new way to setup proxies.

Refs: #2052
@gdemonet gdemonet added this to In progress in Week 52/2019 Dec 31, 2019
alexandre-allard added a commit that referenced this issue Jan 3, 2020
We need to disable proxy usage on metalk8s
internal repositories, otherwise we can't reach
them if any http(s)_proxy variable is set.

Refs: #2052
alexandre-allard added a commit that referenced this issue Jan 3, 2020
Set HTTP_PROXY, HTTPS_PROXY and NO_PROXY
environment variables in containerd systemd unit file,

Refs: #2052
alexandre-allard added a commit that referenced this issue Jan 3, 2020
Move the documentation for proxies to the bootstrap
configuration section and update it with the
new way to setup proxies.

Refs: #2052
@gdemonet gdemonet added this to In progress in Week 02/2020 Jan 6, 2020
Week 51/2019 automation moved this from In progress to Done Jan 6, 2020
Week 52/2019 automation moved this from In progress to Done Jan 6, 2020
Week 02/2020 automation moved this from In progress to Done Jan 6, 2020
alexandre-allard added a commit that referenced this issue Jan 22, 2020
For the sake of clarity, we set 1 env variable
per line, instead of all the variables on a single
`Environment` entry in the containerd unit file

Refs: #2052
alexandre-allard added a commit that referenced this issue Jan 22, 2020
For the sake of clarity, we set 1 env variable
per line, instead of all the variables on a single
`Environment` entry in the containerd unit file

Refs: #2052
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority:urgent Any issue we should jump in as soon as possible topic:deployment Bugs in or enhancements to deployment stages topic:networking Networking-related issues topic:security Security-related issues
Projects
No open projects
Week 02/2020
  
Done
Week 47/2019
  
In progress
Week 48/2019
  
In progress
Week 49/2019
  
In progress
Week 50/2019
  
In progress
Week 51/2019
  
Done
Week 52/2019
  
Done
2 participants