Agent to server communication through an egress proxy

[jieyu]

Currently, the agent to server communication is based on gRPC streams. There're cases where an agent's egress has to go through an egress proxy. In many cases, the egress proxy does not support gRPC protocol directly.

Is there a recommended way to solve this problem? For example, has anyone tried tunnel the grpc http/2 traffic through an HTTP CONNECT based proxy? Is that a supported model?

[cheftako]

Several people have floated the idea of having an alternate to gRPC for Agent <-> Server communication. It seems clear some refactor will need to be done to make that happen. (It certainly took some of Client <-> Server communication). While I think everyone on the project is in support of making that work, it will require someone who actually has a use case to drive it.

[jieyu]

Looks like gRPC already supports HTTP CONNECT tcp tunnel (most proxies support that):
https://github.com/grpc/grpc-go/blob/master/proxy.go#L140
https://github.com/grpc/proposal/blob/master/A1-http-connect-proxy-support.md

I tested this by launching a squid proxy using docker

docker run -v $(pwd)/squid.conf:/etc/squid/squid.conf -p 3128:3128 --rm -ti sameersbn/squid:3.5.27-2

squid.conf

http_port 3128
http_access allow all

Launch the proxy server

./bin/proxy-server --mode=http-connect --server-ca-cert=certs/master/issued/ca.crt --server-cert=certs/master/issued/proxy-master.crt --server-key=certs/master/private/proxy-master.key --cluster-ca-cert=certs/agent/issued/ca.crt --cluster-cert=certs/agent/issued/proxy-master.crt --cluster-key=certs/agent/private/proxy-master.key

Launch the proxy agent by setting HTTPS proxy env vars.

HTTPS_PROXY=127.0.0.1:3128 HTTP_PROXY=127.0.0.1:3128 ./bin/proxy-agent --ca-cert=certs/agent/issued/ca.crt --agent-cert=certs/agent/issued/proxy-agent.crt --agent-key=certs/agent/private/proxy-agent.key --proxy-server-host=10.0.1.120

(Note that you cannot use 127.0.0.1 for proxy-server-host as golang will default to not use proxy for localhost)

So this works as expected.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[cheftako]

/lifecycle frozen

[cheftako]

I believe this is being looked into with KEP kubernetes/enhancements#2025