-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Websocket requests are not proxied correctly in Ansible-based operators #2204
Comments
Note that using - name: Test kubectl exec.
command: >
kubectl exec -n {{ meta.namespace }} {{ tower_pod_name }} date
environment:
KUBECONFIG: '{{ lookup("env", "KUBECONFIG") }}' |
Hi @geerlingguy, Before any further analyse and check could you please check it with the latest version of SDK. I mean, could you upgrade your project to use SDK 0.12? Or let us know if you are able to reproduce this scenario using the Memcached sample? Also, I checked that:
So, note that the python version was upgrade to 3. Please, could you ensure that your project was upgraded properly and you are using python 3 in the env where it has been executed? |
@camilamacedo86 - Thanks for the suggestion! I'll definitely upgrade and test things (see linked issue above)—I hope to get to this soon. |
@camilamacedo86 - I just reproduced the same error on v0.12.0, as well as the current latest version, v0.14.0. Steps to reproduce (requires Molecule, Ansible, and Minikube installed locally):
The operator playbook runs but keeps failing at the
I was speaking with @fabianvf on Slack and he mentioned that the likely problem is the Ansible Operator HTTP proxy that is injected between Kubernetes' API and the operator itself is not handling websockets requests correctly (thus we get this error with the 200 OK handshake—it should be continuing on and streaming the response to Python, which it is not). |
If found this issue upstream in the client-go/rest package: kubernetes/client-go#45 — it seems that issue went stale and was automatically closed. There's a 2017 blog post linked with a workaround: Writing a Custom Kubectl Exec Command, and there's an HTTPWrappersForConfig function that is "exposed to allow more clients that need HTTP-like behavior but then must hijack the underlying connection (like WebSocket or HTTP2 clients)." It would be nice if we could make proxy.go (https://github.com/operator-framework/operator-sdk/blob/master/pkg/ansible/proxy/proxy.go) work with websockets, and also add a test case that uses |
#2716 looks like it resolves this issue 🎉 |
Bug Report
What did you do?
I am trying to an Ansible
k8s_exec
module, which allows running the equivalent ofkubectl exec
commands to exec a command on a Pod via Ansible through the Python Kubernetes library. This allows me to write a task like:Instead of installing
kubectl
on my operator image (addedCOPY --from=lachlanevenson/k8s-kubectl:v1.16.2 /usr/local/bin/kubectl /usr/local/bin/kubectl
to mybuild/Dockerfile
) and writing a task like:What did you expect to see?
When I run the same task as above on my system Ansible against a Kubernetes cluster, or even inside of the operator Pod's ansible container using
ansible-playbook
to run it, it executes successfully and registers the result of the command that was executed.What did you see instead? Under which circumstances?
When it is run via the operator/
ansible-runner
using the operator's proxy, it results in the following error:It should be getting a
101
response from the Kubernetes API websocket.Full error message from the failed task:
Environment
v0.11.0
N/A
1.16.2
Molecule
ansible
Possible Solution
N/A
Additional context
Relates to: geerlingguy/tower-operator#5
The text was updated successfully, but these errors were encountered: