Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to use `oc` when api-resources are secured #1139

Closed
bartoszmajsak opened this issue Oct 3, 2019 · 6 comments · Fixed by #1143

Comments

@bartoszmajsak
Copy link
Contributor

@bartoszmajsak bartoszmajsak commented Oct 3, 2019

Currently, in order to determine if the target cluster is vanilla k8s or openshift, telepresence calls ${SERVER_URL}/apis.

def kubectl_or_oc(server: str) -> str:
"""
Return "kubectl" or "oc", the command-line tool we should use.
:param server: The URL of the cluster API server.
"""
if which("oc") is None:
return "kubectl"
# We've got oc, and possibly kubectl as well. We only want oc for OpenShift
# servers, so check for an OpenShift API endpoint:
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
try:
with urlopen(server + "/apis", context=ctx) as response:
api_group_list = str(response.read())
except (URLError, HTTPError):
return "kubectl"
if "openshift" in api_group_list:
return "oc"
else:
return "kubectl"

If this endpoint is protected, OpenShift will return 403 and Telepresence will assume it's dealing with vanilla k8s. Therefore it will use Deployment instead of DeploymentConfig.

This leads to errors while using telepresence in case of the latter, e.g.:

T: Starting network proxy to cluster using the existing proxy Deployment ratings-v1-v1-circleci-jufar

T: Failed to find deployment ratings-v1-v1-circleci-jufar:
T: Error from server (NotFound): deployments.extensions "ratings-v1-v1-circleci-jufar" not found

T: Exit cleanup in progress

Expected to use DeploymentConfig instead.

telepresence.log
   0.0 TEL | Telepresence 0.101 launched at Wed Oct  2 15:37:34 2019
   0.0 TEL |   /usr/bin/telepresence --namespace ike-tests-epyskbtjlxzngrwx --deployment ratings-v1-v1-bartek-dmaxu --expose 9080 --method inject-tcp --run ike watch --dir '"."' --exclude '"*.log,.git/"' --interval 500 --run 'ruby ratings.rb 9080'
   0.0 TEL | Platform: linux
   0.0 TEL | Python 3.7.4 (default, Jul  9 2019, 16:32:37)
   0.0 TEL | [GCC 9.1.1 20190503 (Red Hat 9.1.1-1)]
   0.0 TEL | [1] Running: uname -a
   0.0   1 | Linux hydrobook 5.2.16-200.fc30.x86_64 #1 SMP Thu Sep 19 16:14:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
   0.0 TEL | [1] ran in 0.00 secs.
   0.0 TEL | BEGIN SPAN main.py:40(main)
   0.0 TEL | BEGIN SPAN startup.py:74(__init__)
   0.0 TEL | Found kubectl -> /home/bartek/.asdf/shims/kubectl
   0.0 TEL | Found oc -> /home/bartek/.asdf/shims/oc
   0.0 TEL | [2] Capturing: kubectl version --short
   0.8 TEL | [2] captured in 0.74 secs.
   0.8 TEL | [3] Capturing: kubectl config current-context
   1.0 TEL | [3] captured in 0.20 secs.
   1.0 TEL | [4] Capturing: kubectl config view -o json
   1.1 TEL | [4] captured in 0.19 secs.
   1.1 TEL | [5] Capturing: kubectl --context *** get ns ike-tests-epyskbtjlxzngrwx
   1.9 TEL | [5] captured in 0.77 secs.
   2.5 TEL | Command: kubectl 1.11.9
   2.5 TEL | Context: -------, namespace: ike-tests-epyskbtjlxzngrwx, version: 1.14.6+73b5d76
   2.5 >>> | Warning: kubectl 1.11.9 may not work correctly with cluster version 1.14.6+73b5d76 due to the version discrepancy. See https://kubernetes.io/docs/setup/version-skew-policy/ for more information.
   2.5 >>> | 
   2.5 TEL | END SPAN startup.py:74(__init__)    2.5s
   2.5 TEL | Found ssh -> /usr/bin/ssh
   2.5 TEL | [6] Capturing: ssh -V
   2.5 TEL | [6] captured in 0.02 secs.
   2.5 TEL | Found ike -> /home/bartek/code/golang/src/github.com/maistra/istio-workspace/dist/ike
   2.5 TEL | Found torsocks -> /usr/bin/torsocks
   2.5 TEL | Found sshfs -> /usr/bin/sshfs
   2.5 TEL | Found fusermount -> /usr/bin/fusermount
   2.5 TEL | [7] Running: kubectl --context *** --namespace ike-tests-epyskbtjlxzngrwx get pods telepresence-connectivity-check --ignore-not-found
   3.5 TEL | [7] ran in 0.96 secs.
   4.2 TEL | Scout info: {'latest_version': '0.101', 'application': 'telepresence', 'notices': []}
   4.2 >>> | Starting network proxy to cluster using the existing proxy Deployment ratings-v1-v1-bartek-dmaxu
   4.2 TEL | [8] Running: kubectl --context *** --namespace ike-tests-epyskbtjlxzngrwx get deployment ratings-v1-v1-bartek-dmaxu
   5.2   8 | No resources found.
   5.2   8 | Error from server (NotFound): deployments.extensions "ratings-v1-v1-bartek-dmaxu" not found
   5.2 TEL | [8] exit 1 in 1.03 secs.
   5.2 >>> | 
   5.2 >>> | Failed to find deployment ratings-v1-v1-bartek-dmaxu:
   5.2 >>> | No resources found.
   5.2 >>> | Error from server (NotFound): deployments.extensions "ratings-v1-v1-bartek-dmaxu" not found
   5.2 >>> | 
   5.2 TEL | EXITING with status code 255
   5.2 >>> | Exit cleanup in progress
   5.2 TEL | (Cleanup) Stop time tracking
   5.2 TEL | END SPAN main.py:40(main)    5.2s
   5.2 TEL | (Cleanup) Remove temporary directory
   5.2 TEL | (Cleanup) Save caches
@bartoszmajsak

This comment has been minimized.

Copy link
Contributor Author

@bartoszmajsak bartoszmajsak commented Oct 3, 2019

I'm looking at possible solutions right now and will provide a PR.

@ark3

This comment has been minimized.

Copy link
Contributor

@ark3 ark3 commented Oct 3, 2019

Is #1138 also talking about this?

@bartoszmajsak

This comment has been minimized.

Copy link
Contributor Author

@bartoszmajsak bartoszmajsak commented Oct 3, 2019

Not sure as it lacks details. However, it mentions DNS so probably not.

In general, it works for Openshift if the endpoint is not secured, which is a default for 3.11 and 4.1.

@ark3

This comment has been minimized.

Copy link
Contributor

@ark3 ark3 commented Oct 3, 2019

In #1138, the part about replacing a DC with a Deployment in the tutorial implies to me that Tel failed to detect OpenShift. I know that tutorial worked for me in the past, but I suspect that won't now with modern OpenShift...

@bartoszmajsak

This comment has been minimized.

Copy link
Contributor Author

@bartoszmajsak bartoszmajsak commented Oct 3, 2019

I know that tutorial worked for me in the past, but I suspect that won't now with modern OpenShift...

I will review the tutorial and chime in the related PR. It might be that there were changes in oc commands between 3.11 and 4.x which we have in the docs.

For modern, 4.1.x (officially released) Telepresence works fine as long as /apis endpoint is accessible for anonymous users (which is the default setting AFAIK). The problem I described here happens only when this endpoint is secured.

@bartoszmajsak

This comment has been minimized.

Copy link
Contributor Author

@bartoszmajsak bartoszmajsak commented Oct 3, 2019

That said, I think I have a solution so stay tuned :)

bartoszmajsak added a commit to Maistra/istio-workspace that referenced this issue Oct 3, 2019
bartoszmajsak added a commit to Maistra/istio-workspace that referenced this issue Oct 4, 2019
* chore(circleci): bumps golang to 1.13.1
* chore: reorganizes e2e tests to avoid duplication of scenario name definition
* chore: adds simple logging to test service
* chore: renames completion projects
* chore: obtains project dir using a func
* fix: aligns ocp with remote cluster
* fix: extracts entire tar
* chore: defines cluster version using env var
* feat: dumps telepresence log after test failure
* test: disables dc tests until telepresenceio/telepresence/issues/1139 is fixed
* chore: configures gocognit linter
* chore: locks down go tools to certain versions
bartoszmajsak added a commit to bartoszmajsak/telepresence that referenced this issue Oct 7, 2019
Previously called `/apis` endpoint which lists all resources in the
cluster can be secured making it impossible to determine if the actual
cluster is Openshift. In such a case Telepresence rolls back to
`kubectl` and assumes `Deployment` is used rather than `DeploymentConfig`
which is commonly used in Openshift.

Proposed solution calls Openshift-specific endpoint and based on the
response decides which cli tool to use.

Fixes telepresenceio#1139
bartoszmajsak added a commit to bartoszmajsak/telepresence that referenced this issue Oct 7, 2019
Previously called `/apis` endpoint which lists all resources in the
cluster can be secured making it impossible to determine if the actual
cluster is Openshift. In such a case Telepresence rolls back to
`kubectl` and assumes `Deployment` is used rather than `DeploymentConfig`
which is commonly used in Openshift.

Proposed solution calls Openshift-specific endpoint and based on the
response decides which cli tool to use.

Fixes telepresenceio#1139
@ark3 ark3 closed this in #1143 Oct 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.