Technical Questions

[one000mph]

• Is there a way to gracefully shutdown teleport or is ^C/kill the right way at the moment?
• Is there a best practice for running in the background? teleport start &?
• From my experimentation it looks like tsh join will work only from hosts that can reach the advertised IP of a node such as a public-facing IP or from a client that can access the private IP. For example, from my machine I can tsh join a remote session if the advertised IP is explicitly set to a public IP but not if it is defaulted to localhost or the private 10.x.x.x IP. This is different from what I expected since it doesn't match the behavior of tsh ssh. In the Architecture section the diagrams are pretty good at showing how data flows through the proxy server using tsh ssh but I'm assuming this doesn't work the same way with tsh join. Is that correct? Any additional feedback here would be great!
• Where are the self-signed certs located? I see /var/lib/teleport/webproxy_cert.pem but that doesn't appear to be the one that is used. At one point I got a ssh handshake error (which I was able to fix) that had these principles listed `["[host_uuid].[hostname]" "[nodename | hostname]" "remote.kube.proxy.teleport.cluster.local"].

[klizhentas]

Good questions! Some answers below:

• Teleport supports graceful reloads, but not shutdowns at the moment. Here is a list of all signals we support:

https://gravitational.com/teleport/docs/admin-guide/#graceful-restarts

• Is there a best practice for running in the background? teleport start &?

We usually recommend to use systemd units for background operatoin

• tsh join will work only from hosts that can reach the advertised IP of a node such as a public-facing IP or from a client that can access the private IP.

This is not an expected behavior, so it might be a bug

• Where are the self-signed certs located? I see /var/lib/teleport/
  Teleport operates it's own internal CA. Every node and proxy certificates are issued using this CA.
  Located in /var/lib/teleport/proc sqlite database

[benarent]

One extra note on the part of

Where are the self-signed certs located?

This can be a gotcha for some people. Teleport requires a TLS/SSL cert for the proxy, this is in reference to /var/lib/teleport/webproxy_cert.pem if you look at some of our examples we show how to use Letsencypt https://gravitational.com/blog/letsencrypt-teleport-ssh/

For hosts that terminate TLS/SSL at the load balancer we provide the --insecure-no-tls flag. This is handy if you're using ALB https://gravitational.com/teleport/docs/admin-guide/#configuration-flags

We also have this example that shows obtaining it with ACM https://gravitational.com/teleport/docs/aws_oss_guide/#prerequisites

[one000mph]

This is not an expected behavior, so it might be a bug
@klizhentas would it be helpful to report the behavior somewhere in your main repo? I can write up steps to reproduce.

[klizhentas]

yes, thank you!

[one000mph]

@benarent On a slightly different tangent with the --insecure flag for Auth service. When I first started up teleport I encountered some issue related to /bin/hostname not matching the proxy subdomain even when using tsh with --insecure flag. Is this a bug also? It seems to contradict the cli docs on the flag: Do not verify server's certificate and host name. Use only in test environments. I get certificate signed by unknown authority without the --insecure flag. When use --insecure and the --proxy=proxy_host does not match the hostname or nodename (when present) I get principal "proxy_host" not in the set of valid principals for given certificate

[klizhentas]

this is a good observation, the way it works is that --insecure is not 100% insecure :) It was introduced to skip Web proxy TLS validation, however it is still at all times verifies the identity of the auth server. In this case we should add better error message and provide some guidance on what users/admins should do, so I would raise it as another UX bug in our OSS github repository.

[one000mph]

it is still at all times verifies the identity of the auth server

OK that's the answer to my question! Thanks.

Updating the CLI docs will definitely help. At a first run I would assume that most people haven't read the CLI docs very deeply so the messaging there isn't as crucial at least for early-stage users. Would it make sense to add an instruction to ensure that nodename matches the value of --proxy during authentication?

I'll think about where to add the info about --insecure - it only affects the proxy cert, but not auth.

[one000mph]

I think these issues on the main repo have already summarized what I experience gravitational#2910 gravitational#2040

If you want a new issue though I'm happy to do that!

[one000mph]

Just filed this bug on tsh join gravitational#3058