Self-hosted, self-signed installation #11268

MRIIOT · 2022-03-18T21:45:18Z

MRIIOT
Mar 18, 2022

Wanted to share my configuration so far on Ubuntu 20.

What works:

node tunneling

What does not work:

tunnel into apps (DNS ?)

What I'm not sure about:

self-signed certs

PROXY+AUTH

INSTALL

sudo rm -rf /var/lib/teleport

cd ~
curl -O https://get.gravitational.com/teleport-v9.0.1-linux-amd64-bin.tar.gz
tar -xzf teleport-v9.0.1-linux-amd64-bin.tar.gz
cd teleport
sudo ./install

GENERATE KEYS

sudo mkdir -p /etc/pki/tls/private
sudo mkdir -p /etc/pki/tls/certs

sudo openssl req -x509 -days 3650 -nodes -newkey rsa:2048 \
	-keyout /etc/pki/tls/private/teleport.key \
	-out /etc/pki/tls/certs/teleport.crt \
	-subj "/CN=toronto-hub-000-dev" \
	-addext "subjectAltName = DNS:toronto-hub-000-dev"

YAML

version: v2

teleport:
  nodename: 000-toronto

  data_dir: /var/lib/teleport

  advertise_ip: toronto-hub-000-dev

  #diag_addr: "toronto-hub-000-dev:3001"

  connection_limits:
    max_connections: 1000
    max_users: 250

  log:
    #output: /var/lib/teleport/teleport.log
    output: stderr
    severity: DEBUG
    format:
      #output: json
      output: text

auth_service:
  enabled: "yes"

  cluster_name: "toronto"

  authentication:
    type: local

    #second_factor: otp
    second_factor: off

  listen_addr: 0.0.0.0:3025

  public_addr: toronto-hub-000-dev:3025

  tokens:
    - "node,app:veidoo0au3shiSie9aim7phieze0iic1"

  session_recording: "node"

  client_idle_timeout: "4h"

  web_idle_timeout: "10m"

  disconnect_expired_cert: no

ssh_service:
  enabled: "yes"

  #listen_addr: 0.0.0.0:3022

  #public_addr: toronto-hub-000-dev:3022

  labels:
    svc: ssh
    id: 000
    owner: toronto
    hub: true

proxy_service:
  enabled: "yes"

  listen_addr: 0.0.0.0:3023

  tunnel_listen_addr: 0.0.0.0:3024

  web_listen_addr: 0.0.0.0:443

  public_addr: toronto-hub-000-dev:443

  #ssh_public_addr: toronto-hub-000-dev:3023

  tunnel_public_addr: toronto-hub-000-dev:3024

  https_keypairs:
    - key_file: /etc/pki/tls/private/teleport.key
      cert_file: /etc/pki/tls/certs/teleport.crt

kubernetes_service:
  enabled: no

db_service:
  enabled: no

windows_desktop_service:
  enabled: no

START AND CREATE USER

sudo teleport start --insecure

sudo tctl users add admin --roles=editor,access --logins=root

LOG

2022-03-18T21:47:42Z WARN             Starting Teleport with a self-signed TLS certificate, this is not safe for production clusters. Using a self-signed certificate opens Teleport users to Man-in-the-Middle attacks. config/configuration.go:725
2022-03-18T21:47:42Z INFO             Generating new host UUID: 24d3f0e1-e243-43ae-ad69-b40748fed12f. service/service.go:663
2022-03-18T21:47:42Z DEBU [SQLITE]    Connected to: file:/var/lib/teleport/proc/sqlite.db?_busy_timeout=10000&_sync=OFF, poll stream period: 1s lite/lite.go:169
2022-03-18T21:47:42Z DEBU [SQLITE]    Synchronous: 0, busy timeout: 10000 lite/lite.go:212
2022-03-18T21:47:42Z DEBU [KEYGEN]    SSH cert authority is going to pre-compute 25 keys. native/native.go:102
2022-03-18T21:47:42Z DEBU [PROC:1]    Using sqlite backend. service/service.go:3742
2022-03-18T21:47:42Z DEBU [SQLITE]    Connected to: file:/var/lib/teleport/backend/sqlite.db?_busy_timeout=10000&_sync=OFF, poll stream period: 1s lite/lite.go:169
2022-03-18T21:47:42Z DEBU [SQLITE]    Synchronous: 0, busy timeout: 10000 lite/lite.go:212
2022-03-18T21:47:42Z INFO [AUTH]      Updating cluster networking configuration: Kind:"cluster_networking_config" Version:"v2" Metadata:<Name:"cluster-networking-config" Namespace:"default" Labels:<key:"teleport.dev/origin" value:"config-file" > > Spec:<ClientIdleTimeout:14400000000000 KeepAliveInterval:300000000000 KeepAliveCountMax:3 WebIdleTimeout:600000000000 > . auth/init.go:434
2022-03-18T21:47:42Z INFO [AUTH]      Updating session recording configuration: Kind:"session_recording_config" Version:"v2" Metadata:<Name:"session-recording-config" Namespace:"default" Labels:<key:"teleport.dev/origin" value:"config-file" > > Spec:<Mode:"node" ProxyChecksHostKeys:"\010\001" > . auth/init.go:452
2022-03-18T21:47:42Z INFO [AUTH]      Updating cluster auth preference: AuthPreference(Type="local",SecondFactor="off"). auth/init.go:416
2022-03-18T21:47:42Z DEBU [AUTH]      Cluster configuration: ClusterName(toronto, ID=16fca234-53c4-473c-9166-a2819fa04922). auth/init.go:303
2022-03-18T21:47:42Z INFO [AUTH]      Updating cluster configuration: StaticTokens([ProvisionToken(Roles=[Node App], Expires=never)]). auth/init.go:309
2022-03-18T21:47:42Z INFO [AUTH]      Created namespace: "default". auth/init.go:316
2022-03-18T21:47:42Z INFO [AUTH]      First start: generating host certificate authority. auth/init.go:326
2022-03-18T21:47:43Z DEBU [SQLITE]    Initialized event ID iterator to 7 lite/periodic.go:155
2022-03-18T21:47:43Z INFO [AUTH]      First start: generating user certificate authority. auth/init.go:326
2022-03-18T21:47:45Z INFO [AUTH]      First start: generating jwt certificate authority. auth/init.go:326
2022-03-18T21:47:45Z WARN [AUTH]      Starting teleport in insecure mode. This is dangerous! Sensitive information will be logged to console and certificates will not be verified. Proceed with caution! auth/init.go:378
2022-03-18T21:47:45Z DEBU [AUTH]      Migrations: skipping local cluster cert authority "toronto". auth/init.go:960
2022-03-18T21:47:45Z INFO [AUTH]      Auth server is running periodic operations. auth/init.go:394
2022-03-18T21:47:45Z DEBU [PROC:1]    This server has local Auth server started, using it to add role to the cluster. service/connect.go:371
2022-03-18T21:47:45Z DEBU [AUTH]      Ticking with period: 12s. auth/auth.go:408
2022-03-18T21:47:45Z DEBU [AUTH:1]    Starting watch. resource-kind:lock services/watcher.go:182
2022-03-18T21:47:45Z DEBU [BUFFER]    Add Watcher(name=auth, prefixes=/locks, capacity=1024, size=0). backend/buffer.go:312
2022-03-18T21:47:46Z DEBU [KEYGEN]    Generated SSH host certificate for role Admin with principals: [24d3f0e1-e243-43ae-ad69-b40748fed12f 000-toronto toronto-hub-000-dev]. native/native.go:231
2022-03-18T21:47:46Z INFO [CA]        Generating TLS certificate {0x8304640 0xc0010be790 1.3.9999.1.7=#1306747572616e6f,CN=24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto,O=Admin,POSTALCODE=null,STREET= 2032-03-15 21:47:46.30779926 +0000 UTC [000-toronto toronto-hub-000-dev *.teleport.cluster.local teleport.cluster.local] [] 5 []}. common_name:24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto dns_names:[000-toronto toronto-hub-000-dev *.teleport.cluster.local teleport.cluster.local] locality:[] not_after:2032-03-15 21:47:46.30779926 +0000 UTC org:[Admin] org_unit:[] tlsca/ca.go:800
2022-03-18T21:47:46Z INFO [PROC:1]    Admin has obtained credentials to connect to the cluster. service/connect.go:416
2022-03-18T21:47:46Z INFO [PROC:1]    The process successfully wrote the credentials and state of Admin to the disk. service/connect.go:457
2022-03-18T21:47:46Z DEBU [PROC:1]    Connected client: Identity(Admin, cert(24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto issued by toronto:272982335074390935157946903432738681364),trust root(toronto:272982335074390935157946903432738681364)) service/connect.go:124
2022-03-18T21:47:46Z DEBU [PROC:1]    Creating in-memory backend for [auth]. service/service.go:1562
2022-03-18T21:47:46Z DEBU [BUFFER]    Add Watcher(name=auth:1:cache, prefixes=/access_requests, /appServers/default, /applications, /apps/servers/default, /apps/sessions, /authentication/preference/general, /authorities, /authservers, /cluster_configuration/audit, /cluster_configuration/name, /cluster_configuration/networking, /cluster_configuration/session_recording, /cluster_configuration/static_tokens, /databaseServers/default, /db, /kubeServices, /locks, /namespaces, /nodes/default, /proxies, /remoteClusters, /restrictions/network, /reverseTunnels, /roles, /tokens, /tunnelConnections, /web/sessions, /web/tokens, /web/users, /windowsDesktop, capacity=8192, size=0). backend/buffer.go:312
2022-03-18T21:47:46Z INFO [AUTH:1:CA] Cache "auth" first init succeeded. cache/cache.go:742
2022-03-18T21:47:46Z INFO [PROC:1]    Service auth is creating new listener on 0.0.0.0:3025. service/signals.go:213
2022-03-18T21:47:46Z INFO [AUTH:1]    Starting Auth service with PROXY protocol support. service/service.go:1341
2022-03-18T21:47:46Z DEBU [AUTH]      GRPC(SERVER): keep alive 1m0s count: 3. auth/grpcserver.go:3909
2022-03-18T21:47:46Z DEBU [PROC:1]    Adding service to supervisor. service:auth.tls service/supervisor.go:201
2022-03-18T21:47:46Z DEBU [PROC:1]    Adding service to supervisor. service:auth.heartbeat.broadcast service/supervisor.go:201
2022-03-18T21:47:46Z DEBU [AUTH:BEAT] Starting Auth heartbeat with announce period: 1m0s, keep-alive period 5m26.19935844s, poll period: 5s srv/heartbeat.go:155
2022-03-18T21:47:46Z DEBU [PROC:1]    Adding service to supervisor. service:auth.heartbeat service/supervisor.go:201
2022-03-18T21:47:46Z DEBU [PROC:1]    Adding service to supervisor. service:auth.shutdown service/supervisor.go:201
2022-03-18T21:47:46Z DEBU [PROC:1]    Adding service to supervisor. service:register.node service/supervisor.go:201
2022-03-18T21:47:46Z DEBU [PROC:1]    Adding service to supervisor. service:ssh.node service/supervisor.go:201
2022-03-18T21:47:46Z DEBU [PROC:1]    Adding service to supervisor. service:ssh.shutdown service/supervisor.go:201
2022-03-18T21:47:46Z DEBU [PROC:1]    Adding service to supervisor. service:register.proxy service/supervisor.go:201
2022-03-18T21:47:46Z DEBU [PROC:1]    Adding service to supervisor. service:proxy.init service/supervisor.go:201
2022-03-18T21:47:46Z DEBU [PROC:1]    Adding service to supervisor. service:common.rotate service/supervisor.go:201
2022-03-18T21:47:46Z DEBU [PROC:1]    No signal pipe to import, must be first Teleport process. service/service.go:882
2022-03-18T21:47:46Z DEBU [PROC:1]    Service has started. service:auth.tls service/supervisor.go:262
2022-03-18T21:47:46Z INFO [AUTH:1]    Auth service 9.0.1:v9.0.1-0-g7bbe6f15c is starting on toronto-hub-000-dev:3025. utils/cli.go:275
[AUTH]         Auth service 9.0.1:v9.0.1-0-g7bbe6f15c is starting on toronto-hub-000-dev:3025.
2022-03-18T21:47:46Z DEBU [PROC:1]    Broadcasting event. event:AuthTLSReady service/supervisor.go:370
2022-03-18T21:47:46Z DEBU [PROC:1]    Service has started. service:auth.heartbeat.broadcast service/supervisor.go:262
2022-03-18T21:47:46Z DEBU [PROC:1]    Connected state: never updated. service/connect.go:140
2022-03-18T21:47:46Z DEBU [PROC:1]    Connected client: Identity(Admin, cert(24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto issued by toronto:272982335074390935157946903432738681364),trust root(toronto:272982335074390935157946903432738681364)) service/connect.go:124
2022-03-18T21:47:46Z DEBU [PROC:1]    Broadcasting event. event:AuthIdentity service/supervisor.go:370
2022-03-18T21:47:46Z DEBU [PROC:1]    Adding service to supervisor. service:auth.broadcast service/supervisor.go:201
2022-03-18T21:47:46Z DEBU [PROC:1]    Service is completed and removed. service:auth.heartbeat.broadcast service/supervisor.go:239
2022-03-18T21:47:46Z DEBU [PROC:1]    Service has started. service:auth.broadcast service/supervisor.go:262
2022-03-18T21:47:46Z DEBU [PROC:1]    Service has started. service:auth.heartbeat service/supervisor.go:262
2022-03-18T21:47:46Z DEBU [PROC:1]    Service has started. service:auth.shutdown service/supervisor.go:262
2022-03-18T21:47:46Z DEBU [PROC:1]    Service has started. service:register.node service/supervisor.go:262
2022-03-18T21:47:46Z DEBU [PROC:1]    This server has local Auth server started, using it to add role to the cluster. service/connect.go:371
2022-03-18T21:47:46Z DEBU [PROC:1]    Service has started. service:ssh.node service/supervisor.go:262
2022-03-18T21:47:46Z DEBU [PROC:1]    Service has started. service:ssh.shutdown service/supervisor.go:262
2022-03-18T21:47:46Z DEBU [PROC:1]    Service has started. service:register.proxy service/supervisor.go:262
2022-03-18T21:47:46Z DEBU [PROC:1]    This server has local Auth server started, using it to add role to the cluster. service/connect.go:371
2022-03-18T21:47:46Z DEBU [PROC:1]    Service has started. service:proxy.init service/supervisor.go:262
2022-03-18T21:47:46Z DEBU [PROC:1]    Service has started. service:common.rotate service/supervisor.go:262
2022-03-18T21:47:47Z DEBU [KEYGEN]    Generated SSH host certificate for role Proxy with principals: [24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto 24d3f0e1-e243-43ae-ad69-b40748fed12f 000-toronto.toronto 000-toronto localhost 127.0.0.1 ::1 toronto-hub-000-dev remote.kube.proxy.teleport.cluster.local]. native/native.go:231
2022-03-18T21:47:47Z INFO [CA]        Generating TLS certificate {0x8304640 0xc000d2c120 1.3.9999.1.7=#1306747572616e6f,CN=24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto,O=Proxy,POSTALCODE=null,STREET= 2032-03-15 21:47:47.153178348 +0000 UTC [000-toronto toronto-hub-000-dev localhost 127.0.0.1 ::1 remote.kube.proxy.teleport.cluster.local *.teleport.cluster.local teleport.cluster.local *.toronto-hub-000-dev] [] 5 []}. common_name:24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto dns_names:[000-toronto toronto-hub-000-dev localhost 127.0.0.1 ::1 remote.kube.proxy.teleport.cluster.local *.teleport.cluster.local teleport.cluster.local *.toronto-hub-000-dev] locality:[] not_after:2032-03-15 21:47:47.153178348 +0000 UTC org:[Proxy] org_unit:[] tlsca/ca.go:800
2022-03-18T21:47:47Z INFO [PROC:1]    Proxy has obtained credentials to connect to the cluster. service/connect.go:416
2022-03-18T21:47:47Z DEBU [PROC:1]    Attempting to connect to Auth Server directly. auth-addrs:[0.0.0.0:3025] service/connect.go:850
2022-03-18T21:47:47Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:47:47Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:47:47Z DEBU [AUTH:1]    Server certificate cert(24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto issued by toronto:272982335074390935157946903432738681364). auth/middleware.go:293
2022-03-18T21:47:47Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:47:47Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:47:47Z DEBU [AUTH:1]    Server certificate cert(24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto issued by toronto:272982335074390935157946903432738681364). auth/middleware.go:293
2022-03-18T21:47:47Z DEBU [PROC:1]    Connected to Auth Server with direct connection. auth-addrs:[0.0.0.0:3025] service/connect.go:853
2022-03-18T21:47:47Z INFO [PROC:1]    The process successfully wrote the credentials and state of Proxy to the disk. service/connect.go:457
2022-03-18T21:47:47Z DEBU [PROC:1]    Connected client: Identity(Proxy, cert(24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto issued by toronto:272982335074390935157946903432738681364),trust root(toronto:272982335074390935157946903432738681364)) service/connect.go:124
2022-03-18T21:47:47Z INFO [PROC:1]    Proxy: features loaded from auth server: Kubernetes:true App:true DB:true Desktop:true  service/connect.go:71
2022-03-18T21:47:47Z DEBU [PROC:1]    Broadcasting event. event:ProxyIdentity service/supervisor.go:370
2022-03-18T21:47:47Z DEBU [PROC:1]    Service is completed and removed. service:register.proxy service/supervisor.go:239
2022-03-18T21:47:47Z DEBU [PROC:1]    Received event "ProxyIdentity". service/service.go:2478
2022-03-18T21:47:47Z DEBU [PROC:1]    Creating sqlite backend for [proxy]. service/service.go:1573
2022-03-18T21:47:47Z DEBU [SQLITE]    Connected to: file:/var/lib/teleport/cache/proxy/sqlite.db?_busy_timeout=10000&_sync=OFF, poll stream period: 100ms lite/lite.go:169
2022-03-18T21:47:47Z DEBU [SQLITE]    Synchronous: 0, busy timeout: 10000 lite/lite.go:212
2022-03-18T21:47:47Z INFO [PROXY:1:C] Cache "proxy" first init succeeded. cache/cache.go:742
2022-03-18T21:47:47Z DEBU [PROC:1]    Setup Proxy: Web Proxy Address: 0.0.0.0:443, Reverse Tunnel Proxy Address: 0.0.0.0:3024 service/service.go:2572
2022-03-18T21:47:47Z INFO [PROC:1]    Service proxy:ssh is creating new listener on 0.0.0.0:3023. service/signals.go:213
2022-03-18T21:47:47Z DEBU [PROC:1]    Setup Proxy: Proxy protocol is enabled for web service, multiplexing is on. service/service.go:2641
2022-03-18T21:47:47Z INFO [PROC:1]    Service proxy:web is creating new listener on 0.0.0.0:443. service/signals.go:213
2022-03-18T21:47:47Z INFO [PROC:1]    Service proxy:tunnel is creating new listener on 0.0.0.0:3024. service/signals.go:213
2022-03-18T21:47:47Z DEBU [PROC:1]    Creating sqlite backend for [reverse toronto]. service/service.go:1573
2022-03-18T21:47:47Z DEBU [SQLITE]    Connected to: file:/var/lib/teleport/cache/reverse/toronto/sqlite.db?_busy_timeout=10000&_sync=OFF, poll stream period: 100ms lite/lite.go:169
2022-03-18T21:47:47Z DEBU [SQLITE]    Synchronous: 0, busy timeout: 10000 lite/lite.go:212
2022-03-18T21:47:47Z DEBU [PROXY]     Starting watch. resource-kind:lock services/watcher.go:182
2022-03-18T21:47:47Z DEBU [PROXY:1]   Starting watch. resource-kind:proxy services/watcher.go:182
2022-03-18T21:47:47Z INFO [REVERSE:T] Cache "remote-proxy" first init succeeded. cache/cache.go:742
2022-03-18T21:47:47Z DEBU [SSH:PROXY] Supported ciphers: ["aes128-gcm@openssh.com" "chacha20-poly1305@openssh.com" "aes128-ctr" "aes192-ctr" "aes256-ctr"]. sshutils/server.go:229
2022-03-18T21:47:47Z DEBU [SSH:PROXY] Supported KEX algorithms: ["curve25519-sha256@libssh.org" "ecdh-sha2-nistp256" "ecdh-sha2-nistp384" "ecdh-sha2-nistp521"]. sshutils/server.go:239
2022-03-18T21:47:47Z DEBU [SSH:PROXY] Supported MAC algorithms: ["hmac-sha2-256-etm@openssh.com" "hmac-sha2-256"]. sshutils/server.go:249
2022-03-18T21:47:47Z DEBU [PROC:1]    Adding service to supervisor. service:proxy.reversetunnel.server service/supervisor.go:201
2022-03-18T21:47:47Z INFO             Loading TLS certificate /etc/pki/tls/certs/teleport.crt and key /etc/pki/tls/private/teleport.key. service/service.go:3368
2022-03-18T21:47:47Z DEBU [PROC:1]    Service has started. service:proxy.reversetunnel.server service/supervisor.go:262
2022-03-18T21:47:47Z INFO [PROXY:SER] Reverse tunnel service 9.0.1:v9.0.1-0-g7bbe6f15c is starting on 0.0.0.0:3024. utils/cli.go:275
[PROXY]        Reverse tunnel service 9.0.1:v9.0.1-0-g7bbe6f15c is starting on 0.0.0.0:3024.
2022-03-18T21:47:47Z INFO [PROXY:SER] Starting 9.0.1:v9.0.1-0-g7bbe6f15c on 0.0.0.0:3024 using sqlite cache will store frequently accessed items service/service.go:2870
2022-03-18T21:47:47Z DEBU [PROC:1]    Broadcasting event. event:ProxyReverseTunnelReady service/supervisor.go:370
2022-03-18T21:47:47Z DEBU [SSH:PROXY] Listening on [::]:443. sshutils/server.go:382
2022-03-18T21:47:47Z DEBU [PROC:1]    Adding service to supervisor. service:proxy.web service/supervisor.go:201
2022-03-18T21:47:47Z DEBU [SSH:PROXY] Supported ciphers: ["aes128-gcm@openssh.com" "chacha20-poly1305@openssh.com" "aes128-ctr" "aes192-ctr" "aes256-ctr"]. sshutils/server.go:229
2022-03-18T21:47:47Z DEBU [SSH:PROXY] Supported KEX algorithms: ["curve25519-sha256@libssh.org" "ecdh-sha2-nistp256" "ecdh-sha2-nistp384" "ecdh-sha2-nistp521"]. sshutils/server.go:239
2022-03-18T21:47:47Z DEBU [SSH:PROXY] Supported MAC algorithms: ["hmac-sha2-256-etm@openssh.com" "hmac-sha2-256"]. sshutils/server.go:249
2022-03-18T21:47:47Z DEBU [PROXY:BEA] Starting Proxy heartbeat with announce period: 1m0s, keep-alive period 5m7.852341936s, poll period: 5s srv/heartbeat.go:155
2022-03-18T21:47:47Z DEBU [PROC:1]    Adding service to supervisor. service:proxy.ssh service/supervisor.go:201
2022-03-18T21:47:47Z DEBU [PROC:1]    Adding service to supervisor. service:proxy.reversetunnel.watcher service/supervisor.go:201
2022-03-18T21:47:47Z DEBU [PROXY:PRO] Reading Kubernetes creds. kubeClusterName: kubeconfigPath: serviceType:1 proxy/auth.go:91
2022-03-18T21:47:47Z DEBU             No local kube credentials on proxy, will not start kubernetes_service heartbeats proxy/server.go:158
2022-03-18T21:47:47Z DEBU [PROC:1]    Adding service to supervisor. service:proxy.kube service/supervisor.go:201
2022-03-18T21:47:47Z DEBU [PROC:1]    Adding service to supervisor. service:proxy.db.postgres service/supervisor.go:201
2022-03-18T21:47:47Z DEBU [PROC:1]    Adding service to supervisor. service:proxy.db.tls service/supervisor.go:201
2022-03-18T21:47:47Z DEBU [PROC:1]    Adding service to supervisor. service:proxy.grpc service/supervisor.go:201
2022-03-18T21:47:47Z DEBU [PROC:1]    Adding service to supervisor. service:proxy.tls.alpn.sni.proxy service/supervisor.go:201
2022-03-18T21:47:47Z DEBU [PROC:1]    Adding service to supervisor. service:proxy.shutdown service/supervisor.go:201
2022-03-18T21:47:47Z INFO [AUDIT:1]   Creating directory /var/lib/teleport/log. service/service.go:2115
2022-03-18T21:47:47Z INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload. service/service.go:2115
2022-03-18T21:47:47Z INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload/sessions. service/service.go:2115
2022-03-18T21:47:47Z INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload/sessions/default. service/service.go:2115
2022-03-18T21:47:47Z INFO [AUDIT:1]   Creating directory /var/lib/teleport/log. service/service.go:2115
2022-03-18T21:47:47Z INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload. service/service.go:2115
2022-03-18T21:47:47Z INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload/streaming. service/service.go:2115
2022-03-18T21:47:47Z INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload/streaming/default. service/service.go:2115
2022-03-18T21:47:47Z DEBU [PROC:1]    Adding service to supervisor. service:fileuploader.service service/supervisor.go:201
2022-03-18T21:47:47Z DEBU [PROC:1]    Adding service to supervisor. service:fileuploader.shutdown service/supervisor.go:201
2022-03-18T21:47:47Z DEBU [PROC:1]    Service is completed and removed. service:proxy.init service/supervisor.go:239
2022-03-18T21:47:47Z DEBU [PROC:1]    Service has started. service:fileuploader.shutdown service/supervisor.go:262
2022-03-18T21:47:47Z DEBU [PROC:1]    Service has started. service:proxy.web service/supervisor.go:262
2022-03-18T21:47:47Z INFO [PROXY:SER] Web proxy service 9.0.1:v9.0.1-0-g7bbe6f15c is starting on 0.0.0.0:443. utils/cli.go:275
[PROXY]        Web proxy service 9.0.1:v9.0.1-0-g7bbe6f15c is starting on 0.0.0.0:443.
2022-03-18T21:47:47Z INFO [PROXY:SER] Web proxy service 9.0.1:v9.0.1-0-g7bbe6f15c is starting on 0.0.0.0:443. service/service.go:2959
2022-03-18T21:47:47Z DEBU [PROC:1]    Broadcasting event. event:ProxyWebServerReady service/supervisor.go:370
2022-03-18T21:47:47Z DEBU [PROC:1]    Service has started. service:proxy.ssh service/supervisor.go:262
2022-03-18T21:47:47Z INFO [PROXY:SER] SSH proxy service 9.0.1:v9.0.1-0-g7bbe6f15c is starting on 0.0.0.0:3023. utils/cli.go:275
[PROXY]        SSH proxy service 9.0.1:v9.0.1-0-g7bbe6f15c is starting on 0.0.0.0:3023.
2022-03-18T21:47:47Z INFO [PROXY:SER] SSH proxy service 9.0.1:v9.0.1-0-g7bbe6f15c is starting on {0.0.0.0:3023 tcp } service/service.go:3000
2022-03-18T21:47:47Z DEBU [PROC:1]    Broadcasting event. event:ProxySSHReady service/supervisor.go:370
2022-03-18T21:47:47Z DEBU [PROC:1]    Service is completed and removed. service:proxy.ssh service/supervisor.go:239
2022-03-18T21:47:47Z DEBU [PROC:1]    Service has started. service:proxy.reversetunnel.watcher service/supervisor.go:262
2022-03-18T21:47:47Z INFO [PROXY:AGE] Starting reverse tunnel agent pool. service/service.go:3033
2022-03-18T21:47:47Z DEBU [PROC:1]    Broadcasting event. event:ProxyAgentPoolReady service/supervisor.go:370
2022-03-18T21:47:47Z DEBU [PROC:1]    Service has started. service:proxy.kube service/supervisor.go:262
2022-03-18T21:47:47Z INFO [PROXY:PRO] Starting Kube proxy on . service/service.go:3092
2022-03-18T21:47:47Z DEBU [PROC:1]    Service has started. service:proxy.db.postgres service/supervisor.go:262
2022-03-18T21:47:47Z INFO [DB:SERVIC] Starting Postgres proxy server on 0.0.0.0:443. service/service.go:3158
2022-03-18T21:47:47Z DEBU [DB:PROXY]  Started database proxy. db/proxyserver.go:196
2022-03-18T21:47:47Z DEBU [PROC:1]    Service has started. service:proxy.db.tls service/supervisor.go:262
2022-03-18T21:47:47Z INFO [DB:SERVIC] Starting Database TLS proxy server on 0.0.0.0:443. service/service.go:3176
2022-03-18T21:47:47Z DEBU [DB:PROXY]  Started database TLS proxy. db/proxyserver.go:282
2022-03-18T21:47:47Z DEBU [PROC:1]    Service has started. service:proxy.grpc service/supervisor.go:262
2022-03-18T21:47:47Z INFO [PROXY:SER] Starting proxy gRPC server on [::]:443. service/service.go:3210
2022-03-18T21:47:47Z DEBU [PROC:1]    Service has started. service:proxy.tls.alpn.sni.proxy service/supervisor.go:262
2022-03-18T21:47:47Z INFO [PROXY:SER] Starting TLS ALPN SNI proxy server on [::]:443. service/service.go:3239
2022-03-18T21:47:47Z DEBU [PROC:1]    Service has started. service:proxy.shutdown service/supervisor.go:262
2022-03-18T21:47:47Z DEBU [PROC:1]    Service has started. service:fileuploader.service service/supervisor.go:262
2022-03-18T21:47:47Z DEBU [SSH:PROXY] Listening on [::]:443. sshutils/server.go:382
2022-03-18T21:47:47Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:47:47Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:47:47Z DEBU [AUTH:1]    Server certificate cert(24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto issued by toronto:272982335074390935157946903432738681364). auth/middleware.go:293
2022-03-18T21:47:48Z DEBU [KEYGEN]    Generated SSH host certificate for role Node with principals: [24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto 24d3f0e1-e243-43ae-ad69-b40748fed12f 000-toronto.toronto 000-toronto localhost 127.0.0.1 ::1 toronto-hub-000-dev]. native/native.go:231
2022-03-18T21:47:48Z INFO [CA]        Generating TLS certificate {0x8304640 0xc0010be2f0 1.3.9999.1.7=#1306747572616e6f,CN=24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto,O=Node,POSTALCODE=null,STREET= 2032-03-15 21:47:48.107406421 +0000 UTC [000-toronto 24d3f0e1-e243-43ae-ad69-b40748fed12f toronto-hub-000-dev] [] 5 []}. common_name:24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto dns_names:[000-toronto 24d3f0e1-e243-43ae-ad69-b40748fed12f toronto-hub-000-dev] locality:[] not_after:2032-03-15 21:47:48.107406421 +0000 UTC org:[Node] org_unit:[] tlsca/ca.go:800
2022-03-18T21:47:48Z INFO [PROC:1]    Node has obtained credentials to connect to the cluster. service/connect.go:416
2022-03-18T21:47:48Z DEBU [PROC:1]    Attempting to connect to Auth Server directly. auth-addrs:[0.0.0.0:3025] service/connect.go:850
2022-03-18T21:47:48Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:47:48Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:47:48Z DEBU [AUTH:1]    Server certificate cert(24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto issued by toronto:272982335074390935157946903432738681364). auth/middleware.go:293
2022-03-18T21:47:48Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:47:48Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:47:48Z DEBU [AUTH:1]    Server certificate cert(24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto issued by toronto:272982335074390935157946903432738681364). auth/middleware.go:293
2022-03-18T21:47:48Z DEBU [PROC:1]    Connected to Auth Server with direct connection. auth-addrs:[0.0.0.0:3025] service/connect.go:853
2022-03-18T21:47:48Z INFO [PROC:1]    The process successfully wrote the credentials and state of Node to the disk. service/connect.go:457
2022-03-18T21:47:48Z DEBU [PROC:1]    Connected client: Identity(Node, cert(24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto issued by toronto:272982335074390935157946903432738681364),trust root(toronto:272982335074390935157946903432738681364)) service/connect.go:124
2022-03-18T21:47:48Z INFO [PROC:1]    Node: features loaded from auth server: Kubernetes:true App:true DB:true Desktop:true  service/connect.go:71
2022-03-18T21:47:48Z DEBU [PROC:1]    Broadcasting event. event:SSHIdentity service/supervisor.go:370
2022-03-18T21:47:48Z DEBU [PROC:1]    Service is completed and removed. service:register.node service/supervisor.go:239
2022-03-18T21:47:48Z DEBU [NODE:1]    Received event "SSHIdentity". service/service.go:1811
2022-03-18T21:47:48Z DEBU [PROC:1]    Creating sqlite backend for [node]. service/service.go:1573
2022-03-18T21:47:48Z DEBU [SQLITE]    Connected to: file:/var/lib/teleport/cache/node/sqlite.db?_busy_timeout=10000&_sync=OFF, poll stream period: 100ms lite/lite.go:169
2022-03-18T21:47:48Z DEBU [SQLITE]    Synchronous: 0, busy timeout: 10000 lite/lite.go:212
2022-03-18T21:47:48Z INFO [NODE:1:CA] Cache "node" first init succeeded. cache/cache.go:742
2022-03-18T21:47:48Z DEBU [BPF]       Enhanced session recording is not enabled, skipping. bpf/bpf.go:124
2022-03-18T21:47:48Z DEBU [RESTRICTE] Restricted session is not enabled, skipping. restrictedsession/restricted.go:93
2022-03-18T21:47:48Z DEBU [SSH:NODE]  Supported ciphers: ["aes128-gcm@openssh.com" "chacha20-poly1305@openssh.com" "aes128-ctr" "aes192-ctr" "aes256-ctr"]. sshutils/server.go:229
2022-03-18T21:47:48Z DEBU [SSH:NODE]  Supported KEX algorithms: ["curve25519-sha256@libssh.org" "ecdh-sha2-nistp256" "ecdh-sha2-nistp384" "ecdh-sha2-nistp521"]. sshutils/server.go:239
2022-03-18T21:47:48Z DEBU [SSH:NODE]  Supported MAC algorithms: ["hmac-sha2-256-etm@openssh.com" "hmac-sha2-256"]. sshutils/server.go:249
2022-03-18T21:47:48Z DEBU [NODE:BEAT] Starting Node heartbeat with announce period: 1m0s, keep-alive period 5m41.464498981s, poll period: 5s srv/heartbeat.go:155
2022-03-18T21:47:48Z INFO [PROC:1]    Service node is creating new listener on 0.0.0.0:3022. service/signals.go:213
2022-03-18T21:47:48Z INFO [NODE:1]    Service 9.0.1:v9.0.1-0-g7bbe6f15c is starting on 0.0.0.0:3022 sqlite cache will store frequently accessed items. service/service.go:1986
2022-03-18T21:47:48Z INFO [NODE:1]    Service 9.0.1:v9.0.1-0-g7bbe6f15c is starting on 0.0.0.0:3022. utils/cli.go:275
[NODE]         Service 9.0.1:v9.0.1-0-g7bbe6f15c is starting on 0.0.0.0:3022.
2022-03-18T21:47:48Z DEBU [PROC:1]    Broadcasting event. event:NodeReady service/supervisor.go:370
2022-03-18T21:47:48Z DEBU [PROC:1]    Broadcasting mapped event. in:NodeReady out:EventMapping(in=[AuthTLSReady NodeReady ProxySSHReady], out=TeleportReady) service/supervisor.go:395
2022-03-18T21:47:48Z INFO [PROC:1]    The new service has started successfully. Starting syncing rotation status with period 10m0s. service/connect.go:469
2022-03-18T21:47:48Z DEBU [NODE:1]    Starting watch. resource-kind:lock services/watcher.go:182
2022-03-18T21:47:48Z DEBU [SSH:NODE]  Listening on [::]:3022. sshutils/server.go:382
2022-03-18T21:47:50Z DEBU [PROXY:1]   Attempting to restart watch after waiting 2.721042592s. resource-kind:proxy services/watcher.go:211
2022-03-18T21:47:50Z WARN [PROXY:1]   Restart watch on error: empty proxy list. resource-kind:proxy services/watcher.go:218
2022-03-18T21:47:50Z DEBU [PROXY:1]   Starting watch. resource-kind:proxy services/watcher.go:182
2022-03-18T21:47:50Z DEBU [PROXY:1]   List of known proxies updated: ["24d3f0e1-e243-43ae-ad69-b40748fed12f"]. services/watcher.go:421
2022-03-18T21:48:07Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:48:07Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:48:07Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:48:07Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:48:07Z DEBU [AUTH:1]    Server certificate cert(24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto issued by toronto:272982335074390935157946903432738681364). auth/middleware.go:293
2022-03-18T21:48:07Z DEBU [AUTH:1]    Server certificate cert(24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto issued by toronto:272982335074390935157946903432738681364). auth/middleware.go:293
2022-03-18T21:48:07Z INFO [AUDIT]     user.create cluster_name:toronto code:T1002I connector:local ei:0 event:user.create expires:0001-01-01T00:00:00Z name:admin roles:[editor access] time:2022-03-18T21:48:07.676Z uid:ec666556-a3f9-45c1-823d-384c0cef2cfb user:24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto events/emitter.go:325
2022-03-18T21:48:07Z INFO [AUTH]      "admin" user created auth/grpcserver.go:949
2022-03-18T21:48:07Z INFO [AUDIT]     reset_password_token.create cluster_name:toronto code:T6000I ei:0 event:reset_password_token.create expires:2022-03-18T22:48:07.789575851Z name:admin time:2022-03-18T21:48:07.79Z ttl:1h0m0s uid:1da3db0d-d74f-49ff-9718-f9dd043bd787 user:24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto events/emitter.go:325
2022-03-18T21:48:16Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:48:16Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:48:16Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:48:16Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:48:16Z WARN [ALPN:PROX] Failed to handle client connection. error:[
ERROR REPORT:
Original Error: *tls.permanentError remote error: tls: unknown certificate
Stack Trace:
  /go/src/github.com/gravitational/teleport/lib/srv/alpnproxy/proxy.go:344 github.com/gravitational/teleport/lib/srv/alpnproxy.(*Proxy).handleConn
  /go/src/github.com/gravitational/teleport/lib/srv/alpnproxy/proxy.go:282 github.com/gravitational/teleport/lib/srv/alpnproxy.(*Proxy).Serve.func1
  /opt/go/src/runtime/asm_amd64.s:1581 runtime.goexit
User Message: remote error: tls: unknown certificate] alpnproxy/proxy.go:290
2022-03-18T21:48:16Z WARN [ALPN:PROX] Failed to handle client connection. error:[
ERROR REPORT:
Original Error: *tls.permanentError remote error: tls: unknown certificate
Stack Trace:
  /go/src/github.com/gravitational/teleport/lib/srv/alpnproxy/proxy.go:344 github.com/gravitational/teleport/lib/srv/alpnproxy.(*Proxy).handleConn
  /go/src/github.com/gravitational/teleport/lib/srv/alpnproxy/proxy.go:282 github.com/gravitational/teleport/lib/srv/alpnproxy.(*Proxy).Serve.func1
  /opt/go/src/runtime/asm_amd64.s:1581 runtime.goexit
User Message: remote error: tls: unknown certificate] alpnproxy/proxy.go:290
2022-03-18T21:48:16Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:48:16Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:48:16Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:48:16Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:48:16Z WARN [ALPN:PROX] Failed to handle client connection. error:[
ERROR REPORT:
Original Error: *tls.permanentError remote error: tls: unknown certificate
Stack Trace:
  /go/src/github.com/gravitational/teleport/lib/srv/alpnproxy/proxy.go:344 github.com/gravitational/teleport/lib/srv/alpnproxy.(*Proxy).handleConn
  /go/src/github.com/gravitational/teleport/lib/srv/alpnproxy/proxy.go:282 github.com/gravitational/teleport/lib/srv/alpnproxy.(*Proxy).Serve.func1
  /opt/go/src/runtime/asm_amd64.s:1581 runtime.goexit
User Message: remote error: tls: unknown certificate] alpnproxy/proxy.go:290
2022-03-18T21:48:16Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:48:16Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:48:16Z WARN [ALPN:PROX] Failed to handle client connection. error:[
ERROR REPORT:
Original Error: *tls.permanentError remote error: tls: unknown certificate
Stack Trace:
  /go/src/github.com/gravitational/teleport/lib/srv/alpnproxy/proxy.go:344 github.com/gravitational/teleport/lib/srv/alpnproxy.(*Proxy).handleConn
  /go/src/github.com/gravitational/teleport/lib/srv/alpnproxy/proxy.go:282 github.com/gravitational/teleport/lib/srv/alpnproxy.(*Proxy).Serve.func1
  /opt/go/src/runtime/asm_amd64.s:1581 runtime.goexit
User Message: remote error: tls: unknown certificate] alpnproxy/proxy.go:290
2022-03-18T21:48:16Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:48:16Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:48:22Z DEBU [KEYGEN]    generated user key for [root] with expiry on (1647683302) 2022-03-19 09:48:22.838890746 +0000 UTC native/native.go:256
2022-03-18T21:48:22Z DEBU [AUTH]      Failed setting default kubernetes cluster for user login (user did not provide a cluster); leaving KubernetesCluster extension in the TLS certificate empty auth/auth.go:1089
2022-03-18T21:48:22Z INFO [CA]        Generating TLS certificate {0x8304640 0xc000f6d3c0 1.3.9999.1.7=#1306747572616e6f,CN=admin,O=editor+O=access,POSTALCODE={\"kubernetes_groups\":[\"\"]\,\"kubernetes_users\":[\"\"]\,\"logins\":[\"root\"]\,\"windows_logins\":null},STREET=toronto,L=root 2022-03-19 09:48:22.844667421 +0000 UTC [] [] 5 []}. common_name:admin dns_names:[] locality:[root] not_after:2022-03-19 09:48:22.844667421 +0000 UTC org:[editor access] org_unit:[] tlsca/ca.go:800
2022-03-18T21:48:22Z INFO [AUDIT]     cert.create cert_type:user cluster_name:toronto code:TC000I ei:0 event:cert.create identity:map[expires:2022-03-19T09:48:22.844667421Z logins:[root] roles:[editor access] route_to_cluster:toronto teleport_cluster:toronto traits:map[kubernetes_groups:[] kubernetes_users:[] logins:[root] windows_logins:<nil>] user:admin] time:2022-03-18T21:48:22.85Z uid:ec9e6dbc-7919-43db-b0b5-164afe6179b6 events/emitter.go:325
2022-03-18T21:48:22Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:48:22Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:48:22Z DEBU [AUTH:1]    Server certificate cert(24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto issued by toronto:272982335074390935157946903432738681364). auth/middleware.go:293
2022-03-18T21:48:23Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:48:23Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:48:23Z DEBU [AUTH:1]    Server certificate cert(24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto issued by toronto:272982335074390935157946903432738681364). auth/middleware.go:293
2022-03-18T21:48:34Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:48:34Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:48:34Z INFO [AUTH]      Node "001-toronto" [91b37407-ef75-4930-923a-658042b551b1] is trying to join with role: Node. auth/join.go:73
2022-03-18T21:48:34Z DEBU [KEYGEN]    Generated SSH host certificate for role Node with principals: [91b37407-ef75-4930-923a-658042b551b1.toronto 91b37407-ef75-4930-923a-658042b551b1 001-toronto.toronto 001-toronto localhost 127.0.0.1 ::1 toronto-spoke-001-dev]. native/native.go:231
2022-03-18T21:48:34Z INFO [CA]        Generating TLS certificate {0x8304640 0xc0016d9980 1.3.9999.1.7=#1306747572616e6f,CN=91b37407-ef75-4930-923a-658042b551b1.toronto,O=Node,POSTALCODE=null,STREET= 2032-03-15 21:48:34.86237987 +0000 UTC [001-toronto 91b37407-ef75-4930-923a-658042b551b1 toronto-spoke-001-dev] [] 5 []}. common_name:91b37407-ef75-4930-923a-658042b551b1.toronto dns_names:[001-toronto 91b37407-ef75-4930-923a-658042b551b1 toronto-spoke-001-dev] locality:[] not_after:2032-03-15 21:48:34.86237987 +0000 UTC org:[Node] org_unit:[] tlsca/ca.go:800
2022-03-18T21:48:34Z INFO [AUTH]      Node "001-toronto" [91b37407-ef75-4930-923a-658042b551b1] has joined the cluster. auth/join.go:165
2022-03-18T21:48:35Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:48:35Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:48:35Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:48:35Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:48:35Z WARN [ALPN:PROX] Failed to handle client connection. error:[
ERROR REPORT:
Original Error: *tls.permanentError remote error: tls: bad certificate
Stack Trace:
  /go/src/github.com/gravitational/teleport/lib/srv/alpnproxy/proxy.go:344 github.com/gravitational/teleport/lib/srv/alpnproxy.(*Proxy).handleConn
  /go/src/github.com/gravitational/teleport/lib/srv/alpnproxy/proxy.go:282 github.com/gravitational/teleport/lib/srv/alpnproxy.(*Proxy).Serve.func1
  /opt/go/src/runtime/asm_amd64.s:1581 runtime.goexit
User Message: remote error: tls: bad certificate] alpnproxy/proxy.go:290
2022-03-18T21:48:35Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:48:35Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:48:35Z WARN [ALPN:PROX] Failed to handle client connection. error:[
ERROR REPORT:
Original Error: *tls.permanentError remote error: tls: bad certificate
Stack Trace:
  /go/src/github.com/gravitational/teleport/lib/srv/alpnproxy/proxy.go:344 github.com/gravitational/teleport/lib/srv/alpnproxy.(*Proxy).handleConn
  /go/src/github.com/gravitational/teleport/lib/srv/alpnproxy/proxy.go:282 github.com/gravitational/teleport/lib/srv/alpnproxy.(*Proxy).Serve.func1
  /opt/go/src/runtime/asm_amd64.s:1581 runtime.goexit
User Message: remote error: tls: bad certificate] alpnproxy/proxy.go:290
2022-03-18T21:48:35Z INFO [AUTH]      Node "001-toronto" [91b37407-ef75-4930-923a-658042b551b1] is trying to join with role: App. auth/join.go:73
2022-03-18T21:48:35Z DEBU [KEYGEN]    Generated SSH host certificate for role App with principals: [91b37407-ef75-4930-923a-658042b551b1.toronto 91b37407-ef75-4930-923a-658042b551b1 001-toronto.toronto 001-toronto localhost 127.0.0.1 ::1]. native/native.go:231
2022-03-18T21:48:35Z INFO [CA]        Generating TLS certificate {0x8304640 0xc001a901c0 1.3.9999.1.7=#1306747572616e6f,CN=91b37407-ef75-4930-923a-658042b551b1.toronto,O=App,POSTALCODE=null,STREET= 2032-03-15 21:48:35.285079405 +0000 UTC [001-toronto 91b37407-ef75-4930-923a-658042b551b1 *.teleport.cluster.local teleport.cluster.local] [] 5 []}. common_name:91b37407-ef75-4930-923a-658042b551b1.toronto dns_names:[001-toronto 91b37407-ef75-4930-923a-658042b551b1 *.teleport.cluster.local teleport.cluster.local] locality:[] not_after:2032-03-15 21:48:35.285079405 +0000 UTC org:[App] org_unit:[] tlsca/ca.go:800
2022-03-18T21:48:35Z INFO [AUTH]      Node "001-toronto" [91b37407-ef75-4930-923a-658042b551b1] has joined the cluster. auth/join.go:165
2022-03-18T21:48:35Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:48:35Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:48:35Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:48:35Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:48:35Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:48:35Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:48:35Z WARN [ALPN:PROX] Failed to handle client connection. error:[
ERROR REPORT:
Original Error: *tls.permanentError remote error: tls: bad certificate
Stack Trace:
  /go/src/github.com/gravitational/teleport/lib/srv/alpnproxy/proxy.go:344 github.com/gravitational/teleport/lib/srv/alpnproxy.(*Proxy).handleConn
  /go/src/github.com/gravitational/teleport/lib/srv/alpnproxy/proxy.go:282 github.com/gravitational/teleport/lib/srv/alpnproxy.(*Proxy).Serve.func1
  /opt/go/src/runtime/asm_amd64.s:1581 runtime.goexit
User Message: remote error: tls: bad certificate] alpnproxy/proxy.go:290
2022-03-18T21:48:35Z WARN [ALPN:PROX] Failed to handle client connection. error:[
ERROR REPORT:
Original Error: *tls.permanentError remote error: tls: bad certificate
Stack Trace:
  /go/src/github.com/gravitational/teleport/lib/srv/alpnproxy/proxy.go:344 github.com/gravitational/teleport/lib/srv/alpnproxy.(*Proxy).handleConn
  /go/src/github.com/gravitational/teleport/lib/srv/alpnproxy/proxy.go:282 github.com/gravitational/teleport/lib/srv/alpnproxy.(*Proxy).Serve.func1
  /opt/go/src/runtime/asm_amd64.s:1581 runtime.goexit
User Message: remote error: tls: bad certificate] alpnproxy/proxy.go:290
2022-03-18T21:48:35Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:48:35Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:48:35Z DEBU [SSH:PROXY] Incoming connection 10.1.10.168:35228 -> 10.1.10.154:3024 version: SSH-2.0-Go, certtype: "host" sshutils/server.go:466
2022-03-18T21:48:35Z DEBU [SSH:PROXY] Incoming connection 10.1.10.168:35232 -> 10.1.10.154:3024 version: SSH-2.0-Go, certtype: "host" sshutils/server.go:466
2022-03-18T21:48:35Z DEBU [PROXY:1]   Transport request: teleport-transport. reversetunnel/srv.go:610
2022-03-18T21:48:35Z DEBU [PROXY:1]   Transport request: teleport-transport. reversetunnel/srv.go:610
2022-03-18T21:48:35Z DEBU [PROXY:1]   Received out-of-band proxy transport request for @remote-auth-server []. reversetunnel/transport.go:202
2022-03-18T21:48:35Z DEBU [PROXY:1]   Attempting to dial through tunnel with server ID "". reversetunnel/transport.go:370
2022-03-18T21:48:35Z DEBU [PROXY:1]   Attempting to dial directly [toronto-hub-000-dev:3025]. reversetunnel/transport.go:387
2022-03-18T21:48:35Z DEBU [PROXY:1]   Returning direct dialed connection to [toronto-hub-000-dev:3025]. reversetunnel/transport.go:393
2022-03-18T21:48:35Z DEBU [PROXY:1]   Successfully dialed to @remote-auth-server "", start proxying. reversetunnel/transport.go:306
2022-03-18T21:48:35Z DEBU [SSH:PROXY] Incoming connection 10.1.10.168:35234 -> 10.1.10.154:3024 version: SSH-2.0-Go, certtype: "host" sshutils/server.go:466
2022-03-18T21:48:35Z DEBU [PROXY:1]   Transport request: teleport-transport. reversetunnel/srv.go:610
2022-03-18T21:48:35Z DEBU [PROXY:1]   Received out-of-band proxy transport request for @remote-auth-server []. reversetunnel/transport.go:202
2022-03-18T21:48:35Z DEBU [PROXY:1]   Attempting to dial through tunnel with server ID "". reversetunnel/transport.go:370
2022-03-18T21:48:35Z DEBU [PROXY:1]   Attempting to dial directly [toronto-hub-000-dev:3025]. reversetunnel/transport.go:387
2022-03-18T21:48:35Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:48:35Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:48:35Z DEBU [PROXY:1]   Returning direct dialed connection to [toronto-hub-000-dev:3025]. reversetunnel/transport.go:393
2022-03-18T21:48:35Z DEBU [AUTH:1]    Server certificate cert(24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto issued by toronto:272982335074390935157946903432738681364). auth/middleware.go:293
2022-03-18T21:48:35Z DEBU [PROXY:1]   Successfully dialed to @remote-auth-server "", start proxying. reversetunnel/transport.go:306
2022-03-18T21:48:35Z DEBU [PROXY:1]   Received out-of-band proxy transport request for @remote-auth-server []. reversetunnel/transport.go:202
2022-03-18T21:48:35Z DEBU [PROXY:1]   Attempting to dial through tunnel with server ID "". reversetunnel/transport.go:370
2022-03-18T21:48:35Z DEBU [PROXY:1]   Attempting to dial directly [toronto-hub-000-dev:3025]. reversetunnel/transport.go:387
2022-03-18T21:48:35Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:48:35Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:48:35Z DEBU [AUTH:1]    Server certificate cert(24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto issued by toronto:272982335074390935157946903432738681364). auth/middleware.go:293
2022-03-18T21:48:35Z DEBU [SSH:PROXY] Incoming connection 10.1.10.168:35230 -> 10.1.10.154:3024 version: SSH-2.0-Go, certtype: "host" sshutils/server.go:466
2022-03-18T21:48:35Z DEBU [PROXY:1]   Returning direct dialed connection to [toronto-hub-000-dev:3025]. reversetunnel/transport.go:393
2022-03-18T21:48:35Z DEBU [PROXY:1]   Successfully dialed to @remote-auth-server "", start proxying. reversetunnel/transport.go:306
2022-03-18T21:48:35Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:48:35Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:48:35Z DEBU [AUTH:1]    Server certificate cert(24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto issued by toronto:272982335074390935157946903432738681364). auth/middleware.go:293
2022-03-18T21:48:35Z DEBU [PROXY:1]   Transport request: teleport-transport. reversetunnel/srv.go:610
2022-03-18T21:48:35Z DEBU [PROXY:1]   Received out-of-band proxy transport request for @remote-auth-server []. reversetunnel/transport.go:202
2022-03-18T21:48:35Z DEBU [PROXY:1]   Attempting to dial through tunnel with server ID "". reversetunnel/transport.go:370
2022-03-18T21:48:35Z DEBU [PROXY:1]   Attempting to dial directly [toronto-hub-000-dev:3025]. reversetunnel/transport.go:387
2022-03-18T21:48:35Z DEBU [PROXY:1]   Returning direct dialed connection to [toronto-hub-000-dev:3025]. reversetunnel/transport.go:393
2022-03-18T21:48:35Z DEBU [PROXY:1]   Successfully dialed to @remote-auth-server "", start proxying. reversetunnel/transport.go:306
2022-03-18T21:48:35Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:48:35Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:48:35Z DEBU [AUTH:1]    Server certificate cert(24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto issued by toronto:272982335074390935157946903432738681364). auth/middleware.go:293
2022-03-18T21:48:35Z DEBU [SSH:PROXY] Incoming connection 10.1.10.168:35238 -> 10.1.10.154:3024 version: SSH-2.0-Go, certtype: "host" sshutils/server.go:466
2022-03-18T21:48:35Z DEBU [PROXY:1]   New tunnel from 10.1.10.168:35238. reversetunnel/srv.go:633
2022-03-18T21:48:35Z DEBU [PROXY:SER] Ping <- 10.1.10.168:35238 nodeID:91b37407-ef75-4930-923a-658042b551b1.toronto trace.fields:map[cluster:toronto] reversetunnel/localsite.go:437
2022-03-18T21:48:35Z DEBU [DISCOVERY] Sending node discovery request with proxies ["24d3f0e1-e243-43ae-ad69-b40748fed12f"] to 10.1.10.168:35238. reversetunnel/conn.go:243
2022-03-18T21:48:36Z DEBU [SSH:PROXY] Incoming connection 10.1.10.168:35242 -> 10.1.10.154:3024 version: SSH-2.0-Go, certtype: "host" sshutils/server.go:466
2022-03-18T21:48:36Z DEBU [PROXY:1]   New tunnel from 10.1.10.168:35242. reversetunnel/srv.go:633
2022-03-18T21:48:36Z DEBU [PROXY:SER] Ping <- 10.1.10.168:35242 nodeID:91b37407-ef75-4930-923a-658042b551b1.toronto trace.fields:map[cluster:toronto] reversetunnel/localsite.go:437
2022-03-18T21:48:36Z DEBU [DISCOVERY] Sending app discovery request with proxies ["24d3f0e1-e243-43ae-ad69-b40748fed12f"] to 10.1.10.168:35242. reversetunnel/conn.go:243
2022-03-18T21:48:40Z DEBU [AUTH:GRPC] Failed to receive heartbeat: rpc error: code = Canceled desc = context canceled auth/grpcserver.go:143
2022-03-18T21:48:41Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:272982335074390935157946903432738681364) auth/middleware.go:612
2022-03-18T21:48:41Z DEBU [AUTH]      ClientCertPool -> cert(toronto issued by toronto:220232097174799242290983425395950007594) auth/middleware.go:612
2022-03-18T21:48:41Z DEBU [AUTH:1]    Server certificate cert(24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto issued by toronto:272982335074390935157946903432738681364). auth/middleware.go:293

NODE

INSTALL

sudo rm -rf /var/lib/teleport

cd ~
curl -O https://get.gravitational.com/teleport-v9.0.1-linux-amd64-bin.tar.gz
tar -xzf teleport-v9.0.1-linux-amd64-bin.tar.gz
cd teleport
sudo ./install

YAML

version: v2

teleport:
  nodename: 001-toronto
  
  data_dir: /var/lib/teleport

  advertise_ip: toronto-spoke-001-dev

  auth_token: veidoo0au3shiSie9aim7phieze0iic1

  auth_servers:
    - toronto-hub-000-dev:443
  
  #ca_pin: <pin>
  
  log:
    output: stderr
    severity: DEBUG
    format:
      output: text

auth_service:
  enabled: no
proxy_service:
  enabled: no
kubernetes_service:
  enabled: no
db_service:
  enabled: no
windows_desktop_service:
  enabled: no

ssh_service:
  enabled: yes
  enhanced_recording:
    enabled: false
  labels:
    svc: ssh
    id: 001
    owner: toronto
  commands:
    - name: host
      command: [hostname]
      period: 1m0s

app_service:
  enabled: yes
  debug_app: false
  apps:
    - name: "some-app-001-toronto"
      uri: "http://localhost:8080"
      insecure_skip_verify: false
      labels:
        svc: some-app
        id: 001
        owner: toronto

START

sudo teleport start --insecure

LOG

2022-03-18T21:48:34Z INFO             Generating new host UUID: 91b37407-ef75-4930-923a-658042b551b1. service/service.go:663
2022-03-18T21:48:34Z DEBU [SQLITE]    Connected to: file:/var/lib/teleport/proc/sqlite.db?_busy_timeout=10000&_sync=OFF, poll stream period: 1s lite/lite.go:169
2022-03-18T21:48:34Z DEBU [SQLITE]    Synchronous: 0, busy timeout: 10000 lite/lite.go:212
2022-03-18T21:48:34Z DEBU [KEYGEN]    SSH cert authority started with no keys pre-compute. native/native.go:106
2022-03-18T21:48:34Z DEBU [PROC:1]    Adding service to supervisor. service:register.node service/supervisor.go:201
2022-03-18T21:48:34Z DEBU [PROC:1]    Adding service to supervisor. service:ssh.node service/supervisor.go:201
2022-03-18T21:48:34Z DEBU [PROC:1]    Adding service to supervisor. service:ssh.shutdown service/supervisor.go:201
2022-03-18T21:48:34Z DEBU [PROC:1]    Adding service to supervisor. service:register.app service/supervisor.go:201
2022-03-18T21:48:34Z DEBU [PROC:1]    Adding service to supervisor. service:apps.start service/supervisor.go:201
2022-03-18T21:48:34Z DEBU [PROC:1]    Adding service to supervisor. service:apps.stop service/supervisor.go:201
2022-03-18T21:48:34Z DEBU [PROC:1]    Adding service to supervisor. service:common.rotate service/supervisor.go:201
2022-03-18T21:48:34Z DEBU [PROC:1]    Service has started. service:ssh.shutdown service/supervisor.go:262
2022-03-18T21:48:34Z DEBU [PROC:1]    No signal pipe to import, must be first Teleport process. service/service.go:882
2022-03-18T21:48:34Z DEBU [PROC:1]    Service has started. service:register.node service/supervisor.go:262
2022-03-18T21:48:34Z DEBU [PROC:1]    Service has started. service:register.app service/supervisor.go:262
2022-03-18T21:48:34Z DEBU [PROC:1]    Service has started. service:apps.start service/supervisor.go:262
2022-03-18T21:48:34Z DEBU [PROC:1]    Service has started. service:apps.stop service/supervisor.go:262
2022-03-18T21:48:34Z DEBU [PROC:1]    Service has started. service:common.rotate service/supervisor.go:262
2022-03-18T21:48:34Z INFO [PROC:1]    Joining the cluster with a secure token. service/connect.go:382
2022-03-18T21:48:34Z DEBU [PROC:1]    Generating new key pair for Node first-time-connect. service/connect.go:289
2022-03-18T21:48:34Z DEBU [PROC:1]    Service has started. service:ssh.node service/supervisor.go:262
2022-03-18T21:48:34Z INFO [PROC:1]    Joining the cluster with a secure token. service/connect.go:382
2022-03-18T21:48:34Z DEBU [AUTH]      Registering node to the cluster. auth-servers:[{toronto-hub-000-dev:443 tcp }] auth/register.go:153
2022-03-18T21:48:34Z DEBU [AUTH]      The first specified auth server appears to be a proxy. auth/register.go:167
2022-03-18T21:48:34Z INFO [AUTH]      Attempting registration via proxy server. auth/register.go:173
2022-03-18T21:48:34Z DEBU [CLIENT]    HTTPS client init(proxyAddr=toronto-hub-000-dev:443, insecure=true) client/weblogin.go:223
WARNING: You are using insecure connection to SSH proxy https://toronto-hub-000-dev:443
2022-03-18T21:48:34Z DEBU [CLIENT]    Attempting https://toronto-hub-000-dev:443/v1/webapi/host/credentials client/https_client.go:85
2022-03-18T21:48:34Z DEBU [PROC:1]    Generating new key pair for App first-time-connect. service/connect.go:289
2022-03-18T21:48:34Z INFO [AUTH]      Successfully registered via proxy server. auth/register.go:180
2022-03-18T21:48:35Z DEBU [AUTH]      Registering node to the cluster. auth-servers:[{toronto-hub-000-dev:443 tcp }] auth/register.go:153
2022-03-18T21:48:35Z DEBU [AUTH]      The first specified auth server appears to be a proxy. auth/register.go:167
2022-03-18T21:48:35Z INFO [AUTH]      Attempting registration via proxy server. auth/register.go:173
2022-03-18T21:48:35Z DEBU [CLIENT]    HTTPS client init(proxyAddr=toronto-hub-000-dev:443, insecure=true) client/weblogin.go:223
WARNING: You are using insecure connection to SSH proxy https://toronto-hub-000-dev:443
2022-03-18T21:48:35Z DEBU [CLIENT]    Attempting https://toronto-hub-000-dev:443/v1/webapi/host/credentials client/https_client.go:85
2022-03-18T21:48:35Z DEBU [PROC:1]    Deleted generated key pair Node first-time-connect. service/connect.go:275
2022-03-18T21:48:35Z INFO [PROC:1]    Node has obtained credentials to connect to the cluster. service/connect.go:416
2022-03-18T21:48:35Z DEBU [PROC:1]    Attempting to connect to Auth Server directly. auth-addrs:[toronto-hub-000-dev:443] service/connect.go:850
2022-03-18T21:48:35Z DEBU [PROC:1]    Failed to connect to Auth Server directly. auth-addrs:[toronto-hub-000-dev:443] service/connect.go:856
2022-03-18T21:48:35Z DEBU [PROC:1]    Attempting to discover reverse tunnel address. auth-addrs:[toronto-hub-000-dev:443] service/connect.go:865
2022-03-18T21:48:35Z DEBU [PROC:1]    Attempting to connect to Auth Server through tunnel. auth-addrs:[toronto-hub-000-dev:443] service/connect.go:867
2022-03-18T21:48:35Z DEBU             Attempting GET toronto-hub-000-dev:443/webapi/find webclient/webclient.go:62
2022-03-18T21:48:35Z INFO [AUTH]      Successfully registered via proxy server. auth/register.go:180
2022-03-18T21:48:35Z DEBU [PROC:1]    Deleted generated key pair App first-time-connect. service/connect.go:275
2022-03-18T21:48:35Z INFO [PROC:1]    App has obtained credentials to connect to the cluster. service/connect.go:416
2022-03-18T21:48:35Z DEBU [PROC:1]    Attempting to connect to Auth Server directly. auth-addrs:[toronto-hub-000-dev:443] service/connect.go:850
2022-03-18T21:48:35Z DEBU             Attempting GET toronto-hub-000-dev:3024/webapi/find webclient/webclient.go:62
2022-03-18T21:48:35Z DEBU             Attempting GET toronto-hub-000-dev:3024/webapi/find webclient/webclient.go:62
2022-03-18T21:48:35Z DEBU [PROC:1]    Failed to connect to Auth Server directly. auth-addrs:[toronto-hub-000-dev:443] service/connect.go:856
2022-03-18T21:48:35Z DEBU [PROC:1]    Attempting to discover reverse tunnel address. auth-addrs:[toronto-hub-000-dev:443] service/connect.go:865
2022-03-18T21:48:35Z DEBU [PROC:1]    Attempting to connect to Auth Server through tunnel. auth-addrs:[toronto-hub-000-dev:443] service/connect.go:867
2022-03-18T21:48:35Z DEBU             Attempting GET toronto-hub-000-dev:443/webapi/find webclient/webclient.go:62
2022-03-18T21:48:35Z DEBU             Attempting GET toronto-hub-000-dev:3024/webapi/find webclient/webclient.go:62
2022-03-18T21:48:35Z DEBU             Attempting GET toronto-hub-000-dev:3024/webapi/find webclient/webclient.go:62
2022-03-18T21:48:35Z DEBU [PROC:1]    Failed to ping web proxy "toronto-hub-000-dev:3024" addr: Get "https://toronto-hub-000-dev:3024/webapi/find": tls: first record does not look like a TLS handshake reversetunnel/transport.go:97
2022-03-18T21:48:35Z DEBU [HTTP:PROX] No valid environment variables found. proxy/proxy.go:337
2022-03-18T21:48:35Z DEBU [HTTP:PROX] No proxy set in environment, returning direct dialer. proxy/proxy.go:244
2022-03-18T21:48:35Z DEBU [PROC:1]    Failed to ping web proxy "toronto-hub-000-dev:3024" addr: Get "https://toronto-hub-000-dev:3024/webapi/find": tls: first record does not look like a TLS handshake reversetunnel/transport.go:97
2022-03-18T21:48:35Z DEBU [HTTP:PROX] No valid environment variables found. proxy/proxy.go:337
2022-03-18T21:48:35Z DEBU [HTTP:PROX] No proxy set in environment, returning direct dialer. proxy/proxy.go:244
2022-03-18T21:48:35Z DEBU [PROC:1]    Failed to ping web proxy "toronto-hub-000-dev:3024" addr: Get "https://toronto-hub-000-dev:3024/webapi/find": tls: first record does not look like a TLS handshake reversetunnel/transport.go:97
2022-03-18T21:48:35Z DEBU [HTTP:PROX] No valid environment variables found. proxy/proxy.go:337
2022-03-18T21:48:35Z DEBU [HTTP:PROX] No proxy set in environment, returning direct dialer. proxy/proxy.go:244
2022-03-18T21:48:35Z DEBU [PROC:1]    Failed to ping web proxy "toronto-hub-000-dev:3024" addr: Get "https://toronto-hub-000-dev:3024/webapi/find": tls: first record does not look like a TLS handshake reversetunnel/transport.go:97
2022-03-18T21:48:35Z DEBU [HTTP:PROX] No valid environment variables found. proxy/proxy.go:337
2022-03-18T21:48:35Z DEBU [HTTP:PROX] No proxy set in environment, returning direct dialer. proxy/proxy.go:244
2022-03-18T21:48:35Z DEBU [PROC:1]    Connected to Auth Server through tunnel. auth-addrs:[toronto-hub-000-dev:443] service/connect.go:879
2022-03-18T21:48:35Z DEBU [PROC:1]    Connected to Auth Server through tunnel. auth-addrs:[toronto-hub-000-dev:443] service/connect.go:879
2022-03-18T21:48:35Z INFO [PROC:1]    The process successfully wrote the credentials and state of App to the disk. service/connect.go:457
2022-03-18T21:48:35Z DEBU [PROC:1]    Connected client: Identity(App, cert(91b37407-ef75-4930-923a-658042b551b1.toronto issued by toronto:272982335074390935157946903432738681364),trust root(toronto:272982335074390935157946903432738681364)) service/connect.go:124
2022-03-18T21:48:35Z INFO [PROC:1]    The process successfully wrote the credentials and state of Node to the disk. service/connect.go:457
2022-03-18T21:48:35Z DEBU [PROC:1]    Connected client: Identity(Node, cert(91b37407-ef75-4930-923a-658042b551b1.toronto issued by toronto:272982335074390935157946903432738681364),trust root(toronto:272982335074390935157946903432738681364)) service/connect.go:124
2022-03-18T21:48:35Z INFO [PROC:1]    App: features loaded from auth server: Kubernetes:true App:true DB:true Desktop:true  service/connect.go:71
2022-03-18T21:48:35Z DEBU [PROC:1]    Broadcasting event. event:AppsIdentity service/supervisor.go:370
2022-03-18T21:48:35Z DEBU [PROC:1]    Service is completed and removed. service:register.app service/supervisor.go:239
2022-03-18T21:48:35Z DEBU [APP:SERVI] Received event "AppsIdentity". service/service.go:3526
2022-03-18T21:48:35Z DEBU [PROC:1]    Creating sqlite backend for [app:service:1]. service/service.go:1573
2022-03-18T21:48:35Z DEBU [SQLITE]    Connected to: file:/var/lib/teleport/cache/app:service:1/sqlite.db?_busy_timeout=10000&_sync=OFF, poll stream period: 100ms lite/lite.go:169
2022-03-18T21:48:35Z INFO [PROC:1]    Node: features loaded from auth server: Kubernetes:true App:true DB:true Desktop:true  service/connect.go:71
2022-03-18T21:48:35Z DEBU [PROC:1]    Broadcasting event. event:SSHIdentity service/supervisor.go:370
2022-03-18T21:48:35Z DEBU [PROC:1]    Service is completed and removed. service:register.node service/supervisor.go:239
2022-03-18T21:48:35Z DEBU [NODE:1]    Received event "SSHIdentity". service/service.go:1811
2022-03-18T21:48:35Z DEBU [PROC:1]    Creating sqlite backend for [node]. service/service.go:1573
2022-03-18T21:48:35Z DEBU [SQLITE]    Connected to: file:/var/lib/teleport/cache/node/sqlite.db?_busy_timeout=10000&_sync=OFF, poll stream period: 100ms lite/lite.go:169
2022-03-18T21:48:35Z DEBU [SQLITE]    Synchronous: 0, busy timeout: 10000 lite/lite.go:212
2022-03-18T21:48:35Z DEBU [SQLITE]    Synchronous: 0, busy timeout: 10000 lite/lite.go:212
2022-03-18T21:48:35Z INFO [NODE:1:CA] Cache "node" first init succeeded. cache/cache.go:742
2022-03-18T21:48:35Z DEBU [BPF]       Enhanced session recording is not enabled, skipping. bpf/bpf.go:124
2022-03-18T21:48:35Z DEBU [RESTRICTE] Restricted session is not enabled, skipping. restrictedsession/restricted.go:93
2022-03-18T21:48:35Z DEBU [SSH:NODE]  Supported ciphers: ["aes128-gcm@openssh.com" "chacha20-poly1305@openssh.com" "aes128-ctr" "aes192-ctr" "aes256-ctr"]. sshutils/server.go:229
2022-03-18T21:48:35Z DEBU [SSH:NODE]  Supported KEX algorithms: ["curve25519-sha256@libssh.org" "ecdh-sha2-nistp256" "ecdh-sha2-nistp384" "ecdh-sha2-nistp521"]. sshutils/server.go:239
2022-03-18T21:48:35Z DEBU [SSH:NODE]  Supported MAC algorithms: ["hmac-sha2-256-etm@openssh.com" "hmac-sha2-256"]. sshutils/server.go:249
2022-03-18T21:48:35Z DEBU [NODE:1]    Starting watch. resource-kind:lock services/watcher.go:182
2022-03-18T21:48:35Z DEBU [NODE:BEAT] Starting Node heartbeat with announce period: 1m0s, keep-alive period 5m12.335659159s, poll period: 5s srv/heartbeat.go:155
2022-03-18T21:48:35Z INFO [AUDIT:1]   Creating directory /var/lib/teleport/log. service/service.go:2115
2022-03-18T21:48:35Z INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload. service/service.go:2115
2022-03-18T21:48:35Z INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload/sessions. service/service.go:2115
2022-03-18T21:48:35Z INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload/sessions/default. service/service.go:2115
2022-03-18T21:48:35Z INFO [AUDIT:1]   Creating directory /var/lib/teleport/log. service/service.go:2115
2022-03-18T21:48:35Z INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload. service/service.go:2115
2022-03-18T21:48:35Z INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload/streaming. service/service.go:2115
2022-03-18T21:48:35Z INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload/streaming/default. service/service.go:2115
2022-03-18T21:48:35Z DEBU [PROC:1]    Adding service to supervisor. service:fileuploader.service service/supervisor.go:201
2022-03-18T21:48:35Z DEBU [PROC:1]    Adding service to supervisor. service:fileuploader.shutdown service/supervisor.go:201
2022-03-18T21:48:35Z DEBU [PROXY:AGE] Starting agent pool 91b37407-ef75-4930-923a-658042b551b1.toronto.toronto... cluster:toronto reversetunnel/agentpool.go:164
2022-03-18T21:48:35Z INFO [NODE:1]    Service is starting in tunnel mode. service/service.go:2025
2022-03-18T21:48:35Z DEBU [PROC:1]    Broadcasting event. event:NodeReady service/supervisor.go:370
2022-03-18T21:48:35Z DEBU [PROC:1]    Service has started. service:fileuploader.service service/supervisor.go:262
2022-03-18T21:48:35Z DEBU [PROC:1]    Service has started. service:fileuploader.shutdown service/supervisor.go:262
2022-03-18T21:48:35Z INFO [APP:SERVI] Cache "apps" first init succeeded. cache/cache.go:742
2022-03-18T21:48:35Z INFO [AUDIT:1]   Creating directory /var/lib/teleport/log. service/service.go:2115
2022-03-18T21:48:35Z INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload. service/service.go:2115
2022-03-18T21:48:35Z INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload/sessions. service/service.go:2115
2022-03-18T21:48:35Z INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload/sessions/default. service/service.go:2115
2022-03-18T21:48:35Z INFO [AUDIT:1]   Creating directory /var/lib/teleport/log. service/service.go:2115
2022-03-18T21:48:35Z INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload. service/service.go:2115
2022-03-18T21:48:35Z INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload/streaming. service/service.go:2115
2022-03-18T21:48:35Z INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload/streaming/default. service/service.go:2115
2022-03-18T21:48:35Z DEBU [PROC:1]    Adding service to supervisor. service:fileuploader.service service/supervisor.go:201
2022-03-18T21:48:35Z DEBU [PROC:1]    Adding service to supervisor. service:fileuploader.shutdown service/supervisor.go:201
2022-03-18T21:48:35Z DEBU [PROXY:AGE] Adding agent(leaseID=1,state=connecting) -> toronto:toronto-hub-000-dev:3024. cluster:toronto reversetunnel/agentpool.go:308
2022-03-18T21:48:35Z DEBU             Attempting GET toronto-hub-000-dev:3024/webapi/find webclient/webclient.go:62
2022-03-18T21:48:35Z DEBU [PROC:1]    Service has started. service:fileuploader.shutdown service/supervisor.go:262
2022-03-18T21:48:35Z DEBU [PROC:1]    Service has started. service:fileuploader.service service/supervisor.go:262
2022-03-18T21:48:35Z INFO [NODE:PROX] Failed to ping web proxy "toronto-hub-000-dev:3024" addr: Get "https://toronto-hub-000-dev:3024/webapi/find": tls: first record does not look like a TLS handshake leaseID:1 target:toronto-hub-000-dev:3024 reversetunnel/agent.go:269
2022-03-18T21:48:35Z DEBU [HTTP:PROX] No valid environment variables found. proxy/proxy.go:337
2022-03-18T21:48:35Z DEBU [HTTP:PROX] No proxy set in environment, returning direct dialer. proxy/proxy.go:244
2022-03-18T21:48:35Z INFO [NODE:PROX] Connected. addr:10.1.10.168:35238 remote-addr:10.1.10.154:3024 leaseID:1 target:toronto-hub-000-dev:3024 reversetunnel/agent.go:403
2022-03-18T21:48:35Z DEBU [NODE:PROX] Agent connected to proxy: [24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto 24d3f0e1-e243-43ae-ad69-b40748fed12f 000-toronto.toronto 000-toronto localhost 127.0.0.1 ::1 toronto-hub-000-dev remote.kube.proxy.teleport.cluster.local]. leaseID:1 target:toronto-hub-000-dev:3024 reversetunnel/agent.go:408
2022-03-18T21:48:35Z DEBU [NODE:PROX] Changing state connecting -> connected. leaseID:1 target:toronto-hub-000-dev:3024 reversetunnel/agent.go:208
2022-03-18T21:48:35Z DEBU [NODE:PROX] Discovery request channel opened: teleport-discovery. leaseID:1 target:toronto-hub-000-dev:3024 reversetunnel/agent.go:520
2022-03-18T21:48:35Z DEBU [NODE:PROX] handleDiscovery requests channel. leaseID:1 target:toronto-hub-000-dev:3024 reversetunnel/agent.go:538
2022-03-18T21:48:36Z DEBU [APP:SERVI] Starting watch. resource-kind:lock services/watcher.go:182
2022-03-18T21:48:36Z DEBU [APP:SERVI] Starting App heartbeat with announce period: 1m0s, keep-alive period 5m43.464651663s, poll period: 5s srv/heartbeat.go:155
2022-03-18T21:48:36Z DEBU [APP:SERVI] Started App(Name=dozzle-001-toronto, PublicAddr=dozzle-001-toronto.toronto-hub-000-dev, Labels=map[id:001 owner:toronto svc:dozzle teleport.dev/origin:config-file]). app/server.go:279
2022-03-18T21:48:36Z DEBU [APP:SERVI] Not initializing application resource watcher. app/watcher.go:68
2022-03-18T21:48:36Z DEBU [PROXY:AGE] Starting agent pool 91b37407-ef75-4930-923a-658042b551b1.toronto.toronto... cluster:toronto reversetunnel/agentpool.go:164
2022-03-18T21:48:36Z DEBU [PROC:1]    Broadcasting event. event:AppsReady service/supervisor.go:370
2022-03-18T21:48:36Z DEBU [PROC:1]    Broadcasting mapped event. in:AppsReady out:EventMapping(in=[NodeReady AppsReady], out=TeleportReady) service/supervisor.go:395
2022-03-18T21:48:36Z INFO [APP:SERVI] All applications successfully started. service/service.go:3698
2022-03-18T21:48:36Z INFO [PROC:1]    The new service has started successfully. Starting syncing rotation status with period 10m0s. service/connect.go:469
2022-03-18T21:48:36Z DEBU [PROXY:AGE] Adding agent(leaseID=1,state=connecting) -> toronto:toronto-hub-000-dev:3024. cluster:toronto reversetunnel/agentpool.go:308
2022-03-18T21:48:36Z DEBU             Attempting GET toronto-hub-000-dev:3024/webapi/find webclient/webclient.go:62
2022-03-18T21:48:36Z INFO [APP:SERVI] Failed to ping web proxy "toronto-hub-000-dev:3024" addr: Get "https://toronto-hub-000-dev:3024/webapi/find": tls: first record does not look like a TLS handshake leaseID:1 target:toronto-hub-000-dev:3024 reversetunnel/agent.go:269
2022-03-18T21:48:36Z DEBU [HTTP:PROX] No valid environment variables found. proxy/proxy.go:337
2022-03-18T21:48:36Z DEBU [HTTP:PROX] No proxy set in environment, returning direct dialer. proxy/proxy.go:244
2022-03-18T21:48:36Z INFO [APP:SERVI] Connected. addr:10.1.10.168:35242 remote-addr:10.1.10.154:3024 leaseID:1 target:toronto-hub-000-dev:3024 reversetunnel/agent.go:403
2022-03-18T21:48:36Z DEBU [APP:SERVI] Agent connected to proxy: [24d3f0e1-e243-43ae-ad69-b40748fed12f.toronto 24d3f0e1-e243-43ae-ad69-b40748fed12f 000-toronto.toronto 000-toronto localhost 127.0.0.1 ::1 toronto-hub-000-dev remote.kube.proxy.teleport.cluster.local]. leaseID:1 target:toronto-hub-000-dev:3024 reversetunnel/agent.go:408
2022-03-18T21:48:36Z DEBU [APP:SERVI] Changing state connecting -> connected. leaseID:1 target:toronto-hub-000-dev:3024 reversetunnel/agent.go:208
2022-03-18T21:48:36Z DEBU [APP:SERVI] Discovery request channel opened: teleport-discovery. leaseID:1 target:toronto-hub-000-dev:3024 reversetunnel/agent.go:520
2022-03-18T21:48:36Z DEBU [APP:SERVI] handleDiscovery requests channel. leaseID:1 target:toronto-hub-000-dev:3024 reversetunnel/agent.go:538

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Self-hosted, self-signed installation #11268

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 0 comments

Select a reply

Uh oh!

Self-hosted, self-signed installation #11268

Uh oh!

Uh oh!

MRIIOT Mar 18, 2022

PROXY+AUTH

INSTALL

GENERATE KEYS

YAML

START AND CREATE USER

LOG

NODE

INSTALL

YAML

START

LOG

Replies: 0 comments

MRIIOT
Mar 18, 2022