Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Application Access does not support websockets #5264

Closed
webvictim opened this issue Jan 11, 2021 · 13 comments · Fixed by #6028 or #6072
Closed

Application Access does not support websockets #5264

webvictim opened this issue Jan 11, 2021 · 13 comments · Fixed by #6028 or #6072
Assignees
Labels
application-access bug coffee Sasha tracks these mission critical issues. feature-request Used for new features in Teleport, improvements to current should be #enhancements
Milestone

Comments

@webvictim
Copy link
Contributor

What happened: Trying to establish a websocket connection through AAP does not allow websockets to connect.

What you expected to happen: Websockets should work.

How to reproduce it (as minimally and precisely as possible): Deploy something that requires websockets and proxy it through AAP. It will not work.

For my testing I used Node-Red. Deploy it in Docker using default port 1880. Set up AAP like so:

/etc/teleport.yaml:

app_service:
  enabled: yes
  apps:
  - name: nodered
    public_addr: nodered.teleport.example.com
    uri: http://hades:1880

Connecting to https://nodered.teleport.example.com through a browser will load the UI, but websocket connections will fail. You can see this in Chrome dev tools:

Screenshot 2021-01-11 at 10 29 39

For contrast, here's the websocket traffic working correctly using Caddy:

Screenshot 2021-01-11 at 10 25 00

I can see no relevant logs outputted from Teleport during these attempted websocket connections, even at DEBUG level. Regardless, here is the full log section for the connection from start to finish:

Jan 11 10:31:41 hades teleport[436900]: DEBU [APP:SERVI] Transport request: teleport-transport. leaseID:1 target:teleport.example.com:3080 reversetunnel/agent.go:428
Jan 11 10:31:41 hades teleport[436900]: DEBU [APP:SERVI] Received out-of-band proxy transport request for @local-node [0065053d-e8e9-4880-aebb-e2d9499b681e.teleport.example.com]. leaseID:1 target:teleport.example.com:3080 reversetunnel/transport.go:229
Jan 11 10:31:41 hades teleport[436900]: DEBU [APP:SERVI] Handing off connection to a local SSH service leaseID:1 target:teleport.example.com:3080 reversetunnel/transport.go:301
Jan 11 10:31:41 hades teleport[436900]: DEBU [AUTH]      ClientCertPool -> cert(teleport.example.com issued by teleport.example.com:134459308287134350927232826641598365919) auth/middleware.go:569
Jan 11 10:31:41 hades teleport[436900]: DEBU [AUTH]      ClientCertPool -> cert(teleport.example.com issued by teleport.example.com:125861568327795827078182794666495814353) auth/middleware.go:569
Jan 11 10:31:41 hades teleport[436900]: DEBU [HTTP:PROX] No valid environment variables found. proxy/proxy.go:222
Jan 11 10:31:41 hades teleport[436900]: DEBU [HTTP:PROX] No proxy set in environment, returning direct dialer. proxy/proxy.go:137
Jan 11 10:31:41 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:41 hades teleport[436900]: DEBU [APP:SERVI] Using async streamer for session e62b9e05-1930-471e-964a-901cc4a9877c. app/session.go:169
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /, code: 304, duration: 3.836414ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU [APP:SERVI] Transport request: teleport-transport. leaseID:1 target:teleport.example.com:3080 reversetunnel/agent.go:428
Jan 11 10:31:42 hades teleport[436900]: DEBU [APP:SERVI] Received out-of-band proxy transport request for @local-node [0065053d-e8e9-4880-aebb-e2d9499b681e.teleport.example.com]. leaseID:1 target:teleport.example.com:3080 reversetunnel/transport.go:229
Jan 11 10:31:42 hades teleport[436900]: DEBU [APP:SERVI] Handing off connection to a local SSH service leaseID:1 target:teleport.example.com:3080 reversetunnel/transport.go:301
Jan 11 10:31:42 hades teleport[436900]: DEBU [APP:SERVI] Transport request: teleport-transport. leaseID:1 target:teleport.example.com:3080 reversetunnel/agent.go:428
Jan 11 10:31:42 hades teleport[436900]: DEBU [APP:SERVI] Received out-of-band proxy transport request for @local-node [0065053d-e8e9-4880-aebb-e2d9499b681e.teleport.example.com]. leaseID:1 target:teleport.example.com:3080 reversetunnel/transport.go:229
Jan 11 10:31:42 hades teleport[436900]: DEBU [APP:SERVI] Handing off connection to a local SSH service leaseID:1 target:teleport.example.com:3080 reversetunnel/transport.go:301
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: DEBU [AUTH]      ClientCertPool -> cert(teleport.example.com issued by teleport.example.com:134459308287134350927232826641598365919) auth/middleware.go:569
Jan 11 10:31:42 hades teleport[436900]: DEBU [AUTH]      ClientCertPool -> cert(teleport.example.com issued by teleport.example.com:125861568327795827078182794666495814353) auth/middleware.go:569
Jan 11 10:31:42 hades teleport[436900]: DEBU [AUTH]      ClientCertPool -> cert(teleport.example.com issued by teleport.example.com:134459308287134350927232826641598365919) auth/middleware.go:569
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /vendor/jquery/css/base/jquery-ui.min.css, code: 304, duration: 1.907483ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU [AUTH]      ClientCertPool -> cert(teleport.example.com issued by teleport.example.com:125861568327795827078182794666495814353) auth/middleware.go:569
Jan 11 10:31:42 hades teleport[436900]: DEBU [APP:SERVI] Transport request: teleport-transport. leaseID:1 target:teleport.example.com:3080 reversetunnel/agent.go:428
Jan 11 10:31:42 hades teleport[436900]: DEBU [APP:SERVI] Received out-of-band proxy transport request for @local-node [0065053d-e8e9-4880-aebb-e2d9499b681e.teleport.example.com]. leaseID:1 target:teleport.example.com:3080 reversetunnel/transport.go:229
Jan 11 10:31:42 hades teleport[436900]: DEBU [APP:SERVI] Handing off connection to a local SSH service leaseID:1 target:teleport.example.com:3080 reversetunnel/transport.go:301
Jan 11 10:31:42 hades teleport[436900]: DEBU [HTTP:PROX] No valid environment variables found. proxy/proxy.go:222
Jan 11 10:31:42 hades teleport[436900]: DEBU [HTTP:PROX] No proxy set in environment, returning direct dialer. proxy/proxy.go:137
Jan 11 10:31:42 hades teleport[436900]: DEBU [HTTP:PROX] No valid environment variables found. proxy/proxy.go:222
Jan 11 10:31:42 hades teleport[436900]: DEBU [HTTP:PROX] No proxy set in environment, returning direct dialer. proxy/proxy.go:137
Jan 11 10:31:42 hades teleport[436900]: DEBU [APP:SERVI] Transport request: teleport-transport. leaseID:1 target:teleport.example.com:3080 reversetunnel/agent.go:428
Jan 11 10:31:42 hades teleport[436900]: DEBU [APP:SERVI] Received out-of-band proxy transport request for @local-node [0065053d-e8e9-4880-aebb-e2d9499b681e.teleport.example.com]. leaseID:1 target:teleport.example.com:3080 reversetunnel/transport.go:229
Jan 11 10:31:42 hades teleport[436900]: DEBU [APP:SERVI] Handing off connection to a local SSH service leaseID:1 target:teleport.example.com:3080 reversetunnel/transport.go:301
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: DEBU [AUTH]      ClientCertPool -> cert(teleport.example.com issued by teleport.example.com:134459308287134350927232826641598365919) auth/middleware.go:569
Jan 11 10:31:42 hades teleport[436900]: DEBU [AUTH]      ClientCertPool -> cert(teleport.example.com issued by teleport.example.com:125861568327795827078182794666495814353) auth/middleware.go:569
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /red/style.min.css, code: 304, duration: 1.838274ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU [AUTH]      ClientCertPool -> cert(teleport.example.com issued by teleport.example.com:134459308287134350927232826641598365919) auth/middleware.go:569
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: DEBU [AUTH]      ClientCertPool -> cert(teleport.example.com issued by teleport.example.com:125861568327795827078182794666495814353) auth/middleware.go:569
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /vendor/font-awesome/css/font-awesome.min.css, code: 304, duration: 2.012946ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /vendor/vendor.js, code: 304, duration: 2.316669ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU [HTTP:PROX] No valid environment variables found. proxy/proxy.go:222
Jan 11 10:31:42 hades teleport[436900]: DEBU [HTTP:PROX] No proxy set in environment, returning direct dialer. proxy/proxy.go:137
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /red/main.min.js, code: 304, duration: 1.86439ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /red/red.min.js, code: 304, duration: 1.672763ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /vendor/purify.min.js.map, code: 404, duration: 3.858245ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU [APP:SERVI] Transport request: teleport-transport. leaseID:1 target:teleport.example.com:3080 reversetunnel/agent.go:428
Jan 11 10:31:42 hades teleport[436900]: DEBU [APP:SERVI] Received out-of-band proxy transport request for @local-node [0065053d-e8e9-4880-aebb-e2d9499b681e.teleport.example.com]. leaseID:1 target:teleport.example.com:3080 reversetunnel/transport.go:229
Jan 11 10:31:42 hades teleport[436900]: DEBU [APP:SERVI] Handing off connection to a local SSH service leaseID:1 target:teleport.example.com:3080 reversetunnel/transport.go:301
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: DEBU [AUTH]      ClientCertPool -> cert(teleport.example.com issued by teleport.example.com:134459308287134350927232826641598365919) auth/middleware.go:569
Jan 11 10:31:42 hades teleport[436900]: DEBU [AUTH]      ClientCertPool -> cert(teleport.example.com issued by teleport.example.com:125861568327795827078182794666495814353) auth/middleware.go:569
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /locales/editor?lng=en-GB, code: 304, duration: 2.788147ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /theme, code: 304, duration: 1.550436ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /locales/node-red?lng=en-GB, code: 304, duration: 2.432359ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /locales/jsonata?lng=en-GB, code: 304, duration: 5.126189ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /locales/editor?lng=en-US, code: 304, duration: 6.561891ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /locales/node-red?lng=en-US, code: 304, duration: 2.726082ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /locales/jsonata?lng=en-US, code: 304, duration: 4.077259ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /locales/infotips?lng=en-GB, code: 304, duration: 5.5184ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /locales/infotips?lng=en-US, code: 304, duration: 5.265811ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /red/images/node-red.svg, code: 304, duration: 2.286183ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /favicon.ico, code: 200, duration: 6.999832ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /settings?_=1610375502103, code: 200, duration: 1.081867ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /settings/user?_=1610375502104, code: 200, duration: 972.548µs tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /red/images/spin.svg, code: 304, duration: 3.144964ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /red/keymap.json, code: 304, duration: 3.607199ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /red/images/deploy-full-o.svg, code: 304, duration: 1.87505ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /red/images/deploy-full.svg, code: 304, duration: 1.681191ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /red/images/deploy-flows.svg, code: 304, duration: 1.955782ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /red/images/grip.png, code: 304, duration: 1.754156ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /red/images/deploy-reload.svg, code: 304, duration: 1.562247ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /red/images/deploy-nodes.svg, code: 304, duration: 1.619736ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /vendor/font-awesome/fonts/fontawesome-webfont.woff2?v=4.7.0, code: 304, duration: 1.732088ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /nodes?_=1610375502105, code: 200, duration: 2.952736ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /nodes/messages?lng=en-GB&_=1610375502106, code: 200, duration: 3.240719ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /nodes/messages?lng=en-US&_=1610375502107, code: 200, duration: 3.845578ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons?_=1610375502108, code: 200, duration: 1.861736ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /nodes?_=1610375502109, code: 200, duration: 8.279227ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /debug/view/debug-utils.js, code: 304, duration: 3.007197ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/debug.svg, code: 304, duration: 2.500075ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/inject.svg, code: 304, duration: 2.110012ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/status.svg, code: 304, duration: 2.935912ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/alert.svg, code: 304, duration: 3.076684ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/link-out.svg, code: 304, duration: 2.517856ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/comment.svg, code: 304, duration: 2.356024ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /flows?_=1610375502110, code: 200, duration: 2.535374ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/trigger.svg, code: 304, duration: 2.47691ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/template.svg, code: 304, duration: 2.22044ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/timer.svg, code: 304, duration: 2.177253ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/cog.svg, code: 304, duration: 2.346898ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red-node-rbe/rbe.png, code: 304, duration: 2.286374ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/swap.svg, code: 304, duration: 3.936646ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/function.svg, code: 304, duration: 4.605236ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/range.svg, code: 304, duration: 4.697525ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/switch.svg, code: 304, duration: 2.371219ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/white-globe.svg, code: 304, duration: 3.221799ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/bridge.svg, code: 304, duration: 3.938426ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /red/images/subflow_tab.svg, code: 304, duration: 1.376884ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/bridge-dash.svg, code: 304, duration: 1.733373ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/split.svg, code: 304, duration: 1.532646ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/join.svg, code: 304, duration: 1.460828ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/sort.svg, code: 304, duration: 2.395725ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/batch.svg, code: 304, duration: 1.260347ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/parser-csv.svg, code: 304, duration: 2.268534ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/parser-json.svg, code: 304, duration: 3.996973ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/parser-html.svg, code: 304, duration: 3.689769ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/file-out.svg, code: 304, duration: 1.917863ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/parser-yaml.svg, code: 304, duration: 2.119178ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/parser-xml.svg, code: 304, duration: 1.355629ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/file-in.svg, code: 304, duration: 2.569952ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: DEBU             Skipping login 95c2d1cd-86d2-4845-aea1-d3a062bc42d5, not a valid Unix login. services/role.go:360
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/file.svg, code: 304, duration: 1.433782ms tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196
Jan 11 10:31:42 hades teleport[436900]: INFO             Round trip: GET /icons/node-red/watch.svg, code: 304, duration: 971.332µs tls:version: 304, tls:resume:true, tls:csuite:1301, tls:server:74656c65706f72742e776562766963742e696d.teleport.cluster.local forward/fwd.go:196

Environment

  • Teleport version (use teleport version): Teleport v5.1.0 git:v5.1.0-0-g46679fb34 go1.15.5

  • OS (e.g. from /etc/os-release): Fedora 32

  • Where are you running Teleport? (e.g. AWS, GCP, Dedicated Hardware): Intel NUC

Browser environment

  • Browser Version (for UI-related issues): Google Chrome Version 87.0.4280.88 (Official Build) (arm64)
@webvictim webvictim added bug feature-request Used for new features in Teleport, improvements to current should be #enhancements application-access labels Jan 11, 2021
@antoine-em
Copy link

@webvictim
I have tried the same kind of configuration as you and got the same problem.
The websocket connection failed but the nodered application was loaded successfully otherwise.

I haven't found any documentation on Teleport for that so I am wondering if it is a current limitation.

@antoine-em
Copy link

antoine-em commented Jan 13, 2021

First I have created a self-signed certificate that I deployed on Node-RED that is therefore running in HTTPS. This solve the websocket (wss://) HTTP/HTTPS mixin issue.

Then I could make it worked using a SSH local port forwarding with tsh only (so https + wss).

# Note : the node is connected using the tunneling approach
tsh ssh -L 20001:nodered-node:31800 --local user@node
# Then from the web browser on https://localhost:20001

If I try from the web portal selecting my app I have the following error

teleport[11700]: INFO [APP:WEB]   Round trip: GET /, code: 500, duration: 3.174555794s tls:version: 304, tls:resume:true, tls:csuite:1303, tls:server:nodered.mydomain.com forward/fwd.go:196

I don't understand what is going wrong there as for other services like Kubernetes the forwarding to an internal cluster using a self signed TLS certificate is working properly.

@klizhentas klizhentas added this to the 6.1 milestone Feb 14, 2021
@klizhentas klizhentas added the coffee Sasha tracks these mission critical issues. label Feb 27, 2021
@tacerus
Copy link

tacerus commented Jul 11, 2021

Hi,

I am still encountering this issue after upgrading from 6.0 to 6.2. I tried different constellations but one app will fail with browsers reporting (in the dev console) wss://<domain>/api/ failed: Error during WebSocket handshake: Unexpected response code: 404 and similar. Is it confirmed that a proxy to an https:// website should also proxy wss:// automatically? Or are rewrites required?

Would appreciate any help. Thanks a lot!

@lord-kyron
Copy link

lord-kyron commented Nov 22, 2021

Hi, I am having the same problem here. Trying to access application which is behind nginx server as ngins id doing some hard rewrite stuff, which teleport is not able to do. However, websockets are working when I am directly accessing the web page from the nginx instance, but when I try to use the teleport app access before the nginx - everything seems to work fine except the websockets. I've tried everything described in this post and with no success. It seems wss:// just does not to pass through teeleport. Using teleport enterprise 7.3.3

@webvictim
Copy link
Contributor Author

cc @r0mant

@webvictim webvictim reopened this Nov 25, 2021
@lord-kyron
Copy link

@webvictim I started using teleport ent 8.0.3 and now in the logs I can see that teleport is returning this:
Unable to read websocket upgrade response: malformed HTTP response "\x00\x00\x12\x04\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x80\x00\x04\x00\x01\x00\x00\x00\x05\x00\xff\xff\xff\x00\x00\x04\b\x00\x00\x00\x00\x00\u007f\xff\x00\x00\x00\x00\b\a\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01"
I am trying to understand why.

@r0mant
Copy link
Collaborator

r0mant commented Dec 8, 2021

My knee jerk reaction looking at this response was that this looks like a beginning of TLS handshake which would've confirmed the issue with proxying secure web sockets but IIRC TLS handshake starts with \x16 (we're using this to detect TLS in multiplexer) so it's probably not it. Also, the forwarder does support dialing wss: https://github.com/gravitational/teleport/blob/v8.0.1/lib/srv/app/transport.go#L336-L350.

We'd need to repro this to see what's going on, I'll see when we can schedule this. @lord-kyron Are you able to try and point Teleport directly to the application bypassing nginx just to try and rule it out, or is the application not available directly to your Teleport app agent?

@lord-kyron
Copy link

I've actually found what my issue is. It is because nginx was serving the server block with http2 enabled and it seems teleport knows that next hop will use http2 and tries to pass the websockets over the http2 and the nginx is returning the http is malformed as it cannot upgrade the request. However when I turned off the http2 on nginx level it is working flawlessly now.
Next step is now to modify the nginx config to do some "guesses" when it should use http2 or 1.1 and leave http2 enabled by default.
Bypassing nginx was not working in this particular case as the app behind was vmware vcenter webui which needs some hard rewrites and redirects in order to be proxied, which teleport itself cannot do.
Thanks again, but my case was resolved! Maybe however you can think if teleport can act differently and not try to use http2 directly for the websockets.

@vdudejon
Copy link

@lord-kyron I'm trying to proxy vCenter as well but I'm not having much luck. Would you be willing to share your NGINX and teleport configurations?

@itmisx
Copy link

itmisx commented Mar 4, 2022

@lord-kyron can share your nginx configurations?

@lord-kyron
Copy link

lord-kyron commented Mar 4, 2022

@vdudejon @itmisx
Here you go:

    server {
       listen 443 ssl;
       server_name YOUR_PUBLIC_TELEPORT_APP_URL;
       ssl_certificate PATH_TO_CRT_FILE;
       ssl_certificate_key PATH_TO_KEY_FILE;
       ssl_verify_client off;
       ssl_protocols TLSv1.1 TLSv1.2 TLSv1.3;
       location / {
          proxy_pass https://YOUR_VCENTER_URL_ADDRESS;
          proxy_http_version 1.1;
          proxy_set_header Upgrade $http_upgrade;
          proxy_set_header Connection "upgrade";
          proxy_set_header Host YOUR_VCENTER_URL_ADDRESS;
          proxy_set_header Origin https://YOUR_VCENTER_URL_ADDRESS;
          proxy_set_header X-Real-IP $remote_addr;
          proxy_ssl_verify off;
          proxy_buffering off;
          client_max_body_size 0;
          proxy_read_timeout 36000s;
          proxy_redirect https://YOUR_VCENTER_URL_ADDRESS/ https://YOUR_PUBLIC_TELEPORT_APP_URL/;
       }

       location /websso/SAML2 {
          sub_filter "YOUR_VCENTER_URL_ADDRESS" "YOUR_PUBLIC_TELEPORT_APP_URL";
          proxy_set_header Host YOUR_VCENTER_URL_ADDRESS;
          proxy_set_header X-Real-IP $remote_addr;
          proxy_ssl_verify off;
          proxy_pass https://YOUR_VCENTER_URL_ADDRESS;
          proxy_http_version 1.1;
          proxy_set_header Upgrade $http_upgrade;
          proxy_set_header Connection "upgrade";
          proxy_buffering off;
          client_max_body_size 0;
          proxy_read_timeout 36000s;
          proxy_ssl_session_reuse on;
          proxy_redirect https://YOUR_VCENTER_URL_ADDRESS/ https://YOUR_PUBLIC_TELEPORT_APP_URL/;
       }
    }

And Teleport Apps configuration should be:

      - name: "NAME_OF_YOUR_APP"
      description: "DESCRIPTION_OF_YOUR_APP"
      uri: "https://URL_OR_IP_ADDRESS_OF_NGINX_REVERSE_PROXY"
      public_addr: "YOUR_PUBLIC_TELEPORT_APP_URL"
      insecure_skip_verify: true
      labels:
         env: "LABEL_OF_YOUR_CHOICE"
      rewrite:
        headers:
        - "Host: YOUR_PUBLIC_TELEPORT_APP_URL"

@flybyray
Copy link

flybyray commented Dec 6, 2022

Bypassing nginx was not working in this particular case as the app behind was vmware vcenter webui which needs some hard rewrites and redirects in order to be proxied, which teleport itself cannot do.

Is it clear what exactly is missing in teleport and is there a follow up issue/ticket for it as feature request?

@pschisa
Copy link
Contributor

pschisa commented Dec 6, 2022

@flybyray #19022 might be the follow up

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
application-access bug coffee Sasha tracks these mission critical issues. feature-request Used for new features in Teleport, improvements to current should be #enhancements
Projects
None yet
10 participants