Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[webapp remote-connection] - webapp:BrokenPipe error when trying to create TCP tunnel for SSH #8831

Open
Primital opened this issue Mar 20, 2019 · 6 comments
Assignees
Labels
App Services az appservice linked linked to an azure devops ticket Service Attention This issue is responsible by Azure service team. Web Apps az webapp
Milestone

Comments

@Primital
Copy link

Primital commented Mar 20, 2019

Describe the bug
After starting a TCP tunnel and trying to ssh to localhost on the specified port, the az tunnel crashes immediately after a BrokenPipeError: [Errno 32] Broken pipe

To reproduce
I don't know the cause. On my system it crashes everytime. Please read my comments in additional context

Expected behavior
I expect the ssh connection to establish

Environment summary
Python3.7.2_2 installed via brew. az-cli installed via brew. Pyenv used for other projects. Installed versions using pyenv are 2.7.15 and 3.6.5

Additional context
I have tried reinstalling everything through brew. I tried brew doctor. I tried brew link python. I tried to reinstall python and then brew link python again. I tried to link brews python versions to pyenv paths. I exhausted all possible search terms on google.

Verbose output from crash

az webapp remote-connection create -g MYRESRCGROUP -n MYAPP -p 1337 --verbose
Port 1337 is open
Creating a socket on port: 1337
Setting socket options
Binding to socket on local address and port
Finished initialization
Status response message: FAILURE:2222:Unable to connect to WebApp
WARNING - Remote debugging may not be setup properly. Reponse content: FAILURE:2222:Unable to connect to WebApp
SSH is available { username: root, password: Docker! }
Start your favorite client and connect to port 1337

In another window i used ssh root@127.0.0.1 -p 1337

Websocket tracing enabled
--- request header ---
GET /AppServiceTunnel/Tunnel.ashx HTTP/1.1
Upgrade: websocket
Connection: Upgrade
Host: MYDOMAIN.scm.azurewebsites.net
Origin: http://MYDOMAIN.scm.azurewebsites.net
Sec-WebSocket-Key: RQM/U3LlGAC45h3cBVfZ+g==
Sec-WebSocket-Version: 13
Authorization: Basic JGtvbW11bnBvcnRhbDpjN3hXaFBuckxxQnVkMktZTWpQRG9ObzhuelFpOGRTcjdscTVna1FaZ0tMclNjNHJ6Sm5xMGMwZGI0QU0=


-----------------------
--- response header ---
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Server: Kestrel
Sec-WebSocket-Accept: Hb1GVBHYowi3+S7XGOYT4aj54fo=
Set-Cookie: ARRAffinity=bf607f7282cfa28719bacd3fbabae13c629c7e590d103982a12542d52319894c;Path=/;HttpOnly;Domain=MYDOMAIN.scm.azurewebsites.net
Connection: Upgrade
Access-Control-Allow-Origin: http://MYDOMAIN.scm.azurewebsites.net
Access-Control-Expose-Headers: ETag,Content-Length,Content-Encoding,Date
Date: Wed, 20 Mar 2019 13:45:24 GMT
-----------------------
Websocket, connected status: True
Got debugger connection... index: 1
Waiting for debugger data, index: 1
Received debugger data, nbytes: 21, index: 1
Sending to websocket, response data: bytearray(b'SSH-2.0-OpenSSH_7.9\r\n'), index: 1
send: b'\x82\x95\x15D\xef\xe7F\x17\xa7\xca\'j\xdf\xcaZ4\x8a\x89F\x17\xa7\xb8"j\xd6\xea\x1f'
Waiting for websocket data, connection status: True, index: 1
Both debugger and websocket threads started...
Successfully connected to local server..
Received frame: fin=1 opcode=8 data=b'\x03\xf3Connection failure. Connection refused 172.19.0.3:2222'
send: b'\x88\x82[\xba\xcf\x9cXR'
Traceback (most recent call last):
  File "/Users/myname/.azure/cliextensions/webapp/azext_webapp/tunnel.py", line 146, in _listen_to_web_socket
    data = ws_socket.recv()
  File "/Users/myname/.azure/cliextensions/webapp/azext_webapp/tunnel.py", line 32, in recv
    data = super(TunnelWebSocket, self).recv()
  File "/usr/local/Cellar/azure-cli/2.0.60/libexec/lib/python3.7/site-packages/websocket/_core.py", line 313, in recv
    opcode, data = self.recv_data()
  File "/usr/local/Cellar/azure-cli/2.0.60/libexec/lib/python3.7/site-packages/websocket/_core.py", line 330, in recv_data
    opcode, frame = self.recv_data_frame(control_frame)
  File "/usr/local/Cellar/azure-cli/2.0.60/libexec/lib/python3.7/site-packages/websocket/_core.py", line 357, in recv_data_frame
    self.send_close()
  File "/usr/local/Cellar/azure-cli/2.0.60/libexec/lib/python3.7/site-packages/websocket/_core.py", line 390, in send_close
    self.send(struct.pack('!H', status) + reason, ABNF.OPCODE_CLOSE)
  File "/usr/local/Cellar/azure-cli/2.0.60/libexec/lib/python3.7/site-packages/websocket/_core.py", line 253, in send
    return self.send_frame(frame)
  File "/usr/local/Cellar/azure-cli/2.0.60/libexec/lib/python3.7/site-packages/websocket/_core.py", line 278, in send_frame
    l = self._send(data)
  File "/usr/local/Cellar/azure-cli/2.0.60/libexec/lib/python3.7/site-packages/websocket/_core.py", line 448, in _send
    return send(self.sock, data)
  File "/usr/local/Cellar/azure-cli/2.0.60/libexec/lib/python3.7/site-packages/websocket/_socket.py", line 151, in send
    return _send()
  File "/usr/local/Cellar/azure-cli/2.0.60/libexec/lib/python3.7/site-packages/websocket/_socket.py", line 136, in _send
    return sock.send(data)
  File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 984, in send
    return self._sslobj.write(data)
ssl.SSLError: [SSL: BAD_LENGTH] bad length (_ssl.c:2337)
Traceback (most recent call last):
  File "/Users/myname/.azure/cliextensions/webapp/azext_webapp/tunnel.py", line 175, in _listen_to_client
    ws_socket.send_binary(responseData)
  File "/usr/local/Cellar/azure-cli/2.0.60/libexec/lib/python3.7/site-packages/websocket/_core.py", line 284, in send_binary
    return self.send(payload, ABNF.OPCODE_BINARY)
  File "/usr/local/Cellar/azure-cli/2.0.60/libexec/lib/python3.7/site-packages/websocket/_core.py", line 253, in send
    return self.send_frame(frame)
  File "/usr/local/Cellar/azure-cli/2.0.60/libexec/lib/python3.7/site-packages/websocket/_core.py", line 278, in send_frame
    l = self._send(data)
  File "/usr/local/Cellar/azure-cli/2.0.60/libexec/lib/python3.7/site-packages/websocket/_core.py", line 448, in _send
    return send(self.sock, data)
  File "/usr/local/Cellar/azure-cli/2.0.60/libexec/lib/python3.7/site-packages/websocket/_socket.py", line 151, in send
    return _send()
  File "/usr/local/Cellar/azure-cli/2.0.60/libexec/lib/python3.7/site-packages/websocket/_socket.py", line 136, in _send
    return sock.send(data)
  File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 984, in send
    return self._sslobj.write(data)
BrokenPipeError: [Errno 32] Broken pipe
Both debugger and websocket threads stopped...
Stopped local server..
@adewaleo adewaleo added Web Apps az webapp bug This issue requires a change to an existing behavior in the product in order to be resolved. Needs Triage and removed bug This issue requires a change to an existing behavior in the product in order to be resolved. labels Mar 20, 2019
@patricklee2
Copy link
Contributor

patricklee2 commented Apr 22, 2019

@Primital can you open https://${app-name}.scm.azurewebsites.net/AppServiceTunnel/Tunnel.ashx?GetStatus in a browser?
what is the name of the app?, can you confirm that the webapp and scm sites are running?
what is the result of az webapp ssh?

@bsiegel bsiegel added the Service Attention This issue is responsible by Azure service team. label Jun 14, 2019
@mozehgir mozehgir added the App Services az appservice label Jul 9, 2019
@yugangw-msft yugangw-msft removed the App Services az appservice label Aug 1, 2019
@btardif btardif added this to Tirage in Compute Features Aug 30, 2019
@yonzhan yonzhan removed the triage label Sep 2, 2019
@btardif btardif added the App Services az appservice label Sep 25, 2019
@panchagnula panchagnula changed the title BrokenPipe error when trying to create TCP tunnel for SSH webapp:BrokenPipe error when trying to create TCP tunnel for SSH Sep 26, 2019
@panchagnula
Copy link
Contributor

panchagnula commented Sep 26, 2019

Closing this due to inactivity.

Compute Features automation moved this from Tirage to done Sep 26, 2019
@haroldrandom haroldrandom added App Services az appservice needs-triage This is a new issue that needs to be triaged to the appropriate team. Service Attention This issue is responsible by Azure service team. Web Apps-cli labels Oct 25, 2019
@ghost
Copy link

ghost commented Apr 23, 2020

Hi. Sorry to revive an old issue, but I'm having this same problem.

When I go to <webapp>.scm.azurewebsites.net/AppServiceTunnel/Tunnel.ashx?GetStatus I see the following:

FAILURE:2222:Unable to connect to WebApp

Response when I run az webapp create-remote-connection --subscription <...> --resource-group <...> -n <...> -p <...> --verbose :

Status response message: FAILURE:2222:Unable to connect to WebApp
WARNING - Remote debugging may not be setup properly. Reponse content: FAILURE:2222:Unable to connect
to WebApp

And the response from az webapp ssh:

This command is in preview. It may be changed/removed in a future release.
Connection is not ready yet, please wait
.
.
...
.
Timeout Error, Unable to establish a connection

@gguerrero
Copy link

gguerrero commented May 4, 2020

Same issue here:

az webapp ssh --name <myApp> --resource-group <myResourceGroup> --verbose
This command is in preview. It may be changed/removed in a future release.
No port defined, creating on random free port
Creating a socket on port: 0
Setting socket options
Binding to socket on local address and port
Auto-selecting port: 50693
Finished initialization
Status response message: FAILURE:2222:Unable to connect to WebApp
WARNING - Remote debugging may not be setup properly. Reponse content: FAILURE:2222:Unable to connect to WebApp
Connection is not ready yet, please wait
.
.
.
Timeout Error, Unable to establish a connection

Connecting properly to the webapp as I can do any az webapp show or any other command from my shell.
Is there any missing configuration in the webapp side?

@yonzhan yonzhan added the Web Apps az webapp label May 4, 2020
@msftbot
Copy link

msftbot bot commented May 4, 2020

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @AzureAppServiceCLI, @antcp.

@msftbot msftbot bot removed the needs-triage This is a new issue that needs to be triaged to the appropriate team. label May 4, 2020
@btardif btardif reopened this Sep 23, 2020
Compute Features automation moved this from done to to-do Sep 23, 2020
@btardif btardif assigned panchagnula and unassigned patricklee2 Sep 23, 2020
@btardif btardif added this to the S177 milestone Sep 23, 2020
@btardif btardif changed the title webapp:BrokenPipe error when trying to create TCP tunnel for SSH [SSH] - webapp:BrokenPipe error when trying to create TCP tunnel for SSH Sep 23, 2020
@btardif btardif moved this from to-do to backlog in Compute Features Sep 29, 2020
@btardif btardif modified the milestones: S177, S178 Sep 29, 2020
@btardif btardif modified the milestones: S178, S179 Oct 28, 2020
@panchagnula panchagnula changed the title [SSH] - webapp:BrokenPipe error when trying to create TCP tunnel for SSH [webapp] - webapp:BrokenPipe error when trying to create TCP tunnel for SSH Oct 28, 2020
@btardif btardif moved this from backlog to Tirage in Compute Features Nov 4, 2020
@btardif btardif moved this from Tirage to backlog in Compute Features Nov 4, 2020
@btardif btardif changed the title [webapp] - webapp:BrokenPipe error when trying to create TCP tunnel for SSH [webapp remote-connection] - webapp:BrokenPipe error when trying to create TCP tunnel for SSH Dec 2, 2020
@btardif btardif modified the milestones: S179, S181 Dec 3, 2020
@btardif btardif moved this from backlog to Tirage in Compute Features Dec 3, 2020
@panchagnula panchagnula modified the milestones: S181, Backlog Jan 13, 2021
@coderintherye
Copy link

coderintherye commented Jun 15, 2021

In case it is helpful I get the same error messaging:
Creating a socket on port: 2222
Setting socket options
Binding to socket on local address and port
Finished initialization
Status response message: FAILURE:2222:Unable to connect to WebApp

after an app instance goes unhealthy, restarting the app instance does not resolve the problem, but going to webssh via the browser does, after that the remote connection works fine. The browser access seems to indicate it is creating a new Docker container...perhaps that somehow resolves the issue.

@btardif btardif assigned seligj95 and unassigned btardif Jun 25, 2021
@seligj95 seligj95 added the linked linked to an azure devops ticket label Jul 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
App Services az appservice linked linked to an azure devops ticket Service Attention This issue is responsible by Azure service team. Web Apps az webapp
Projects
No open projects
Compute Features
  
Triage
Development

No branches or pull requests