Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

./pants server broken (TypeError) with Python 3 #7443

Closed
codealchemy opened this issue Mar 27, 2019 · 6 comments

Comments

Projects
4 participants
@codealchemy
Copy link
Contributor

commented Mar 27, 2019

schmitt :: ~/Workspace/pants ‹master› » pyenv shell 3.6.6
schmitt :: ~/Workspace/pants ‹master› » ./pants killserver
INFO] No server found.

schmitt :: ~/Workspace/pants ‹master› » ./pants server
INFO] Launched server with pid 10393 at http://localhost:51318
INFO] To kill, run `./pants killserver`

schmitt :: ~/Workspace/pants ‹master› » ----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 51320)
Traceback (most recent call last):
  File "/Users/schmitt/.pyenv/versions/3.6.6/lib/python3.6/socketserver.py", line 317, in _handle_request_noblock
    self.process_request(request, client_address)
  File "/Users/schmitt/.pyenv/versions/3.6.6/lib/python3.6/socketserver.py", line 348, in process_request
    self.finish_request(request, client_address)
  File "/Users/schmitt/.pyenv/versions/3.6.6/lib/python3.6/socketserver.py", line 361, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/Users/schmitt/Workspace/pants/src/python/pants/reporting/reporting_server.py", line 396, in __init__
    PantsHandler.__init__(self, settings, renderer, request, client_address, server)
  File "/Users/schmitt/Workspace/pants/src/python/pants/reporting/reporting_server.py", line 60, in __init__
    http.server.BaseHTTPRequestHandler.__init__(self, request, client_address, server)
  File "/Users/schmitt/.pyenv/versions/3.6.6/lib/python3.6/socketserver.py", line 721, in __init__
    self.handle()
  File "/Users/schmitt/.pyenv/versions/3.6.6/lib/python3.6/http/server.py", line 418, in handle
    self.handle_one_request()
  File "/Users/schmitt/.pyenv/versions/3.6.6/lib/python3.6/http/server.py", line 406, in handle_one_request
    method()
  File "/Users/schmitt/Workspace/pants/src/python/pants/reporting/reporting_server.py", line 76, in do_GET
    self._handle_runs('', {})
  File "/Users/schmitt/Workspace/pants/src/python/pants/reporting/reporting_server.py", line 89, in _handle_runs
    self._send_content(self._renderer.render_name('base.html', args), 'text/html')
  File "/Users/schmitt/Workspace/pants/src/python/pants/reporting/reporting_server.py", line 323, in _send_content
    self.wfile.write(content)
  File "/Users/schmitt/.pyenv/versions/3.6.6/lib/python3.6/socketserver.py", line 800, in write
    self._sock.sendall(b)
TypeError: a bytes-like object is required, not 'str'
----------------------------------------
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 51322)
Traceback (most recent call last):
  File "/Users/schmitt/.pyenv/versions/3.6.6/lib/python3.6/socketserver.py", line 317, in _handle_request_noblock
    self.process_request(request, client_address)
  File "/Users/schmitt/.pyenv/versions/3.6.6/lib/python3.6/socketserver.py", line 348, in process_request
    self.finish_request(request, client_address)
  File "/Users/schmitt/.pyenv/versions/3.6.6/lib/python3.6/socketserver.py", line 361, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/Users/schmitt/Workspace/pants/src/python/pants/reporting/reporting_server.py", line 396, in __init__
    PantsHandler.__init__(self, settings, renderer, request, client_address, server)
  File "/Users/schmitt/Workspace/pants/src/python/pants/reporting/reporting_server.py", line 60, in __init__
    http.server.BaseHTTPRequestHandler.__init__(self, request, client_address, server)
  File "/Users/schmitt/.pyenv/versions/3.6.6/lib/python3.6/socketserver.py", line 721, in __init__
    self.handle()
  File "/Users/schmitt/.pyenv/versions/3.6.6/lib/python3.6/http/server.py", line 418, in handle
    self.handle_one_request()
  File "/Users/schmitt/.pyenv/versions/3.6.6/lib/python3.6/http/server.py", line 406, in handle_one_request
    method()
  File "/Users/schmitt/Workspace/pants/src/python/pants/reporting/reporting_server.py", line 76, in do_GET
    self._handle_runs('', {})
  File "/Users/schmitt/Workspace/pants/src/python/pants/reporting/reporting_server.py", line 89, in _handle_runs
    self._send_content(self._renderer.render_name('base.html', args), 'text/html')
  File "/Users/schmitt/Workspace/pants/src/python/pants/reporting/reporting_server.py", line 323, in _send_content
    self.wfile.write(content)
  File "/Users/schmitt/.pyenv/versions/3.6.6/lib/python3.6/socketserver.py", line 800, in write
    self._sock.sendall(b)
TypeError: a bytes-like object is required, not 'str'
----------------------------------------

Works without issue under Python 2. ☝️ should be reproduceable with Python 3.6.6

@Eric-Arellano Eric-Arellano self-assigned this Mar 27, 2019

@illicitonion

This comment has been minimized.

Copy link
Contributor

commented Mar 27, 2019

What sha do you have checked out? This worked for me at 37654aa and your line numbers don't line up with mine...

@codealchemy

This comment has been minimized.

Copy link
Contributor Author

commented Mar 27, 2019

I'm on 37654aa as well - updated snippet above to represent the full repro steps for me.

@illicitonion

This comment has been minimized.

Copy link
Contributor

commented Mar 27, 2019

I still don't repro, but I'd suggest trying something along the lines of:

$ git diff
diff --git src/python/pants/reporting/reporting_server.py src/python/pants/reporting/reporting_server.py
index cf50c7c..852ca11 100644
--- src/python/pants/reporting/reporting_server.py
+++ src/python/pants/reporting/reporting_server.py
@@ -320,7 +320,7 @@ class PantsHandler(http.server.BaseHTTPRequestHandler):
     self.send_header('Content-Type', content_type)
     self.send_header('Content-Length', str(len(content)))
     self.end_headers()
-    self.wfile.write(content)
+    self.wfile.write(content.encode('utf-8'))
 
   def _client_allowed(self):
     """Check if client is allowed to connect to this server."""
@Eric-Arellano

This comment has been minimized.

Copy link
Contributor

commented Mar 29, 2019

@illicitonion I finally reproduce. The key step we were missing is to actually go to the URL created by the server. Once you do that, you immediately get this crash.

I will work on a fix.

@Eric-Arellano Eric-Arellano added this to the 1.15.x milestone Mar 29, 2019

@codealchemy

This comment has been minimized.

Copy link
Contributor Author

commented Mar 29, 2019

@Eric-Arellano 👍yes that's it, thanks for calling that out - realizing now I should have been clearer on that part of the repro steps.

@stuhood

This comment has been minimized.

Copy link
Member

commented Mar 29, 2019

I repro this.

Eric-Arellano added a commit that referenced this issue Mar 30, 2019

Fix `./pants server` not working with Python 3 (#7458)
### Problem
Fixes #7443.

Due to various unicode issues, `./pants server` would always crash when you clicked the link to the server.

### Solution
Fix unicode issues.

Also add an assertion that the argument to `self._send_content()` is bytes so that Python 2 behaves the same as Python 3. Notably, we do not try to coerce the content in this method because we cannot assume which encoding it should use, e.g. assets do not use UTF-8.

### Result
`./pants server` works consistently, including taking all actions like browsing the codebase.

@Eric-Arellano Eric-Arellano moved this from To do to Done in Adding Python 3 support Mar 30, 2019

stuhood added a commit that referenced this issue Apr 1, 2019

Fix `./pants server` not working with Python 3 (#7458)
### Problem
Fixes #7443.

Due to various unicode issues, `./pants server` would always crash when you clicked the link to the server.

### Solution
Fix unicode issues.

Also add an assertion that the argument to `self._send_content()` is bytes so that Python 2 behaves the same as Python 3. Notably, we do not try to coerce the content in this method because we cannot assume which encoding it should use, e.g. assets do not use UTF-8.

### Result
`./pants server` works consistently, including taking all actions like browsing the codebase.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.