New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Binary Payload Support - Pact mock service returning 500 error: Encoding::UndefinedConversionError - "\xFF" from ASCII-8BIT to UTF-8 #97
Comments
I have pretty the same error, I assume
I attached a reproducible example: Implementation of simple client:import gzip
import json
import requests
def compress(data: dict):
return gzip.compress(data=json.dumps(data).encode("utf-8"))
class Client:
def __init__(self, base_url="http://localhost:5000"):
self.url = base_url
def send(self, content):
compressed_content = compress(content)
response = requests.post(
self.url, data=compressed_content, headers={"Content-Encoding": "gzip"}
)
return response.status_code Test for this client:import socket
from contextlib import closing
import pytest
from pact import Consumer, Pact, Provider
from client import Client
def find_free_port(host):
with closing(socket.socket(socket.AF_INET, socket.SOCK_STREAM)) as s:
s.bind((host, 0))
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
return s.getsockname()[1]
@pytest.fixture(scope="session")
def host():
return "localhost"
@pytest.fixture(scope="session")
def port(host):
return find_free_port(host)
@pytest.fixture(scope="session")
def pact(host, port) -> Pact:
pact = (
Consumer("Client", tag_with_git_branch=True, tags=["broken", "mock-server"])
.has_pact_with(
Provider("Server"), host_name=host, port=port, pact_dir="tests/data/contracts"
)
)
pact.start_service()
yield pact
pact.stop_service()
@pytest.fixture(scope="session")
def client(host, port) -> Client:
return Client(f"http://{host}:{port}")
def test_client_sending_messages(pact, client):
exp_content = {"data": [1, 2, 3]}
pact_definition = (
pact.given("Sending requests")
.upon_receiving("binary (compressed data / files / etc...)")
.with_request("POST", "/", body=exp_content)
.will_respond_with(status=201)
)
with pact_definition:
result = client.send(exp_content)
assert result == 201 Result
but if i remove compression, it works fine diff --git a/client.py b/client.py
index aca214f..4b6f0c5 100644
--- a/client.py
+++ b/client.py
@@ -13,8 +13,5 @@ class Client:
self.url = base_url
def send(self, content):
- compressed_content = compress(content)
- response = requests.post(
- self.url, data=compressed_content, headers={"Content-Encoding": "gzip"}
- )
+ response = requests.post(self.url, json=content)
return response.status_code |
Also attached full log: I, [2022-09-21T15:09:23.473267 #97187] INFO -- : Cleared interactions
I, [2022-09-21T15:09:23.479048 #97187] INFO -- : Registered expected interaction POST /
D, [2022-09-21T15:09:23.479159 #97187] DEBUG -- : {
"description": "binary (compressed data / files / etc...)",
"providerState": "Sending requests",
"request": {
"method": "POST",
"path": "/",
"body": {
"data": [
1,
2,
3
]
}
},
"response": {
"status": 201,
"headers": {
}
},
"metadata": null
}
I, [2022-09-21T15:09:23.485407 #97187] INFO -- : Received request POST /
D, [2022-09-21T15:09:23.489456 #97187] DEBUG -- : {:path=>"/", :query=>"", :method=>"post", :body=>"\x1F\x8B\b\x00\xF3\xFE*c\x02\xFF\xABVJI,IT\xB2R\x886\xD4Q0\xD2Q0\x8E\xAD\x05\x00\xEAk\xD2\x02\x13\x00\x00\x00", :headers=>{"Content-Length"=>"39", "Host"=>"localhost:63061", "User-Agent"=>"python-requests/2.28.1", "Accept-Encoding"=>"gzip, deflate", "Accept"=>"*/*", "Connection"=>"keep-alive", "Content-Encoding"=>"gzip", "Version"=>"HTTP/1.1"}}
E, [2022-09-21T15:09:23.490421 #97187] ERROR -- : No matching interaction found for POST /
E, [2022-09-21T15:09:23.490451 #97187] ERROR -- : Interaction diffs for that route:
E, [2022-09-21T15:09:23.490747 #97187] ERROR -- : Error ocurred in mock service: Encoding::UndefinedConversionError - "\x8B" from ASCII-8BIT to UTF-8
E, [2022-09-21T15:09:23.490805 #97187] ERROR -- : /Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/json-2.5.1/lib/json/common.rb:406:in `encode'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/json-2.5.1/lib/json/common.rb:406:in `generate'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/json-2.5.1/lib/json/common.rb:406:in `pretty_generate'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-support-1.17.0/lib/pact/matchers/unix_diff_formatter.rb:64:in `generate_string'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-support-1.17.0/lib/pact/matchers/unix_diff_formatter.rb:37:in `to_s'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-support-1.17.0/lib/pact/matchers/unix_diff_formatter.rb:31:in `call'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-support-1.17.0/lib/pact/matchers/unix_diff_formatter.rb:27:in `call'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/mock_service/interactions/interaction_mismatch.rb:59:in `to_s'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/mock_service/interactions/interaction_mismatch.rb:25:in `collect'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/mock_service/interactions/interaction_mismatch.rb:25:in `to_s'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/mock_service/request_handlers/interaction_replay.rb:162:in `log'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/mock_service/request_handlers/interaction_replay.rb:143:in `call'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/mock_service/request_handlers/interaction_replay.rb:74:in `handle_unrecognised_request'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/mock_service/request_handlers/interaction_replay.rb:58:in `find_response'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/mock_service/request_handlers/interaction_replay.rb:45:in `respond'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/mock_service/request_handlers/base_request_handler.rb:17:in `call'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/rack-2.1.4/lib/rack/cascade.rb:35:in `block in call'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/rack-2.1.4/lib/rack/cascade.rb:26:in `each'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/rack-2.1.4/lib/rack/cascade.rb:26:in `call'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/consumer/mock_service/cors_origin_header_middleware.rb:11:in `call'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/consumer/mock_service/error_handler.rb:13:in `call'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/mock_service/app.rb:34:in `call'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/consumer/mock_service/set_location.rb:14:in `call'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/rack-2.1.4/lib/rack/handler/webrick.rb:88:in `service'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/webrick-1.3.1/lib/webrick/httpserver.rb:138:in `service'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/webrick-1.3.1/lib/webrick/httpserver.rb:94:in `run'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/webrick-1.3.1/lib/webrick/server.rb:191:in `block in start_thread'
E, [2022-09-21T15:09:23.496018 #97187] ERROR -- : Error ocurred in mock service: Encoding::UndefinedConversionError - "\x8B" from ASCII-8BIT to UTF-8
E, [2022-09-21T15:09:23.496094 #97187] ERROR -- : /Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/json-2.5.1/lib/json/common.rb:406:in `encode'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/json-2.5.1/lib/json/common.rb:406:in `generate'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/json-2.5.1/lib/json/common.rb:406:in `pretty_generate'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-support-1.17.0/lib/pact/matchers/unix_diff_formatter.rb:64:in `generate_string'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-support-1.17.0/lib/pact/matchers/unix_diff_formatter.rb:37:in `to_s'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-support-1.17.0/lib/pact/matchers/unix_diff_formatter.rb:31:in `call'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-support-1.17.0/lib/pact/matchers/unix_diff_formatter.rb:27:in `call'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/mock_service/interactions/interaction_mismatch.rb:59:in `to_s'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/mock_service/interactions/interaction_mismatch.rb:25:in `collect'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/mock_service/interactions/interaction_mismatch.rb:25:in `to_s'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/mock_service/request_handlers/verification_get.rb:55:in `collect'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/mock_service/request_handlers/verification_get.rb:55:in `to_s'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/mock_service/request_handlers/verification_get.rb:30:in `respond'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/mock_service/request_handlers/base_request_handler.rb:17:in `call'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/rack-2.1.4/lib/rack/cascade.rb:35:in `block in call'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/rack-2.1.4/lib/rack/cascade.rb:26:in `each'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/rack-2.1.4/lib/rack/cascade.rb:26:in `call'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/consumer/mock_service/cors_origin_header_middleware.rb:11:in `call'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/consumer/mock_service/error_handler.rb:13:in `call'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/mock_service/app.rb:34:in `call'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.9.0/lib/pact/consumer/mock_service/set_location.rb:14:in `call'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/rack-2.1.4/lib/rack/handler/webrick.rb:88:in `service'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/webrick-1.3.1/lib/webrick/httpserver.rb:138:in `service'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/webrick-1.3.1/lib/webrick/httpserver.rb:94:in `run'
/Users/denis.korytkin/.virtualenvs/pact-bug/lib/python3.7/site-packages/pact/bin/pact/lib/vendor/ruby/2.2.0/gems/webrick-1.3.1/lib/webrick/server.rb:191:in `block in start_thread' |
Yes, the Ruby core doesn't support binary payloads. When we update to the rust core (tentatively Q4 this year) we should be able to support that. See https://docs.pact.io/faq#how-do-i-test-binary-files-in-responses-such-as-a-download for what to do in the mean time. |
I've marked this, and will mark others with needs-pact-rust-core label, to note it won't be fixed until we move to the rust core, or will be fixed by moving to the rust core. For those wishing to contribute to getting pact-python into v3 and beyond via the rust core, please shout out here |
Binary payload support will be delivered in #367 Will update the title of the issue to reflect this |
Making a put request with image data as a payload results in the mock service returning a 500. The following errors are logged in
A repo for replication can be found here:
https://github.com/meadsteve/pact-django-image-error
Any thoughts on what's happening? Is there something I've missed?
The text was updated successfully, but these errors were encountered: