Cannot upload a binary file via when using multipart and Part

[liiight]

Describe the bug
Uplink throws an exception when trying to send a binary file using the multipart decorator:

Traceback (most recent call last):
  File "/Users/orcarmi/PycharmProjects/shield/f_poc.py", line 34, in <module>
    r = bin.post(foo=files)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/builder.py", line 99, in __call__
    self._request_definition.define_request(request_builder, args, kwargs)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/commands.py", line 268, in define_request
    request_builder, func_args, func_kwargs
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/arguments.py", line 153, in handle_call
    self.handle_call_args(request_builder, call_args)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/arguments.py", line 158, in handle_call_args
    annotation.modify_request(request_builder, call_args[name])
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/arguments.py", line 182, in modify_request
    self._modify_request(request_builder, converter(value))
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/converters/interfaces.py", line 6, in __call__
    return self.convert(*args, **kwargs)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/converters/standard.py", line 19, in convert
    return self._converter(value)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/converters/interfaces.py", line 6, in __call__
    return self.convert(*args, **kwargs)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/converters/standard.py", line 30, in convert
    dumped = json.dumps(value, default=self._default_json_dumper)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/converters/standard.py", line 25, in _default_json_dumper
    return obj.__dict__  # pragma: no cover
AttributeError: 'bytes' object has no attribute '__dict__'

To Reproduce

class HTTPBin(Consumer):

    @multipart
    @post('post')
    def post(self, foo: Part):
        pass


file_path = Path('some_binary.file')
bin = HTTPBin('https://httpin.org')
files = {file_path.name: file_path.read_bytes()}
r = bin.post(foo=files)

Expected behavior
This should be the equivalent of the following requests code (which works):

files = {file_path.name: file_path.read_bytes()}
r = requests.post('https://httpbin.org/post', files=files)

When trying to implement a suggested workaround:

@register_default_converter_factory
class PassThroughConverter(interfaces.Factory):
    def create_request_body_converter(self, type_, *args, **kwargs):
        return lambda value: value

I get a different error:

Traceback (most recent call last):
  File "/Users/orcarmi/PycharmProjects/shield/f_poc.py", line 34, in <module>
    r = bin.post(foo=files)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/builder.py", line 107, in __call__
    (request_builder.method, request_builder.url, request_builder.info)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/clients/io/execution.py", line 97, in start
    return self._io.execute(self)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/clients/io/execution.py", line 122, in execute
    return self._io.execute(executable)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/clients/io/blocking_strategy.py", line 31, in execute
    return executable.execute()
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/clients/io/execution.py", line 93, in execute
    return self.state.execute(self)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/clients/io/state.py", line 36, in execute
    return execution.before_request(self._request)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/clients/io/execution.py", line 56, in before_request
    return self.execute()
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/clients/io/execution.py", line 93, in execute
    return self.state.execute(self)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/clients/io/state.py", line 106, in execute
    self._request, self.SendCallback(execution, self._request)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/clients/io/execution.py", line 73, in send
    return self._io.invoke(self._client.send, (request,), {}, callback)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/clients/io/execution.py", line 116, in invoke
    return self._io.invoke(func, args, kwargs, callback)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/clients/io/blocking_strategy.py", line 19, in invoke
    return callback.on_failure(type(error), error, tb)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/clients/io/state.py", line 102, in on_failure
    return self._context.execute()
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/clients/io/execution.py", line 93, in execute
    return self.state.execute(self)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/clients/io/state.py", line 158, in execute
    self._request, self._exc_type, self._exc_val, self._exc_tb
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/clients/io/execution.py", line 70, in after_exception
    return self.execute()
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/clients/io/execution.py", line 93, in execute
    return self.state.execute(self)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/clients/io/state.py", line 191, in execute
    return execution.fail(self._exc_type, self._exc_val, self._exc_tb)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/clients/io/execution.py", line 82, in fail
    return self._io.fail(exc_type, exc_val, exc_tb)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/clients/io/execution.py", line 154, in fail
    return self._invoke(self._errback, exc_type, exc_val, exc_tb)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/clients/io/execution.py", line 133, in _invoke
    return self._io.invoke(func, args, kwargs, FinishingCallback(self._io))
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/clients/io/blocking_strategy.py", line 19, in invoke
    return callback.on_failure(type(error), error, tb)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/clients/io/execution.py", line 108, in on_failure
    return self._io.fail(exc_type, exc_val, exc_tb)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/clients/io/interfaces.py", line 303, in fail
    compat.reraise(exc_type, exc_val, exc_tb)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/six.py", line 693, in reraise
    raise value
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/clients/io/blocking_strategy.py", line 16, in invoke
    response = func(*arg, **kwargs)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/hooks.py", line 109, in handle_exception
    compat.reraise(exc_type, exc_val, exc_tb)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/six.py", line 693, in reraise
    raise value
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/clients/io/blocking_strategy.py", line 16, in invoke
    response = func(*arg, **kwargs)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/uplink/clients/requests_.py", line 50, in send
    return self.__session.request(method=method, url=url, **extras)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/requests/sessions.py", line 519, in request
    prep = self.prepare_request(req)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/requests/sessions.py", line 462, in prepare_request
    hooks=merge_hooks(request.hooks, self.hooks),
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/requests/models.py", line 316, in prepare
    self.prepare_body(data, files, json)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/requests/models.py", line 504, in prepare_body
    (body, content_type) = self._encode_files(files, data)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/requests/models.py", line 169, in _encode_files
    body, content_type = encode_multipart_formdata(new_fields)
  File "/Users/orcarmi/PycharmProjects/shield/venv/lib/python3.6/site-packages/urllib3/filepost.py", line 90, in encode_multipart_formdata
    body.write(data)
TypeError: a bytes-like object is required, not 'dict'

Additional context
Using uplink 0.9.0

[joeld1]

Hello, I'm fairly new to this package so I'm using the Petstore Swagger API to practice. Unfortunately, I'm also trying to upload a file but I'm unsure on how to do so.

I'm able to execute 1 call (add pet to petstore -> in order to get a pet_id), but I'm unable to upload a file using multipart (uploadFile).

When I call uploadFile, I get the following message:

A successful request returns the following response:

As such, I was wondering if you could please help and share what the proper way of uploading a file and structuring the Pets class is, for the class that I created using Uplink?

Thank you!

[prkumar]

@joeld1 - The file upload behavior you are encountering may be caused by an issue identified in #183. Can you try adding the workaround mentioned in #183 (comment) to your script?

[yyolk]

I've tried using the workaround mentioned here #183 (comment)
but there is still an issue with it. Also I wasn't sure where value is coming from in the suggested workaround, but after walking through it I understand it's returning a callable that will simply pass-through the value given to it.

edit: i am able to send a file with the appropriate content-type if I pass in a tuple that requests would expect and using the workaround mentioned here #183 (comment) this isn't ideal (I'd like to keep the ability to switch out the underlying client) but it's working for me right now.

edit 2: realized I didn't share my workaround with my last update, here it is for others. also interested @prkumar in your thoughts on the content-type per part.

from mimetypes import guess_type

from uplink.converters import StandardConverter
from uplink import (
    Consumer,
    Part,
    post,
    multipart,
    returns,
    headers,
)



def pass_through_request_body_converter(self, type_, *args, **kwargs):
    return lambda value: value


StandardConverter.create_request_body_converter = pass_through_request_body_converter


@headers({"Accept": "application/json"})
class BlobClient(Consumer):
    def post_blob(self, blob_fp, something_interesting):
        blob_type, _ = guess_type(blob_fp.name)
        return self._post_blob(
            (blob_fp.name, blob_fp, blob_type), something_interesting
        )

    @multipart
    @returns.json
    @post("/")
    def _post_blob(self, blob: Part, something_interesting: Part):
        pass


    ...