Too much data before UTF-8 encoding

[igor-wl]

I'm trying to push an 8KB message to pusher and while it works for ascii characters it breaks if for example there is even a single "🦄" char in the dictionary because now sys.getsizeof returns size of the string object as 34KB.

Considering that the final dictionary is encoded to UTF-8 before sending to pusher I think that the length check should also be performed on the UTF-8 encoded string and not on the python object.

The issue is with this code:

if sys.getsizeof(data) > 30720:
    raise ValueError("Too much data")

Small demo how much more bytes is consumed for python object compared to UTF-8 string.

[benjamin-tang-pusher]

Hi, I was able to send that character from my Python backend.

Are you using the latest version of our library? We made a change where ensure_ascii=False at

pusher-http-python/pusher/util.py

Line 113 in 239d67b

return json.dumps(data, cls=json_encoder, ensure_ascii=False)

utf-8 characters passed to our library remains as-is (before we accidentally escaped unicode characters so the size ballooned, but this shouldn't be the case anymore)

[benjamin-tang-pusher]

Closing due to no response.