Problem reading gzipped kong.service.response.get_raw_body() #156
Comments
|
cc @fffonion |
|
@dwoctor could you share with me the base64 encoded output of both lua and js plugin? |
|
@fffonion, I have been testing against https://rickandmortyapi.com/api/character/1 with In the javascript plugin, I used the following to get the base64: const rawBodyCompressed = await kong.service.response.getRawBody();
const rawBodyCompressedBase64 = Buffer.from(rawBodyCompressed, "binary").toString("base64");The base64 generated by the javascript plugin was as follows: In the lua plugin, I used the following to get the base64: local rawBodyCompressed = kong.service.response.get_raw_body()
local rawBodyCompressedBase64 = base64.encode(rawBodyCompressed)The base64 generated by the lua plugin was as follows: |
seeing lots of wrong characters become |
|
@fffonion is there any news? |
|
I've tested with python pdk with: import kong_pdk.pdk.kong as kong
Schema = (
{ "message": { "type": "string" } },
)
version = '0.1.0'
priority = 0
class Plugin(object):
def __init__(self, config):
self.config = config
def access(self, kong: kong.kong):
a = kong.request.get_raw_body()
kong.log(a)and a random binary string as request body: I got: It doesn't seem related to gzip or js. Any binary request body could reproduce this behavior. |
|
The behaviour I was looking at was on the service response side not the request. |
I have just gone through the code for a while. We are using msgpack for RPC, and its default encoding of strings is "string_compact". A reasonable guess is that, js and go interpreter all string as encoded by UTF-8 or something, and all RPC calls (implemented with msgpack) is influenced by this behavior. So you can expect this behavior also appear in headers, I will test go PDK later(it uses protobuf, not msgpack). |
|
I've tested on go PDK and it also uses msgpack. However, it doesn't have the same problem. The reason seems to be that golang can handle arbitrary bytes in a string(just like what Lua does). |
|
The return value type for Lua and Golang's strings are sure to be capable to hold arbitrary binary data; This problem is due to the design of interface. |
|
We have 2 solutions:
We decide to apply solution 2. However, there are still decisions to make for 2:
Anyway, we will migrate to protobuf, it will not affect too much. So we can simply leave those behaviors unchanged, except for fixing the raw_body. |
I have been trying to read the
kong.service.response.get_raw_body()in a kong js plugin.I'm able to read the data fine when it's not gzipped.
However, when the response is gzipped, I'm unable to gunzip.
I wrote a version of the plugin in lua and was able to perform the task successfully.
When I compared the base64 encodings of both the js and lua responses from
kong.service.response.get_raw_body()I found they did not match. Which to me points to an issue with the pdk, but I'm not 100% sure.If any assistance can be provided in trying to rectify this problem I'm experiencing, it would be greatly appreciated.
The text was updated successfully, but these errors were encountered: