feature(kafka-logger): need to read the request body data if it was cached to local file

[membphis]

https://github.com/apache/apisix/blob/master/apisix/utils/log-util.lua#L67

The file name is not enough.
we need to read the file content.

[swayamraina]

Hi @membphis
I would like to pick this up. Can you describe what exactly is required here?

[membphis]

https://github.com/apache/apisix/blob/master/apisix/utils/log-util.lua#L67

local body_file = ngx.req.get_body_file()
if body_file then
    -- we only readed the file name of request body, it is not enough.
    -- we need to read the file content by file name.
    log.request.body_file = body_file
end

for how to read file content by file name, you can take a look at this: https://stackoverflow.com/questions/11201262/how-to-read-data-from-a-file-in-lua

[membphis]

@swayamraina I have assigned this issue to you, many thx

[swayamraina]

@membphis I got what exactly is required here and will raise a PR.
I had one question though, why would request body data get cached to a local file and be absent in the request body

[sshniro]

@membphis any idea how to create this scenario? Where the body is kept in a local file? Does it occur if the data is too big?

[membphis]

Does it occur if the data is too big?

you are right. for more detail information, you can take a look at: https://github.com/openresty/lua-nginx-module#ngxreqread_body

I think that if the request body is big, we can only read the first 1 MB of the request body data.

[spacewander]

@membphis
Is this feature still required?

[membphis]

@membphis Is this feature still required?

yes, we need this feature.

[spacewander]

I don't think it is a good idea. As the disk operation is expensive and it only happens when the body is too large. Reading truncated body is not so helpful in the log.

[tzssangglass]

Would it be better to adjust the parameters to avoid the request being logged in the file?

[tokers]

Would it be better to adjust the parameters to avoid the request being logged in the file?

There has a trade-off between the memory usage and the disk operations, but it should be decided by the user.

[membphis]

Hope to hear more users' voices.