adding the possibility to log locally request and response payload

[lucamaf]

Referring to #291 I think it make still sense to offer the option to customers of logging Request and Response payload locally on the gateway and enabling such an option by Docker parameter (no req/resp sent to the API manager).
To manage access log dimensions growing much bigger there would be some options like:

• having separate access logs for different server locations
• enabling a log rotation policy
• redirecting access logs to syslog server

[andrewdavidmackenzie]

If running in docker, then we'd be logging to standard out, and let the person deploying it decide what to do with log output - rather than logging to a file and implementing rotate etc?

[whitingjr]

@andrewdavidmackenzie the issue I've found with not having log rotation is apicast pods are occasionally evicted in the cluster. Due pod ephemeral storage being exhausted.

[eloycoto]

hi @whitingjr

nowadays, APICast logs are sent to stdout, so currently, the issue fixed. Check the following env variable to send the logs to the place that you want.

APICAST_ACCESS_LOG_FILE
https://github.com/3scale/APIcast/blob/master/doc/parameters.md#apicast_access_log_file

[whitingjr]

hi @eloycoto
Thanks for pointing out the env variable. I had seen it beforehand. It does not solve the issue I am seeing.
Console output to stdout get's written to disk. It's only a matter of time before the file starts to cause ephemeral storage quota limits to be exceeded. Only by killing apicast pods is the storage space freed up.
I guess the question is do you see that as an acceptable way of operation for a service/pod ?

[srinivma1]

I believe the original query is still not answered. We need a way to log API request and response payload for all the methods(GET, POST) etc. How can this be achieved?

[eloycoto]

Regarding logging the Payload, there is a reason why we don't do that, and mainly is because of the size, and the need to read the body, that it's expensive.

If you need to log the payload, a custom policy can be added, and you can follow similar code as logging policy.

APIcast/gateway/src/apicast/policy/logging/logging.lua

Lines 138 to 152 in be91600

	function _M:log(context)
	ngx.var[ngx_var_access_logs_enabled] = self.enable_access_logs_val
	if self:use_default_access_logs() then
	return
	end
	-- Extended log is now enaled, disable the default access_log
	disable_default_access_logs()
	local extended_context = get_request_context(context or {})
	if self.condition and not self.condition:evaluate(extended_context) then
	-- Access log is disabled here, request does not match, so log is disabled
	-- for this request
	disable_extended_access_log()
	return
	end

[lucamaf]

created the policy for this myself