499 reflect on service

[trevezani]

I tries capture the close connection in the service associate with the ingress-nginx but I cant find a way. When this situation occur the log of nginx shows the correct status code on log:

172.21.0.224 - user [29/Apr/2024:13:21:41 +0000] "POST /execute HTTP/1.1" 499 0 "-" "Apache-HttpClient/4.5.12 (Java/1.8.0_382)" 409 2.702 [service-8080] [] 172.21.72.106:8080 0 2.702 - da6c85d6276498086c97ccb1380ed096

But in the service the process continue. I think the correct behavior is the nginx close the connection with the service to interrupt the process.

I'm following the instructions to install on aws changing only VPC CIDR and ACM. I installed the version 1.10.0 on k8s version 1.29.

In the documentation I only found the flag proxy_ignore_client_abort .

Have a way to change this behavior?

[k8s-ci-robot]

This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[longwuyuan]

/remove-kind feature
/kind support

If the backend app returned a 499 and it was logged and also the client received the 499, I think the controller has done the job of routing the request and the response.

Can you provide more details about this issue. Specially what was the request sent as a curl command exactly as executed from shell.

One of the questions asked in the new bug report template is this curl command and also many other info like kubectl describe output of the ingress resource. Without all this data, its not possible for a reader to reproduce the problem or even create an environment that is just like your environment.

If your app is designed to start a process or a thread and then not respond back to the client appropriately, then you need to explain what is expected from the controller as per K8S Ingress-API specs.

When you have provided this information, please re-open this issue. Please click the new bug report button just to see the questions asked there. Then don't create a new issue but edit this issue description and answer those questions in md format, so the readers here can replicate the problem and comment on the issue.

Please use the image from httpbun.com to create a test if possible. Readers here can not use your application image as that is not a standard way to debug ingress-nginx controller features.

thanks.

/close

[k8s-ci-robot]

@longwuyuan: Closing this issue.

In response to this:

/remove-kind feature
/kind support

If the backend app returned a 499 and it was logged and also the client received the 499, I think the controller has done the job of routing the request and the response.

Can you provide more details about this issue. Specially what was the request sent as a curl command exactly as executed from shell.

One of the questions asked in the new bug report template is this curl command and also many other info like kubectl describe output of the ingress resource. Without all this data, its not possible for a reader to reproduce the problem or even create an environment that is just like your environment.

If your app is designed to start a process or a thread and then not respond back to the client appropriately, then you need to explain what is expected from the controller as per K8S Ingress-API specs.

When you have provided this information, please re-open this issue. Please click the new bug report button just to see the questions asked there. Then don't create a new issue but edit this issue description and answer those questions in md format, so the readers here can replicate the problem and comment on the issue.

Please use the image from httpbun.com to create a test if possible. Readers here can not use your application image as that is not a standard way to debug ingress-nginx controller features.

thanks.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[trevezani]

I simulated using jmeter calling a endpoint that have a delay of 10s but with 2 I interrupted the call. In the log of the ingress-nginx-controller (kubectl logs -f ingress-nginx-controller-598d89fc8d-5567n -n ingress-nginx) was printed

172.21.0.224 - user [30/Apr/2024:16:49:27 +0000] "GET /test/slow/5000 HTTP/1.1" 499 0 "-" "Apache-HttpClient/4.5.12 (Java/1.8.0_382)" 301 1.867 [rjapi-service-eks-test-8080] [] 172.21.66.9:8080 0 1.866 - 820429946f9c36efded31fa093f7e104

but the service continued the process.

This service was write in java with springboot. In the log of service I could see the complete process.

2024-04-30T16:49:52.980Z INFO 1 --- [service-test] [nio-8080-exec-5] b.c.r.controller.TestController : CALL /slow/10000
2024-04-30T16:50:02.981Z INFO 1 --- [service-test] [nio-8080-exec-5] b.c.r.controller.TestController : FINISHED /slow

I made a another test creating a LoadBalancer Service and I had the behavior that I want.

Calling this url I have this behavior when I interrupted the connection.

org.springframework.web.context.request.async.AsyncRequestNotUsableException: ServletOutputStream failed to flush: java.io.IOException: Connection reset by peer
Caused by: java.io.IOException: Connection reset by peer

/reopen

[trevezani]

@longwuyuan: Closing this issue.

In response to this:

/remove-kind feature
/kind support
If the backend app returned a 499 and it was logged and also the client received the 499, I think the controller has done the job of routing the request and the response.
Can you provide more details about this issue. Specially what was the request sent as a curl command exactly as executed from shell.
One of the questions asked in the new bug report template is this curl command and also many other info like kubectl describe output of the ingress resource. Without all this data, its not possible for a reader to reproduce the problem or even create an environment that is just like your environment.
If your app is designed to start a process or a thread and then not respond back to the client appropriately, then you need to explain what is expected from the controller as per K8S Ingress-API specs.
When you have provided this information, please re-open this issue. Please click the new bug report button just to see the questions asked there. Then don't create a new issue but edit this issue description and answer those questions in md format, so the readers here can replicate the problem and comment on the issue.
Please use the image from httpbun.com to create a test if possible. Readers here can not use your application image as that is not a standard way to debug ingress-nginx controller features.
thanks.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/reopen

[k8s-ci-robot]

@trevezani: Reopened this issue.

In response to this:

@longwuyuan: Closing this issue.

In response to this:

/remove-kind feature
/kind support
If the backend app returned a 499 and it was logged and also the client received the 499, I think the controller has done the job of routing the request and the response.
Can you provide more details about this issue. Specially what was the request sent as a curl command exactly as executed from shell.
One of the questions asked in the new bug report template is this curl command and also many other info like kubectl describe output of the ingress resource. Without all this data, its not possible for a reader to reproduce the problem or even create an environment that is just like your environment.
If your app is designed to start a process or a thread and then not respond back to the client appropriately, then you need to explain what is expected from the controller as per K8S Ingress-API specs.
When you have provided this information, please re-open this issue. Please click the new bug report button just to see the questions asked there. Then don't create a new issue but edit this issue description and answer those questions in md format, so the readers here can replicate the problem and comment on the issue.
Please use the image from httpbun.com to create a test if possible. Readers here can not use your application image as that is not a standard way to debug ingress-nginx controller features.
thanks.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[longwuyuan]

What are you expecting the controller to do ? Can you explain in detail what you expect ?

[longwuyuan]

Also, it seems like you have a java app in your testing.
Please do your tests using an image like nginx:alpine like kubectl create deploy test0 --image nginx:alpine. The controller behavior is same for both your image as well as vanilla nginx image but the vanilla nginx image removes the complexity of considering custom apps or frameworks to be a factor in closing connections to upstream.

[trevezani]

I expected the ingress closes the connection with the service and in the service the exception happens like below:

org.springframework.web.context.request.async.AsyncRequestNotUsableException: ServletOutputStream failed to flush: java.io.IOException: Connection reset by peer
Caused by: java.io.IOException: Connection reset by peer

[longwuyuan]

Using your app is not a conclusive information that the controller is not closing connection as required.

Please use the nginx:alpine image like this

• kubectl create deployment test0 --image nginx:alpine
• kubectl expose deployment test0 --port 80
• kubectl create ing test0 --class nginx --rule test0.mydomain.com/"*"=test0:80
• curl --resolve test0,mydomain.com:80:my ingresscontroller external ip test0,mydomain.com

Do not use your app

Rreport all the outputs of commands like kubectl describe and kubectl logs and curl command etc or

[trevezani]

curl -k --resolve test0.mydomain.com:443:x.x.x.x https://test0.mydomain.com

<title>Welcome to nginx!</title> <style> html { color-scheme: light dark; } body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style>

Welcome to nginx!

If you see this page, the nginx web server is successfully installed and working. Further configuration is required.

For online documentation and support please refer to nginx.org.
Commercial support is available at nginx.com.

Thank you for using nginx.

log ingress-nginx

172.21.0.224 - - [30/Apr/2024:19:38:48 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/8.4.0" 81 0.001 [test-test0-80] [] 172.21.47.222:80 615 0.001 200 23475003b3b98b4d6c7e8457b8064172

log nginx pod

172.21.41.192 - - [30/Apr/2024:19:38:48 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/8.4.0" "172.21.0.224"

---

curl -k -m .470 --resolve test0.mydomain.com:443:x.x.x.x https://test0.mydomain.com

log ingress-nginx

172.21.0.224 - - [30/Apr/2024:19:37:13 +0000] "GET / HTTP/1.1" 499 0 "-" "curl/8.4.0" 81 0.000 [test-test0-80] [] 172.21.47.222:80 0 0.000 - 2889e565a82f4db8e58280450ad03e61

log nginx pod

172.21.41.192 - - [30/Apr/2024:19:37:13 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/8.4.0" "172.21.0.224"

[longwuyuan]

I think the issue description from you is clear now.

• This will not see any change from the project as there is nothing to be done
• This code 499 seems special to nginx because nginx has multi-threaded design for handling requests
• After searching on internet, it seems clear that 499 is returned to you because curl client closed connection
• You closed connection within .470 seconds so you got 499 response which is accurate response
• The service --type LoadBalancer does not use nginx so you don't see this behavior there

[trevezani]

Right, when I force the ingress to close connection the nginx identify the problem and put on log the correct behavior... But for me the correct is the nginx make the same with the service... I don't know if is possible. When I use a traefik like a reverse proxy and close the connection, the traefik close the connection with the backend associate with him.

This scenario I use in a docker environment.

[longwuyuan]

Its clear what you expect.
But it is not possible.

Please close the case if no more questions