Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adds integration test for clients that cancel via process exit #887

Merged
merged 3 commits into from Aug 9, 2019

Conversation

@shamsimam
Copy link
Contributor

commented Jul 25, 2019

Changes proposed in this PR

  • adds integration test for clients that cancel via process exit

Why are we making these changes?

A gRPC client exiting abruptly should also be treated as a client cancellation.

waiter logs: request identified as client error

2019-07-25 17:05:23,482 INFO  waiter.core [qtp709883348-181] - [CID=wgttgbsce143478578297273] request received: {:internal-protocol HTTP/2.0, :request-id 8282a4881311-f0370db67dd9ecd-http, :remote-addr 127.0.0.1, :client-protocol HTTP/2.0, :headers {te trailers, grpc-accept-encoding gzip, host 127.0.0.1:9091, x-cid wgttgbsce143478578297273, content-type application/grpc, x-waiter-token ^SERVICE-ID#w9091-wgttgbsce143476858830466-29bb9aee2ea50f08cdc2044458bbba19, user-agent grpc-java-netty/1.20.0, grpc-timeout 59716136u}, :content-length nil, :content-type application/grpc, :character-encoding nil, :uri /courier.Courier/CollectPackages, :query-string nil, :router-id r9091-7f0c6731ef91-92848c202af0dda, :scheme :http, :request-method :post}
2019-07-25 17:05:23,489 INFO  waiter.process-request [async-dispatch-29] - [CID=wgttgbsce143478578297273] suggested instance: w9091-wgttgbsce143476858830466-29bb9aee2ea50f08cdc2044458bbba19.827dd65a2e7d-6455d7bb45bb0638 127.0.0.4 10500
2019-07-25 17:05:29,164 ERROR waiter.process-request [qtp709883348-181] - [CID=wgttgbsce143478578297273] unable to stream request bytes, aborting request
2019-07-25 17:05:29,170 INFO  waiter.process-request [qtp709883348-181] - [CID=wgttgbsce143478578297273] streamed 1480 bytes from request input stream
2019-07-25 17:05:29,171 INFO  waiter.process-request [async-dispatch-37] - [CID=wgttgbsce143478578297273] exception occurred while streaming response for w9091-wgttgbsce143476858830466-29bb9aee2ea50f08cdc2044458bbba19
2019-07-25 17:05:29,172 INFO  waiter.process-request [async-dispatch-37] - [CID=wgttgbsce143478578297273] java.nio.channels.ClosedChannelException nil identified as :instance-error
2019-07-25 17:05:29,172 INFO  waiter.process-request [async-dispatch-37] - [CID=wgttgbsce143478578297273] clojure.lang.ExceptionInfo error in frontend request identified as :client-error
2019-07-25 17:05:29,172 INFO  waiter.process-request [async-dispatch-37] - [CID=wgttgbsce143478578297273] clojure.lang.ExceptionInfo error occurred after streaming 487 bytes in response identified as :client-error
2019-07-25 17:05:29,172 INFO  waiter.process-request [async-dispatch-37] - [CID=wgttgbsce143478578297273] sending poison pill to response channel
2019-07-25 17:05:29,172 INFO  waiter.process-request [async-dispatch-37] - [CID=wgttgbsce143478578297273] done processing request :client-error

Test state output: cancel and cancel_handler present

2019-07-25 17:05:31,205 INFO  waiter.grpc-test [main] - [CID=wgttgbsce143478578297273] received response StateReply{cid=wgttgbsce143478578297273, response=[INIT, READY, RECEIVE_MESSAGE, SEND_HEADERS, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, RECEIVE_MESSAGE, SEND_MESSAGE, CANCEL, CANCEL_HANDLER, CLOSE]}

Script output: script exiting during cancellation

wgttgbsce143478578297273 initializing plaintext client at 127.0.0.1:9091
wgttgbsce143478578297273 will try to send package from from ...
wgttgbsce143478578297273 acquiring semaphore before sending request id-0
wgttgbsce143478578297273 sending message CourierRequest{id=id-0, from=from, message.length=9}
wgttgbsce143478578297273 acquiring semaphore before sending request id-1
wgttgbsce143478578297273 headers received from server:Metadata(content-type=application/grpc,x-cid=wgttgbsce143478578297273,grpc-encoding=identity,grpc-accept-encoding=gzip,set-cookie=x-waiter-auth=TlBZAvYN3wc%2FxRya1Zf%2BK2aiUrdZEM6M3r8lUQv2tN8bpZTLI96fUGnSqkDuIow%2BZ
F2jBg%3D%3D;Max-Age=86400;Path=/;HttpOnly=true)
wgttgbsce143478578297273 received response CourierSummary{count=1, length=9}
wgttgbsce143478578297273 releasing semaphore after receiving response
wgttgbsce143478578297273 sending message CourierRequest{id=id-1, from=from, message.length=9}
...
wgttgbsce143478578297273 sending message CourierRequest{id=id-49, from=from, message.length=10}
wgttgbsce143478578297273 received response CourierSummary{count=50, length=490}
wgttgbsce143478578297273 releasing semaphore after receiving response
wgttgbsce143478578297273 cancelling sending messages

@shamsimam shamsimam added the wip label Jul 25, 2019

@shamsimam shamsimam self-assigned this Jul 25, 2019

@shamsimam shamsimam force-pushed the grpc-client-exit-cancellation-support branch 9 times, most recently from d254139 to a5add2b Jul 25, 2019

@shamsimam shamsimam removed the wip label Jul 30, 2019

@shamsimam shamsimam force-pushed the grpc-client-exit-cancellation-support branch from b8fde0b to 2bcf4d8 Jul 31, 2019

@shamsimam shamsimam requested a review from sradack Jul 31, 2019

(println (du/date-to-str (t/now) date-formatter) correlation-id message)))]
(GrpcClient. host port log-function)))

(try

This comment has been minimized.

Copy link
@sradack

sradack Jul 31, 2019

Contributor

Is this a command line utility?

This comment has been minimized.

Copy link
@shamsimam

shamsimam Jul 31, 2019

Author Contributor

Yes, this is invoked from the test in a separate process as it will exit the JVM.

{:keys [err out]} (sh/sh "lein" "exec" "-p"
"test-files/grpc/grpc_client_exit.clj"
host (str h2c-port) service-id correlation-id cookie)]
(println "exec stdout:" out) ;; TODO shams - change to log/info

This comment has been minimized.

Copy link
@sradack

sradack Jul 31, 2019

Contributor

Open TODOs

This comment has been minimized.

Copy link
@shamsimam

shamsimam Jul 31, 2019

Author Contributor

Removed.

@shamsimam shamsimam force-pushed the grpc-client-exit-cancellation-support branch 4 times, most recently from 856fc88 to ab26bf3 Jul 31, 2019

@shamsimam shamsimam force-pushed the grpc-client-exit-cancellation-support branch 2 times, most recently from b42f490 to 64deca6 Aug 2, 2019

@sradack

sradack approved these changes Aug 2, 2019

@shamsimam shamsimam force-pushed the grpc-client-exit-cancellation-support branch from 64deca6 to 8595574 Aug 8, 2019

@shamsimam

This comment has been minimized.

Copy link
Contributor Author

commented Aug 8, 2019

@sradack build is green

@sradack

sradack approved these changes Aug 9, 2019

@sradack sradack merged commit 5584aa6 into master Aug 9, 2019

2 checks passed

Mergeable Mergeable Run has been Completed!
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@sradack sradack deleted the grpc-client-exit-cancellation-support branch Aug 9, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.