ssl doent work with stream:t set

[nightshade427]

When setting stream to "t" when calling tcp-ssl-connect cl-async fails. This is used in libraries likedrakma-async.

[orthecreedence]

Should be fixed in 939644c =]

[nightshade427]

I get HTTP connection timeout: -1: HTTP stream client timed out. when proxying several ssl resources at the same time. I dont get this error when proxying lots of non-ssl resources.

[orthecreedence]

Is your proxy terminating SSL and requesting HTTP resources? Or is it terminating SSL then requesting other SSL resources?

[nightshade427]

It is non ssl to ssl:

http-app.js => wookie => carrier/drakma => https-app.js on another server

[nightshade427]

Actually this seems to happen on non ssl too under load.

Proxing a webpage that has serveral assets (css, images, js, etc) on it, and repeatedly hitting refresh shows timeouts.

[nightshade427]

It errors with the above HTTP connection timeout: -1: HTTP stream client timed out. with drakma. With carrier it seems as though the the error just gets absorbed and doesnt surface but it never completets.

Its as though there is a lower level cl-async error and its triggering http-connection timeout error in drakma, and in carrier its just ignoring the error and silently bombing on it?

I will try and dig further to see what the error might be.

[nightshade427]

Okay I found a case that repos it every time. Going to try and see why its erroring.

[nightshade427]

It seems as though the finish-request callback isnt being called. Its like fast-http reads the data fine each time, but never triggers the finshed callback, fast-http does process the headers and start processing the body though, but never finishes.

This times out (finish callback never called, but it reads the headers and starts to read the body)(You may have to try it a couple times to get it to repo):

(as:with-event-loop (:catch-app-errors t)                                                                      
           (bb:catcher                                                                                                  
            (bb:multiple-promise-bind (body status headers) (das:http-request "https://www.firma8.com/assets/bg.png")   
              (format t "~&status: ~s~&headers: ~s" status headers))                                                    
            (t (e) (format t "error: ~a" e))))

This hangs forever:

(as:with-event-loop (:catch-app-errors t)                                                                                                                                
           (bb:catcher                                                                                                                                                            
            (bb:multiple-promise-bind (body status headers) (carrier:request "https://www.firma8.com/assets/bg.png" :return-body t)                                               
              (format t "~&status: ~s~&headers: ~s" status headers))                                                                                                              
            (t (e) (format t "error: ~a" e))))

This doesnt have any issue:

(as:with-event-loop (:catch-app-errors t)                                                                      
           (bb:catcher                                                                                                  
            (bb:multiple-promise-bind (body status headers) (drakma:http-request "https://www.firma8.com/assets/bg.png")   
              (format t "~&status: ~s~&headers: ~s" status headers))                                                    
            (t (e) (format t "error: ~a" e))))

[nightshade427]

Let me know if you need anything else to help repo the issue.

Its like the stream stops getting data from the server somehow?

[nightshade427]

Okay I got ssl disabled for the site above.
This works:

(as:with-event-loop (:catch-app-errors t)                                                                      
           (bb:catcher                                                                                                  
            (bb:multiple-promise-bind (body status headers) (das:http-request "http://www.firma8.com/assets/bg.png")    
              (format t "~&status: ~s~&headers: ~s" status headers))                                                    
            (t (e) (format t "error: ~a" e))))

This does not:

(as:with-event-loop (:catch-app-errors t)                                                                      
           (bb:catcher                                                                                                  
            (bb:multiple-promise-bind (body status headers) (das:http-request "https://www.firma8.com/assets/bg.png")    
              (format t "~&status: ~s~&headers: ~s" status headers))                                                    
            (t (e) (format t "error: ~a" e))))

The only difference is the ssl. I think it has something to do with ssl-run-state, its like it isnt seeing new data from the socket.

[nightshade427]

It also only seems to effect files of 32k or larger? When I do png's over ssl that are smaller than that they work.

[nightshade427]

Found and fixed once I saw ones under 32K worked. The size of the ssl response chunks are 16k, and the code currently called ssl-run-state twice, so if the size was larger it was never called again and data was left hanging in the stream.

Making pull request now for the fix. Will link to here for context.

[orthecreedence]

Thanks a ton for taking a look at this. Sorry I've been MIA the past week or so, had a lot of busy days/nights. Have you found this works for all larger file sizes?

[nightshade427]

All the large ones I've tried so far :)

[orthecreedence]

Perfect, thanks!