Problem working with Squid web proxy

[aburnheimer]

I'm having a problem going through Squid despite the changes bgreenlee suggested 3 years ago in Issue #2.

Using Twitter::JSONStream as shown at https://gist.github.com/4179836.js?file=Listen_in_on_Campfire_room.rb , I get the following results - with a little bit of annotation sprinkled in at bottom of JSONStream#send_request, and temporarily backing out the "full url" fix bgreenlee suggested, please read on:

[aburnheimer@hostname ~]$ ./Listen_in_on_Campfire_room.rb
options: {:port=>443, :method=>"GET", :content_type=>"application/x-www-f
orm-urlencoded", :auto_reconnect=>true, :content=>"", :timeout=>0, :param
s=>{}, :user_agent=>"TwitterStream", :host=>"streaming.campfirenow.com", :
proxy=>"http://...proxy_host_name removed...:3128", :auth=>"1234567890abc
def123467890abcdef123456789:X", :ssl=>true, :oauth=>{}, :path=>"/room/123
456/live.json", :filters=>[]}
send_request: ["GET /room/123456/live.json HTTP/1.1", "Host: streaming.ca
mpfirenow.com", "Accept: */*", "User-Agent: TwitterStream", "Authorizatio
n: Basic ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCD", "\r\n
"]
Tried 11 times to connect.

Meanwhile, using curl, I can generate the exact same headers, and it works...

[aburnheimer@hostname ~]$ echo $http_proxy
http://...proxy_host_name removed...:3128
[aburnheimer@hostname ~]$ echo $HTTP_PROXY
http://...proxy_host_name removed...:3128
[aburnheimer@hostname ~]$ curl -k -v -A "TwitterStream" -u 123467890abcdef12346
7890abcdef123456789:X -x $http_proxy https://streaming.campfirenow.com/room/123
467/live.json
* About to connect() to proxy ...proxy_host_name removed... port 3128 (#0)
*   Trying proxy_ip removed... connected
* Connected to proxy_host_name removed (proxy_ip removed) port 3128 (#0)
* Establish HTTP proxy tunnel to streaming.campfirenow.com:443
* Server auth using Basic with user '123467890abcdef123467890abcdef123456789'
> CONNECT streaming.campfirenow.com:443 HTTP/1.1
> Host: streaming.campfirenow.com:443
> User-Agent: TwitterStream
> Proxy-Connection: Keep-Alive
>
< HTTP/1.0 200 Connection established
<
* Proxy replied OK to CONNECT request
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* warning: ignoring value of ssl.verifyhost
* skipping SSL peer certificate verification
* SSL connection using TLS_DHE_RSA_WITH_AES_256_CBC_SHA
* Server certificate:
*   subject: CN=*.campfirenow.com,OU=Domain Control Validated - RapidSSL(R),OU=S
ee www.rapidssl.com/resources/cps (c)11,OU=GT23036785,O=*.campfirenow.com,C=US,
serialNumber=JoGozIJmZwhtvHRQCuYEMvwbe6AYqcjR
*   start date: Dec 24 15:40:13 2011 GMT
*   expire date: Jan 25 19:52:44 2014 GMT
*   common name: *.campfirenow.com
*   issuer: CN=RapidSSL CA,O="GeoTrust, Inc.",C=US
* Server auth using Basic with user '123467890abcdef123467890abcdef123456789'
> GET /room/123467/live.json HTTP/1.1
> Authorization: Basic ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCD
> User-Agent: TwitterStream
> Host: streaming.campfirenow.com
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx
< Date: Sat, 01 Dec 2012 00:41:52 GMT
< Content-Type: application/json
< Transfer-Encoding: chunked
< Connection: keep-alive
<
{"room_id":123467,"created_at":"2012/12/01 00:41:54 +0000","body":"Lorem ipsum
dolor sit amet, consectetur adipiscing elit. Donec a diam lectus. Sed sit amet i
psum mauris. Maecenas congue ligula ac quam viverra nec consectetur ante hendre
rit. Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit ame
t vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. D
onec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit
amet, c

Even just advice on where or how to investigate further would be appreciated. I've been annotating twitter-stream-0.1.16 and eventmachine-1.0.0 to try and determine where these may be diverting from whatever curl is doing right. Unfortunately, I haven't found any clues after lots of trials and errors.

My next move is to switch to the pure_ruby of eventmachine, but the initial resistance I found with that suggested to me to maybe reach out for help first.

[aburnheimer]

Doing as voloko suggested, and restoring the full url as bgreenlee suggested did not result in anymore success here:

[aburnheimer@hostname ~]$ ./Listen_in_on_Campfire_room.rb
options: {:port=>443, :method=>"GET", :content_type=>"application/x-www-form-ur
lencoded", :auto_reconnect=>true, :content=>"", :timeout=>0, :params=>{}, :user
_agent=>"TwitterStream", :host=>"proxy_ip removed", :proxy=>"http://...proxy_ho
stname removed...:3128", :auth=>"123467890abcdef123467890abcdef123456789:X", :s
sl=>true, :oauth=>{}, :path=>"/room/123456/live.json", :filters=>[]}
send_request: ["GET https://proxy_ip removed:443/room/123456/live.json HTTP/1.1
", "Host: streaming.campfirenow.com", "Accept: */*", "User-Agent: TwitterStream
", "Authorization: Basic ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZAB
CD", "\r\n"]

nor here - note the port number is present in the Host header for this attempt:

[aburnheimer@hostname ~]$ ./Listen_in_on_Campfire_room.rb
options: {:port=>443, :method=>"GET", :content_type=>"application/x-www-form-ur
lencoded", :auto_reconnect=>true, :content=>"", :timeout=>0, :params=>{}, :user
_agent=>"TwitterStream", :host=>"proxy_ip removed", :proxy=>"http://...proxy_ho
stname removed...:3128", :auth=>"123467890abcdef123467890abcdef123456789:X", :s
sl=>true, :oauth=>{}, :path=>"/room/123456/live.json", :filters=>[]}
send_request: ["GET https://proxy_ip removed:443/room/123456/live.json HTTP/1.1
", "Host: streaming.campfirenow.com:443", "Accept: */*", "User-Agent: TwitterSt
ream", "Authorization: Basic ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWX
YZABCD", "\r\n"]

[aburnheimer]

Found it... Have to have the options look as such, just as voloko suggested way-back:

options: {:auth=>"123467890abcdef123467890abcdef123456789:X", :method=>"GET",
:auto_reconnect=>true, :content=>"", :content_type=>"application/x-www-form-url
encoded", :user_agent=>"TwitterStream", :host=>"proxy_ip removed", :params=>{}, :t
imeout=>0, :proxy=>nil, :port=>3128, :oauth=>{}, :path=>"https://streaming.camp
firenow.com:443/room/123456/live.json", :ssl=>false, :filters=>[]}
send_request: ["GET https://streaming.campfirenow.com:443/room/123456/live.json
 HTTP/1.1", "Host: streaming.campfirenow.com", "Accept: */*", "User-Agent: Twit
m", "Authorization: Basic ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZA
BCD", "\r\n"]

I'll wrap this into a pull request, or something similar later this evening.