Provide access to raw request data

[Zapotek]

It would be nice if we could access the raw HTTP request data for later inspection/auditing/debugging.

[hanshasselberg]

I agree and it is possible! There is a similar ticket for Typhoeus:
typhoeus/typhoeus#247.
On Jul 13, 2013 9:21 PM, "Tasos Laskos" notifications@github.com wrote:

It would be nice if we could access the raw HTTP request data for later
inspection/auditing/debugging.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/63
.

[Zapotek]

Yeah I was hoping to access this via Typhoeus too but I figured it was up to Ethon to provide that access so I opened the ticket here.

[hanshasselberg]

You are right!
On Jul 15, 2013 2:08 PM, "Tasos Laskos" notifications@github.com wrote:

Yeah I was hoping to access this via Typhoeus too but I figured it was up
to Ethon to provide that access so I opened the ticket here.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/63#issuecomment-20965416
.

[linrock]

Here's an example snippet that prints out the raw request headers using CURLOPT_DEBUGFUNCTION.

callback = proc do |handle, type, data, size|
  if type == 2 # CURLINFO_HEADER_OUT
    data = FFI::Pointer.new(data) unless data.is_a? FFI::Pointer
    request_headers = data.read_string_length size.to_i
    puts request_headers
  end
end

easy = Ethon::Easy.new url: "www.example.com",
                       followlocation: true,
                       verbose: true,
                       debugfunction: callback
easy.perform

Prints out:

GET / HTTP/1.1
Host: www.example.com
Accept: */*

GET / HTTP/1.0
Host: example.iana.org
Accept: */*

In the proc, data should be a pointer and size should be a number, but for some reason the types are reversed. Here's the expected prototype:

int curl_debug_callback (CURL *, curl_infotype, char *, size_t, void *);

Source: http://curl.haxx.se/libcurl/c/curl_easy_setopt.html#CURLOPTDEBUGFUNCTION

[linrock]

To support always saving the raw request headers in ethon, a callback could be added to Ethon::Easy::Callbacks using debugfunction to grab the request headers, but the verbose option would also need to be set to true.

Should this be the default behavior?

[hanshasselberg]

@linrock I think so.

[hanshasselberg]

@linrock do you want to take a stab at it? Your code works, should be a matter of putting it into the right place.

[linrock]

@i0rek Sure, I'll work on it.

[linrock]

Please take a look! I'll updated as needed.

After this set of changes, I did run into an issue with running the specs, where if I run them individually, they all work fine, but if I run them all at once via rspec without arguments, it hangs and eventually segfaults on the "when no follow location" test. Reproduced this on both mac and linux on ruby 2.0.0-p247. Not sure why this happens.

[hanshasselberg]

Thanks to @linrock this is now available in master via easy.debug_info