[Question/Feature Request] Http Library lacks headers on toFile requests

[mohsin]

Hi, I'm using the Http class for my network requests and I found that when I set the toFile flag, it also sets CURLOPT_HEADER to false. I can't even override this using $http->setOption(CURLOPT_HEADER, true); because the code that sets this occurs after the request headers are set. Is there any reason this is built this way?

Use case: I'm calling an API which returns bulk data that is paginated with the next page Link parameter in the header so continue fetching data since they follow the IETF standard RFC 5988 with an expiry timeout. Processing the data based on my application business logic consumes too much time so I fetch each Link and dump the data into temp files to process them later. However, when I dump the first page data and check the headers it's returns empty due to this cURL option being set.

[bennothommo]

@SaifurRahmanMohsin CURLOPT_HEADER sets whether the headers will be included in the output - essentially, when downloading as a file, this would result in the headers being injected into the actual file content. If you want some sort of header information included in the file content, it should be part of the response body.

[mohsin]

Thank you for the explanation @bennothommo. I still feel that Http Rain Library could be better to include the header on file requests too as it leads to a more consistent API (because that appears to be the goal of using this over usual cURL calls).

My ideas:

1. There seems to be a CURLOPT_HEADERFUNCTION which could be used to get the header data. Maybe use this function to fetch the headers into the header array?
2. Alternatively, if that won't work and is a bad solution then I found one solution that uses CURLOPT_WRITEHEADER to write headers separately to a file. What if this is instead written to a tmp file and the tmp file is read immediately into the headers array after the curl_close call in the send function and the tmp file is purged.

Wouldn't these be good solutions? Kindly let me know your thoughts and if possible I could even make a PR if you think this might work and is an enhancement worth having.

[daftspunk]

Hey @SaifurRahmanMohsin

I've added this to the API for you:

$a = function($curl, $header) {
    echo 'I am a header: '. $header;
};

Http::get('https://google.com', function($http) use ($a) {
    $http->headerCallback($a);
});

It'll work with toFile without tainting the file contents. It'll be released soon after I finish closing out all the issues ✌️