http_client fails to handle 301 with just a path properly

[ryan-dyer-sp]

Bug report

What operating system and version are you using?

debian

What version of osquery are you using?

4.0.2_1

What steps did you take to reproduce the issue?

• Run osquery on AWS where the EC2 instance has a role assigned to it which would allow pushes to firehose.
• Do not supply any credentials to osquery (it should attempt to use ec2 metadata to determine creds as final fall through)

What did you expect to see?

attempts to push to firehose succeed

What did you see instead?

Exception making HTTP request to URL (http://169.254.169.254/latest/meta-data/iam/security-credentials): Invalid URL

I built my own version of osquery to debug further and found that when osquery attempts to reach this url it receives a 301 redirect to "Location: /latest/meta-data/iam/security-credentials/". It then passes this value as the url to URI which barfs as it doesnt match the expected URL format.

The follow_redirect logic needs to ensure the Location is translated into a proper URL.

[theopolis]

IIRC the AWS-based loggers are already setting "follow redirects" to true. If this is the case then this behavior is concerning.

[theopolis]

The client is created here: https://github.com/osquery/osquery/blob/master/osquery/utils/aws/aws_util.cpp#L389 and it does not set follow-redirects. We should follow up and turn this on by default.

[ryan-dyer-sp]

The issue is that the follow_redirects logic https://github.com/osquery/osquery/blob/master/osquery/remote/http_client.cpp#L415
does not account for the Location header being a relative URL.

[ryan-dyer-sp]

Can confirm this PR fixes the issue we were seeing. Able to successfully push to kinesis.