Scripting the request causes the Host header to change

[nilayp]

Proxyman version? (Ex. Proxyman 1.4.3)

2.18

macOS Version? (Ex. mac 10.14)

10.15.7

Steps to reproduce

1. Start capturing requests with SSL Proxy enabled. Execute the request without scripting enabled. Request is successful and the Host header has no port number.

2. Enable scripting for the request.

3. Resend the same request. Proxy will fail because Proxyman appended the Host header to include the port number.

The request I'm making is a very basic GET request with an Authorization token and one Query parameter called VersionId. This is executing against AWS S3.

Nothing I can do allows me to remove the port number from Proxyman. I tried to edit the request manually, or use a script to edit it. I tried to toggle the "preserveHostHeader" flag. If the scripting is enabled, a port number is appended. If the script is not enabled, no port number is appended.

I also copied the request as a cUrl command from Proxyman, removed the port number from the Host header and it was successful. I'm 100% confident my request is correct, if the port number isn't present.

The script I used it very basic and does nothing more than log the request content.

function onRequest(context, url, request) {
  console.log(request);
  return request;
}

The logged console output does not include the port number appended to the Host header. It seems the port is appended somewhere after the script execution.

---------------------------------
11:37:27.379: [onRequest] with Request ID=13
{
	"headers": {
		"x-amz-content-sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
		"User-Agent": "python-urllib3/1.26.3",
		"Host": "nilayp-versioned.s3-us-west-1.amazonaws.com",
		"x-amz-date": "20210219T193727Z",
		"Accept-Encoding": "identity",
		"Authorization": [omitted]
	},
	"schema": "https",
	"path": "/mytextfile.txt",
	"port": 443,
	"host": "nilayp-versioned.s3-us-west-1.amazonaws.com",
	"queries": {
		"versionId": [omitted]
	},
	"method": "GET",
	"preserveHostHeader": false
}
11:37:27.380 onRequest() is executed!
----------------------------------

[NghiaTranUIT]

Hi @nilayp, by default, Proxyman will construct the Host header from the Host and port, so it appears as you see.

It's useful if we'd map to different Host and Port (e.g. mydomain.com to localhost:3000).

I suppose that I should omit the port if it's using a default port (80 for HTTP and 443 for HTTPS), so it can resolve your problem 😄

[NghiaTranUIT]

@nilayp Please try this Beta build: https://proxyman.s3.us-east-2.amazonaws.com/beta/Proxyman_2.18.0_Omit_Default_Port_In_Host_Header_Script.dmg

This build will omit the default port: 443 for HTTPS and 80 for HTTP 👍

[nilayp]

@NghiaTranUIT - Thank you so much. I can confirm this has worked.

Just so you know why this is important - AWS request headers are signed on the client side. When received by AWS (or service that is using AWS compatible requests) - elements of the request are used to compute a signature... if the newly computed signature does not matches the signature provided in the request, the request is rejected by the service.

One of the required headers in the signature calculation is "Host". When it changed, the signatures no longer matched.

Thanks again for the super fast turnaround! It is much appreciated.

[NghiaTranUIT]

Glad to know that it works for you 🎉