HTTP2 and JA3 Not matching browser fingerprint

[FloppaFriday]

Description

as i already specified here: #101
the HTTP2 fingerprint of cycletls doesnt match the fingerprint of the browser im attempting to imitate. This not matching data can be found both in chrome and firefox.
Especially the HTTP2 issue is an problem because it causes detection. Frames send are incorrect for example the settings frame sends entirely different content as opposed to the browser.
Because of that the values that tls.peet.ws calls akamai_fingerprint and akamai_fingerpring_hash are both different from the actual browser.
additional to that newer chrome versions are returning Extension {{ 17513 }} is not Supported by CycleTLS please raise an issue when an attempt at an request is made
Also another issue to look into would be the fact that some useragents arent sending any headers for example requesting tls.peet.ws with "Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36" as a header returns the response "no useragent"

Issue Type

Bug

Operating System

Windows 10

Node Version

Other

Golang Version

No response

Relevant Log Output

No response

[Danny-Dasilva]

additional to that newer chrome versions are returning Extension {{ 17513 }} is not Supported by CycleTLS please raise an issue when an attempt at an request is made

This extension has been supported since this release. This is a custom error handler I wrote into the code. I would update your local version to the latest release since you cannot get this error unless you are at least on version 0.0.19 or earlier.

I understand the importance of Frame headers. As I mentioned in the last issue I added specific checks we are sending the correct chrome and firefox frame headers depending on the user agent.

Here's an example for the latest chrome version on windows

CycleTLS/tests/frameHeader.test.ts

Lines 6 to 48 in c84d740

	const ja3 =
	"771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-17513,29-23-24,0";
	const UA =
	"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36";
	const response = await cycleTLS.get("https://tls.fxnatic.dev/api/fp", {
	body: "",
	ja3: ja3,
	userAgent: UA,
	});
	const expectedSentFrames0 = {
	frame_type: "SETTINGS",
	length: 24,
	settings: [
	"HEADER_TABLE_SIZE = 65536",
	"MAX_CONCURRENT_STREAMS = 1000",
	"INITIAL_WINDOW_SIZE = 6291456",
	"MAX_HEADER_LIST_SIZE = 262144",
	],
	stream_id: 0,
	};
	const expectedSentFrames1 = {
	frame_type: "WINDOW_UPDATE",
	increment: 15663105,
	length: 4,
	stream_id: 0,
	};
	if (typeof response.body === "object") {
	expect(response.body?.tls?.ja3).toEqual(ja3);
	expect(response.body?.user_agent).toEqual(UA);
	if (process.platform != "darwin") {
	expect(response.body?.http2?.sent_frames[0]).toMatchObject(
	expectedSentFrames0
	);
	}
	expect(response.body?.http2?.sent_frames[1]).toMatchObject(
	expectedSentFrames1
	);
	} else {
	throw "Object decode error";
	}
	cycleTLS.exit();

Here's the lest firefox on windows

CycleTLS/tests/frameHeader.test.ts

Lines 53 to 93 in c84d740

	const ja3 =
	"771,4865-4867-4866-49195-49199-52393-52392-49196-49200-49162-49161-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-34-51-43-13-45-28,29-23-24-25-256-257,0";
	const UA =
	"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:101.0) Gecko/20100101 Firefox/101.0";
	const response = await cycleTLS.get("https://tls.fxnatic.dev/api/fp", {
	body: "",
	ja3: ja3,
	userAgent: UA,
	});
	const expectedSentFrames0 = {
	frame_type: "SETTINGS",
	length: 18,
	settings: [
	"HEADER_TABLE_SIZE = 65536",
	"INITIAL_WINDOW_SIZE = 131072",
	"MAX_FRAME_SIZE = 16384",
	],
	stream_id: 0,
	};
	const expectedSentFrames1 = {
	frame_type: "WINDOW_UPDATE",
	increment: 12517377,
	length: 4,
	stream_id: 0,
	};
	if (typeof response.body === "object") {
	expect(response.body?.tls?.ja3).toEqual(ja3);
	expect(response.body?.user_agent).toEqual(UA);
	expect(response.body?.http2?.sent_frames[0]).toMatchObject(
	expectedSentFrames0
	);
	expect(response.body?.http2?.sent_frames[1]).toMatchObject(
	expectedSentFrames1
	);
	} else {
	throw "Object decode error";
	}
	cycleTLS.exit();

I am specifically testing chrome and firefox headers on the latest browser versions. I'm not sure how to reproduce your issue. I would try updating to the latest version.

[FloppaFriday]

I investigated the issue by myself for a bit and figured out that the header order isnt matching the actual header order of firefox.
Also even after upgrading to the latest version i still have TLS_GREASE in my curves and im lacking the padding extension.
Both header length and order arent matching. I tested it multiple times but instead of the browser(length 312) the library submits headers with length 307.
Here are both results from tls.peet.ws in pastebin:
browser: https://pastebin.com/FC5vvZ2U
cycletls: https://pastebin.com/SsphAtf7

[FloppaFriday]

are there any updates to the issue?

[FloppaFriday]

The issue still occurs.
When testing i figured out older browser versions are less likely to face this bug but the newer ones do.