Skip to content

Request priority in HAR file for Chromium based browsers#164

Open
ozcoder wants to merge 8 commits intocloudflare:mainfrom
ozcoder:request_priority
Open

Request priority in HAR file for Chromium based browsers#164
ozcoder wants to merge 8 commits intocloudflare:mainfrom
ozcoder:request_priority

Conversation

@ozcoder
Copy link
Copy Markdown
Contributor

@ozcoder ozcoder commented Mar 2, 2026

Adds _priority and _initialPriority to the HAR file when testing with a Chromium based browser.

Closes #16

Store the initial priority of each request.
Store if the priority changes.
Insert the initial priority value into the HAR file.
Insert the final priority value into the HAR file.
Set types correctly on newPriorities and priorities.
Check before using variables.
Removed console debugging.
Added fields to HarEntry.
@ozcoder ozcoder requested a review from a team March 2, 2026 05:59
@sergeychernyshev sergeychernyshev added the ticket This label indicates that internal ticket was created to track it. label Mar 2, 2026
@sufian-cf
Copy link
Copy Markdown
Contributor

The change itself looks reasonable, but I'm not able to verify it locally. How can I run telescope so that there are _priority and _initialPriority fields produced within the har?

@ozcoder
Copy link
Copy Markdown
Contributor Author

ozcoder commented Mar 10, 2026

@sufian-cf if you run from the command line
npx . -u https://www.cloudflare.com/ -b chrome --timeout 60000 --debug
it will generate the files in a sub directory of the results directory.
Inside that is the pageload.har file which you can make pretty however you want (command or online) to make it easier to read. Eg. cat pageload.har | jq . > pageload.har.json
Edit that JSON file and search for a url entry that has Hero-globe as part of the filename. At the end of its entry it should have "_initialPriority": "Low" and the _priority value should be different, probably High since the browser would want to upgrade the priority of that image.
Other images would be upgraded too, but not all of them.

MildMax
MildMax previously approved these changes Mar 16, 2026
Copy link
Copy Markdown

@MildMax MildMax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

verified working order of the initialPriority , priority, and resourceType in the chrome browser

Copy link
Copy Markdown
Member

@sergeychernyshev sergeychernyshev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a good solution, we can make a couple improvements, see comments below.

Also, if you can make a fixture for testing the case with changing priority, it would be great. And maybe for testing duplicate requests.

Comment thread src/chromeRunner.ts Outdated
const fullURL = request.url + (request.urlFragment || '');
const resourceType = type || 'Other';

if (occasion.priorities[fullURL as keyof PriorityInfo]) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use requestId as a key instead of request URL?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mapping by URL will break if there are multiple requests to the same URL.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was using an array to get around that.
Now, using a slightly more complicated way, I can get the requestId from the CDP universe to the Playwright universe.

Comment thread src/chromeRunner.ts Outdated
Comment thread src/testRunner.ts
Comment thread src/chromeRunner.ts Outdated
ozcoder added 2 commits March 22, 2026 12:30
Create mapping of hash of url and starting time to requestId.
Gather resourceType is a better place.
Fixup types.
Comment thread src/chromeRunner.ts
const { requestId, response } = params;

if (response.timing) {
const hashString = crypto.createHash('sha256').update(response.url + response.timing.sendStart).digest('hex');
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we hash response.url + response.timing.sendStart

But when we look the hash up (in src/testRunner.ts), we use request.url + request.timing.requestStart

Is this difference intentional?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are the same value. It's because we are dealing with the CDP universe in one place and the Playwright universe in the other place. I had to find a stable attribute that was known to the request and the response and would be different when the same URL is requested more than once.

Comment thread src/chromeRunner.ts
const client: CDPSession = await page.context().newCDPSession(page);
await client.send('Network.enable');

let occasion = this;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: this closure isn't needed -- this will still be bound to the same object in the event handlers below, since they're arrow functions.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yes. Old habit.

Comment thread src/chromeRunner.ts
async createPage(browser: BrowserContext): Promise<Page> {
const page = browser.pages()[0];
const client: CDPSession = await page.context().newCDPSession(page);
await client.send('Network.enable');
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible Network.requestWillBeSent / other events below will fire before this client.send('Network.enable')?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't happen since even preparePage hasn't been called until the end of this method.

@sufian-cf
Copy link
Copy Markdown
Contributor

Hey @ozcoder, my apologies for the delay. I've taken a look at this approach and have manually verified that it sometimes works, but not for all requests. I've left a few comments, and overall am a bit concerned with the approach.

That being said, I think this has surfaced a real bug in our existing code. If the page load triggers multiple requests with the same URL, we're going to only be updating performance information on the first one, since we look up the corresponding entry via findIndex on just the URL.

I've just created #243 and will follow up with a fix that introduces a unique id per request we can use to match things.

@sufian-cf
Copy link
Copy Markdown
Contributor

Ok, I have a PR for the fix in #243 which should give each request a unique identifier that can be used to cross-reference: #244

Once that mechanism is used, and there's some automated test coverage verifying the presence of the priority fields in the HAR, this PR should be good to get in

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ticket This label indicates that internal ticket was created to track it.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Capture network/request priority in Chrome

4 participants