Iframe proxied requests may have invalid Referer header

[KarolNov]

What is your Test Scenario?

Currently I'm implementing third party integration which uses iframe to get their SDK. They use API keys and allow-list to either deny iframe request or to accept it. E2E tests using Testcafe are failing with the "deny" error whilst manual tests on the same environment work. Localhost host which Testcafe is using is whitelisted for the given API key. The issues seems to be connected to the request being sent with invalid header:
Here's how I found it

class 3rdPartySDKHook extends RequestHook {
    constructor() {
        const requestFilterRules = [/* regex filtering requests to only those iframe is performing */];
        super(requestFilterRules);
    }

    public async onRequest(event: any): Promise<void> {
        console.log(event.requestOptions.headers);
    }

    public async onResponse(): Promise<void> {
        // noop
    }
}

const hook = new 3rdPartySDKHook();

fixture`My fixture`
.page(TEST_URL).requestHooks(hook);

What is the Current behavior?

The Referer header is set to unproxied version of iframe src.

What is the Expected behavior?

The Referer header is set to url of page rendering iframe.

What is your web application and your TestCafe test code?

Working workaround:

class 3rdPartySDKHook extends RequestHook {
    constructor() {
        const requestFilterRules = [/* regex filtering requests to only those iframe is performing */];
        super(requestFilterRules);
    }

    public async onRequest(event: any): Promise<void> {
        event.requestOptions.headers.referer = TEST_URL;
    }

    public async onResponse(): Promise<void> {
        // noop
    }
}

const hook = new 3rdPartySDKHook();

fixture`My fixture`
.page(TEST_URL).requestHooks(hook);

Steps to Reproduce:

1. Use iframe with src that uses ACL based on origin.
2. See if the origin header gets set properly.

Your Environment details:

• testcafe version: 1.9.4
• node.js version: lts/erbium
• command-line arguments: chrome --disable-web-security --allow-insecure-localhost --use-fake-device-for-media-stream --use-file-for-fake-audio-capture
• browser name and version: Chrome 90
• platform and version: macOS 11.2.3

[wentwrong]

I wasn't able to reproduce this issue. Could you please clarify how the iframe with src should look?

I used the following code:

require('http')
    .createServer((req, res) => {
        if (req.url === '/') {
            res.writeHead(200, { 'content-type': 'text/html' });
            res.end(`<iframe src="http://localhost:2022/" referrerpolicy="unsafe-url"></iframe>`);
        } else
            res.destroy();
    })
    .listen(2021, () => console.log('http://localhost:2021/'));

require('http')
    .createServer((req, res) => {
        if (req.url === '/') {
            res.writeHead(200, { 'content-type': 'text/html' });
            res.end(`Referer header: ${req.headers.referer}`);
        } else
            res.destroy();
    })
    .listen(2022, () => console.log('http://localhost:2022/'));

It creates two servers. The first one renders the iframe with the src attribute, which points to the url of the second one. Then I used the following test code:

import { RequestLogger } from 'testcafe';

const logger = RequestLogger('', {
    logRequestHeaders: true
});

fixture `My fixture`
    .requestHooks(logger)
    .page `http://localhost:2021/`

test(`Referer should be set to url of the page rendering iframe`, async t => {
    await t
        .expect(logger.requests[1].request.headers.referer).eql('http://localhost:2021/');
});

It passed the check that the Referer header was set to the url of the page that renders the iframe ('http://localhost:2021/') as expected.

Also, please update your TestCafe version to the latest version (v1.14.2 right now) to be sure that the issue can still be reproduced with it.

[KarolNov]

Hey, I'll try doing that outside of work, since we're fine with workaround and didn't intend on updating Testcafe version for now. It might be possibly fixed between current version and one I was using. I'll also try to provide a better example without actually revealing what iframe src was since I can't do that.

[wentwrong]

Thank you for the reply. I'll close this ticket then. Feel free to reopen it or open a new one once you can share an example.