ScienceDirect snapshots are missing styles

[adomasven]

Hi Gildas,

We are getting reports from our users that saving ScienceDirect pages (e.g. https://www.sciencedirect.com/science/article/abs/pii/S030193222100152X) produces a broken snapshot. I can reproduce it with the default SingleFile extension. It is reproducible on Firefox and Chrome, with Zotero Connector disabled. Zotero uses the option to not capture frames on the page, but that doesn't seem to have an impact when toggled in SingleFile options.

[gildas-lormeau]

Thank you. I was able to reproduce and fix the issue in SingleFile. The fix will be available in the next version.

You will have to adapt the code on your end. The fetch function passed as parameter here https://github.com/zotero/zotero-connectors/blob/d3b870f525137c79a567398b668b87ee19d60fb7/src/common/singlefile.js#L60 should try to fetch the resource via a script injected into the page (i.e. hostFetch in SingleFile, window.wrappedJSObject.fetch might work in Firefox). This script has to fetch the resource from the page itself and outside the extension world. This allows keeping the referrer and origin HTTP header values unchanged (this is the cause of the issue).

[gildas-lormeau]

Actually simply passing the option referrerPolicy: "strict-origin-when-cross-origin" to fetch might be sufficient. FYI, the latest version of single-file-core (v1.5.2) adds this option automatically when calling fetch.

[adomasven]

I've updated SF in the Connector to the latest version which uses referrerPolicy: "strict-origin-when-cross-origin", but it did not help neither in Firefox, nor in Chrome. Are you sure it works without sending the correct referrer?

[gildas-lormeau]

Yes, I'm sure it works but for that I have to call fetch outside the content script, in the page itself. You should just have to adapt the code below.

SingleFile/src/lib/single-file/fetch/content/content-fetch.js

Lines 96 to 138 in dc9d100

	async function hostFetch(url, options) {
	if (hostFetchSupported === undefined) {
	hostFetchSupported = false;
	document.addEventListener(FETCH_SUPPORTED_RESPONSE_EVENT, () => hostFetchSupported = true, false);
	document.dispatchEvent(new CustomEvent(FETCH_SUPPORTED_REQUEST_EVENT));
	}
	if (hostFetchSupported) {
	const result = new Promise((resolve, reject) => {
	document.dispatchEvent(new CustomEvent(FETCH_REQUEST_EVENT, { detail: JSON.stringify({ url, options }) }));
	document.addEventListener(FETCH_RESPONSE_EVENT, onResponseFetch, false);
	function onResponseFetch(event) {
	if (event.detail) {
	if (event.detail.url == url) {
	document.removeEventListener(FETCH_RESPONSE_EVENT, onResponseFetch, false);
	if (event.detail.response) {
	resolve({
	status: event.detail.status,
	headers: new Map(event.detail.headers),
	arrayBuffer: async () => event.detail.response
	});
	} else {
	reject(event.detail.error);
	}
	}
	} else {
	reject();
	}
	}
	});
	try {
	return await result;
	} catch (error) {
	if (error && error.message == ERR_HOST_FETCH) {
	return fetch(url, options);
	} else {
	throw error;
	}
	}
	} else {
	return fetch(url, options);
	}
	}

This code interacts with the following code in single-file-core.

https://github.com/gildas-lormeau/single-file-core/blob/3921d35301b2904025abbc8705f87e1279d4f52c/processors/hooks/content/content-hooks-frames-web.js#L136-L148.