Skip to content

Commit

Permalink
linkPreviewFetch: Increase maximum HTML we'll process to 1000*1024 bytes
Browse files Browse the repository at this point in the history
Co-authored-by: Scott Nonnenberg <scott@signal.org>
  • Loading branch information
automated-signal and scottnonnenberg-signal committed Oct 14, 2021
1 parent b08d163 commit 1b7ceff
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 3 deletions.
4 changes: 2 additions & 2 deletions ts/linkPreviews/linkPreviewFetch.ts
Expand Up @@ -27,9 +27,9 @@ const MAX_CONTENT_TYPE_LENGTH_TO_PARSE = 100;

// Though we'll accept HTML of any Content-Length (including no specified length), we
// will only load some of the HTML. So we might start loading a 99 gigabyte HTML page
// but only parse the first 500 kilobytes. However, if the Content-Length is less than
// but only parse the first 1000 kilobytes. However, if the Content-Length is less than
// this, we won't waste space.
const MAX_HTML_BYTES_TO_LOAD = 500 * 1024;
const MAX_HTML_BYTES_TO_LOAD = 1000 * 1024;

// `<title>x` is 8 bytes. Nothing else (meta tags, etc) will even fit, so we can ignore
// it. This is mostly to protect us against empty response bodies.
Expand Down
5 changes: 4 additions & 1 deletion ts/test-electron/linkPreviews/linkPreviewFetch_test.ts
Expand Up @@ -843,7 +843,7 @@ describe('link preview fetching', () => {
sinon.assert.notCalled(shouldNeverBeCalled);
});

it('stops reading bodies after 500 kilobytes', async function test() {
it('stops reading bodies after 1000 kilobytes', async function test() {
const shouldNeverBeCalled = sinon.stub();

const fakeFetch = stub().resolves(
Expand All @@ -855,6 +855,9 @@ describe('link preview fetching', () => {
const spaces = new Uint8Array(250 * 1024).fill(32);
yield spaces;
yield spaces;
yield spaces;
yield spaces;
yield spaces;
shouldNeverBeCalled();
yield new TextEncoder().encode(
'<meta property="og:description" content="should be ignored">'
Expand Down

0 comments on commit 1b7ceff

Please sign in to comment.