Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add `onDownloadProgress` option #34

Merged
merged 43 commits into from May 10, 2019
Merged
Changes from 39 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
dc8990d
Streams
szmarczak Feb 22, 2019
b0f698c
onProgress & stream
szmarczak Feb 22, 2019
0e6fe67
Update browser.js
szmarczak Feb 22, 2019
4ad995f
Update browser.js
szmarczak Feb 22, 2019
73c82cc
Update index.js
szmarczak Feb 22, 2019
e6340e6
Tests
szmarczak Feb 22, 2019
2094a20
Update index.js
szmarczak Apr 9, 2019
2b025da
Update readme.md
szmarczak Apr 9, 2019
b70363e
Update readme.md
szmarczak Apr 9, 2019
469ebce
Update index.js
szmarczak Apr 9, 2019
ed371d6
Update browser.js
szmarczak Apr 9, 2019
7a0008d
Update browser.js
szmarczak Apr 9, 2019
14bc3e8
Update index.js
szmarczak Apr 9, 2019
69d36af
Fix cov
szmarczak Apr 9, 2019
eed391d
Mention `ky-universal` in the readme
sindresorhus Feb 22, 2019
9cd0f5b
Fix note about non-2xx status codes in readme (#104)
callumlocke Feb 26, 2019
4811366
Remove slash in prefixUrl example (#108)
tusbar Mar 11, 2019
6e4c645
Clarify usage of `.json()` (#111)
mesqueeb Mar 25, 2019
643b364
Fix Error types (#113)
stramel Apr 3, 2019
e91bb49
Fix TypeScript types to allow hooks to return a Promise (#123)
etienne-dldc Apr 9, 2019
037e3e6
Fix problem with debugging timed-out requests (#122)
arty-name Apr 9, 2019
64581b4
Improve the TypeScript definition
sindresorhus Apr 9, 2019
b48de7d
0.9.1
sindresorhus Apr 9, 2019
0f7e8a6
Don't export the `Ky` TypeScript namespace
sindresorhus Apr 9, 2019
2bae27f
Fix regression for environments without `AbortController` (#125)
gmaclennan Apr 19, 2019
7af3243
Set accept header for ky shortcut methods (#118)
selrond Apr 19, 2019
b686e3d
Make it possible to install Ky in Node.js 8
sindresorhus Apr 19, 2019
ee6fa03
0.10.0
sindresorhus Apr 19, 2019
6829d06
Fix Travis
sindresorhus Apr 19, 2019
accf022
Fixes
szmarczak Apr 21, 2019
824a76e
Merge branch 'master' into stream
szmarczak Apr 22, 2019
c5fcecc
Update index.js
szmarczak Apr 22, 2019
c1d40cc
Workaround for undefined
szmarczak Apr 22, 2019
fc36dcf
Update browser.js
szmarczak Apr 22, 2019
16ab480
tests
szmarczak Apr 22, 2019
0f3f37e
Merge branch 'master' into stream
szmarczak Apr 23, 2019
48d7443
Update index.d.ts
sindresorhus Apr 24, 2019
acfdc38
Merge branch 'master' into stream
szmarczak May 10, 2019
8923275
final touches
szmarczak May 10, 2019
48060e6
The final final (?) touches
szmarczak May 10, 2019
910e3cf
Update index.d.ts
sindresorhus May 10, 2019
c156a06
Update readme.md
sindresorhus May 10, 2019
badf3a1
Update index.d.ts
sindresorhus May 10, 2019
File filter...
Filter file types
Jump to…
Jump to file or symbol
Failed to load files and symbols.

Always

Just for now

@@ -6,6 +6,12 @@ export type BeforeRequestHook = (options: Options) => void | Promise<void>;

export type AfterResponseHook = (response: Response) => Response | void | Promise<Response | void>;

export interface DownloadProgress {
percent: number;
transferredBytes: number;
totalBytes: number;
}

export interface Hooks {
/**
Before the request is sent.
@@ -62,6 +68,13 @@ export interface Options extends RequestInit {
*/
timeout?: number | false;

/**
Download progress event handler.
If it's not possible to retrieve the body size, `totalBytes` will be `0`.
This conversation was marked as resolved by szmarczak

This comment has been minimized.

Copy link
@sindresorhus

sindresorhus May 10, 2019

Owner

This should be documented on the actual totalBytes property.

*/
onDownloadProgress?: (progress: DownloadProgress, chunk?: Uint8Array) => void;
This conversation was marked as resolved by szmarczak

This comment has been minimized.

Copy link
@sindresorhus

sindresorhus May 10, 2019

Owner

I think this should be placed last.


/**
Hooks allow modifications during the request lifecycle. Hook functions may be async and are run serially.
*/
@@ -24,12 +24,14 @@ const getGlobal = property => {
const document = getGlobal('document');
const Headers = getGlobal('Headers');
const Response = getGlobal('Response');
const ReadableStream = getGlobal('ReadableStream');
const fetch = getGlobal('fetch');
const AbortController = getGlobal('AbortController');
const FormData = getGlobal('FormData');

const isObject = value => value !== null && typeof value === 'object';
const supportsAbortController = typeof getGlobal('AbortController') === 'function';
const supportsAbortController = typeof AbortController === 'function';
const supportsStreams = typeof ReadableStream === 'function';
const supportsFormData = typeof FormData === 'function';

const deepMerge = (...sources) => {
@@ -234,6 +236,20 @@ class Ky {
throw new HTTPError(response);
}

// If `onDownloadProgress` is passed, use stream API internally
This conversation was marked as resolved by szmarczak

This comment has been minimized.

Copy link
@sindresorhus

sindresorhus May 10, 2019

Owner
Suggested change
// If `onDownloadProgress` is passed, use stream API internally
// If `onDownloadProgress` is passed it uses the API internally
/* istanbul ignore next */
if (this._options.onDownloadProgress) {
if (typeof this._options.onDownloadProgress !== 'function') {
throw new TypeError('The `onDownloadProgress` option must be a function');
}

if (!supportsStreams) {
throw new Error('Streams are not supported in your environment. `ReadableStream` is missing.');
}

return this._stream(response.clone(), this._options.onDownloadProgress);
}

return response;
};

@@ -311,6 +327,43 @@ class Ky {

return timeout(fetch(this._input, this._options), this._timeout, this.abortController);
}

/* istanbul ignore next */
_stream(response, onDownloadProgress) {
const totalBytes = Number(response.headers.get('content-length')) || 0;
let transferredBytes = 0;

return new Response(
new ReadableStream({
This conversation was marked as resolved by szmarczak

This comment has been minimized.

Copy link
@sindresorhus

sindresorhus Feb 22, 2019

Owner

These globals should use our globals thing, so it's possible to polyfill them.

start(controller) {
const reader = response.body.getReader();

if (onDownloadProgress) {
onDownloadProgress({percent: 0, transferredBytes: 0, totalBytes});
}

async function read() {
const {done, value} = await reader.read();
if (done) {
controller.close();
return;
}

if (onDownloadProgress) {
transferredBytes += value.byteLength;
const percent = totalBytes === 0 ? 0 : transferredBytes / totalBytes;
onDownloadProgress({percent, transferredBytes, totalBytes}, value);
}

controller.enqueue(value);
read();
}

read();
}
})
);
}
}

const validateAndMerge = (...sources) => {
@@ -1,5 +1,5 @@
import {expectType} from 'tsd';
import ky, {HTTPError, TimeoutError, ResponsePromise} from '.';
import ky, {HTTPError, TimeoutError, ResponsePromise, DownloadProgress} from '.';

const url = 'https://sindresorhus';

@@ -81,3 +81,11 @@ interface Result {
value: number;
}
expectType<Promise<Result>>(ky(url).json<Result>());

// `onDownloadProgress` option
ky(url, {
onDownloadProgress: (progress, chunk) => {
expectType<DownloadProgress>(progress);
expectType<Uint8Array | undefined>(chunk);
}
});
@@ -196,6 +196,27 @@ Default: `10000`
Timeout in milliseconds for getting a response. Can not be greater than 2147483647.
If set to `false`, there will be no timeout.

##### onDownloadProgress

Type: `Function`

Download progress event handler. The function takes `progress` and `chunk` arguments.<br>
This conversation was marked as resolved by szmarczak

This comment has been minimized.

Copy link
@sindresorhus

sindresorhus May 10, 2019

Owner
Suggested change
Download progress event handler. The function takes `progress` and `chunk` arguments.<br>
Download progress event handler. The function receives a `progress` and `chunk` argument.<br>
The `progress` object contains following elements: `percent`, `transferredBytes` and `totalBytes`. If it's not possible to retrieve the body size, total will be `0`.
This conversation was marked as resolved by szmarczak

This comment has been minimized.

Copy link
@sindresorhus

sindresorhus May 10, 2019

Owner
Suggested change
The `progress` object contains following elements: `percent`, `transferredBytes` and `totalBytes`. If it's not possible to retrieve the body size, total will be `0`.
The `progress` object contains the following elements: `percent`, `transferredBytes` and `totalBytes`. If it's not possible to retrieve the body size, `totalBytes` will be `0`.

For the first call, `chunk` will be always `undefined`.
This conversation was marked as resolved by szmarczak

This comment has been minimized.

Copy link
@sindresorhus

sindresorhus May 10, 2019

Owner

Instead of undefined, wouldn't it be better to just emit an empty chunk (empty Uint8Array)? Then people would not need to implement conditional handling and the types would be simpler.

This comment has been minimized.

Copy link
@szmarczak

szmarczak May 10, 2019

Collaborator

Will do. Great idea.



```js
This conversation was marked as resolved by szmarczak

This comment has been minimized.

Copy link
@sindresorhus

sindresorhus May 10, 2019

Owner
Suggested change
```js
```js
import ky from 'ky';
await ky('https://example.com', {
onProgress: (progress, chunk) => {
// Example output:
// `0% - 0 of 1271 bytes`
// `100% - 1271 of 1271 bytes`
console.log(`${progress.percent * 100}% - ${progress.transferredBytes} of ${progress.totalBytes} bytes`);
}
})
```

##### hooks

Type: `Object<string, Function[]>`<br>
@@ -17,6 +17,7 @@ test('prefixUrl option', withPage, async (t, page) => {
await t.throwsAsync(async () => {
return page.evaluate(() => {
window.ky = window.ky.default;

return window.ky('/foo', {prefixUrl: '/'});
});
}, /`input` must not begin with a slash when using `prefixUrl`/);
@@ -52,6 +53,7 @@ test('aborting a request', withPage, async (t, page) => {

const error = await page.evaluate(url => {
window.ky = window.ky.default;

const controller = new AbortController();
const request = window.ky(`${url}/test`, {signal: controller.signal}).text();
controller.abort();
@@ -86,3 +88,90 @@ test('throws TimeoutError even though it does not support AbortController', with

await server.close();
});

test('onDownloadProgress works', withPage, async (t, page) => {
const server = await createTestServer();

server.get('/', (request, response) => {
response.writeHead(200, {
'content-length': 4
});

response.write('me');
setTimeout(() => {
response.end('ow');
}, 1000);
});

await page.goto(server.url);
await page.addScriptTag({path: './umd.js'});

const result = await page.evaluate(async url => {
window.ky = window.ky.default;

// `new TextDecoder('utf-8').decode` hangs up?
const decodeUTF8 = array => String.fromCharCode(...array);

const data = [];
const text = await window.ky(url, {
onDownloadProgress: (progress, chunk) => {
const stringifiedChunk = chunk instanceof Uint8Array ? decodeUTF8(chunk) : typeof chunk;
data.push([progress, stringifiedChunk]);
}
}).text();

return {data, text};
}, server.url);

t.deepEqual(result.data, [
[{percent: 0, transferredBytes: 0, totalBytes: 4}, 'undefined'],
[{percent: 0.5, transferredBytes: 2, totalBytes: 4}, 'me'],
[{percent: 1, transferredBytes: 4, totalBytes: 4}, 'ow']
]);
t.is(result.text, 'meow');

await server.close();
});

test('throws if onDownloadProgress is not a function', withPage, async (t, page) => {
const server = await createTestServer();

server.get('/', (request, response) => {
response.end();
});

await page.goto(server.url);
await page.addScriptTag({path: './umd.js'});

const error = await page.evaluate(url => {
window.ky = window.ky.default;

const request = window.ky(url, {onDownloadProgress: 1}).text();
return request.catch(error => error.toString());
}, server.url);
t.is(error, 'TypeError: The `onDownloadProgress` option must be a function');

await server.close();
});

test('throws if does not support ReadableStream', withPage, async (t, page) => {
const server = await createTestServer();

server.get('/', (request, response) => {
response.end();
});

await page.goto(server.url);
await page.addScriptTag({path: './test/helpers/disable-stream-support.js'});
await page.addScriptTag({path: './umd.js'});

const error = await page.evaluate(url => {
window.ky = window.ky.default;

const request = window.ky(url, {onDownloadProgress: () => {}}).text();
return request.catch(error => error.toString());
}, server.url);
t.is(error, 'Error: Streams are not supported in your environment. `ReadableStream` is missing.');

await server.close();
});
@@ -0,0 +1 @@
window.ReadableStream = undefined;
ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.