Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
Ky-like body response transformations (#704)
Fixes #671
  • Loading branch information
szmarczak authored and sindresorhus committed Jan 17, 2019
1 parent 857ea62 commit a6a7d5a
Show file tree
Hide file tree
Showing 19 changed files with 272 additions and 199 deletions.
7 changes: 4 additions & 3 deletions advanced-creation.md
Expand Up @@ -54,7 +54,7 @@ const settings = {
return next(options);
},
options: got.mergeOptions(got.defaults.options, {
json: true
responseType: 'json'
})
};
Expand Down Expand Up @@ -110,9 +110,10 @@ const defaults = {
followRedirect: true,
stream: false,
form: false,
json: false,
cache: false,
useElectronNet: false
useElectronNet: false,
responseType: 'text',
resolveBodyOnly: 'false'
},
mutableDefaults: false
};
Expand Down
12 changes: 4 additions & 8 deletions migration-guides.md
Expand Up @@ -44,7 +44,6 @@ These Got options are the same as with Request:

- [`url`](https://github.com/sindresorhus/got#url) (+ we accept [`URL`](https://developer.mozilla.org/en-US/docs/Web/API/URL) instances too!)
- [`body`](https://github.com/sindresorhus/got#body)
- [`json`](https://github.com/sindresorhus/got#json)
- [`followRedirect`](https://github.com/sindresorhus/got#followRedirect)
- [`encoding`](https://github.com/sindresorhus/got#encoding)

Expand Down Expand Up @@ -77,6 +76,7 @@ To use streams, just call `got.stream(url, options)` or `got(url, {stream: true,

#### Breaking changes

- The `json` option is not a `boolean`, it's an `Object`. It will be stringified and used as a body.
- No `form` option. You have to pass a [`form-data` instance](https://github.com/form-data/form-data) through the [`body` option](https://github.com/sindresorhus/got#body).
- No `oauth`/`hawk`/`aws`/`httpSignature` option. To sign requests, you need to create a [custom instance](advanced-creation.md#signing-requests).
- No `agentClass`/`agentOptions`/`pool` option.
Expand Down Expand Up @@ -105,21 +105,17 @@ const gotInstance = got.extend({
hooks: {
init: [
options => {
// Save the original option, so we can look at it in the `afterResponse` hook
options.originalJson = options.json;
if (options.json && options.jsonReplacer) {
if (options.jsonReplacer) {
options.body = JSON.stringify(options.body, options.jsonReplacer);
options.json = false; // We've handled that on our own
}
}
],
afterResponse: [
response => {
const options = response.request.gotOptions;
if (options.originalJson && options.jsonReviver) {
if (options.jsonReviver && options.responseType === 'json') {
options.responseType = '';
response.body = JSON.parse(response.body, options.jsonReviver);
options.json = false; // We've handled that on our own
}
return response;
Expand Down
113 changes: 83 additions & 30 deletions readme.md
Expand Up @@ -35,7 +35,7 @@ Got is for Node.js. For browsers, we recommend [Ky](https://github.com/sindresor
- [Handles gzip/deflate](#decompress)
- [Timeout handling](#timeout)
- [Errors with metadata](#errors)
- [JSON mode](#json)
- [JSON mode](#json-mode)
- [WHATWG URL support](#url)
- [Hooks](#hooks)
- [Instances with custom defaults](#instances)
Expand Down Expand Up @@ -157,14 +157,44 @@ Returns a `Stream` instead of a `Promise`. This is equivalent to calling `got.st

Type: `string` `Buffer` `stream.Readable` [`form-data` instance](https://github.com/form-data/form-data)

**Note:** If you provide this option, `got.stream()` will be read-only.
**Note:** The `body` option cannot be used with the `json` or `form` option.

The body that will be sent with a `POST` request.
**Note:** If you provide this option, `got.stream()` will be read-only.

If present in `options` and `options.method` is not set, `options.method` will be set to `POST`.

The `content-length` header will be automatically set if `body` is a `string` / `Buffer` / `fs.createReadStream` instance / [`form-data` instance](https://github.com/form-data/form-data), and `content-length` and `transfer-encoding` are not manually set in `options.headers`.

###### json

Type: `Object` `Array` `number` `string` `boolean` `null`

**Note:** If you provide this option, `got.stream()` will be read-only.

JSON body. The `Content-Type` header will be set to `application/json` if it's not defined.

###### responseType

Type: `string`<br>
Default: `text`

**Note:** When using streams, this option is ignored.

Parsing method used to retrieve the body from the response. Can be `text`, `json` or `buffer`. The promise has `.json()` and `.buffer()` and `.text()` functions which set this option automatically.

Example:

```js
const {body} = await got(url).json();
```

###### resolveBodyOnly

Type: `string`<br>
Default: `false`

When set to `true` the promise will return the [Response body](#body-1) instead of the [Response](#response) object.

###### cookieJar

Type: [`tough.CookieJar` instance](https://github.com/salesforce/tough-cookie#cookiejar)
Expand All @@ -182,25 +212,13 @@ Default: `'utf8'`

###### form

Type: `boolean`<br>
Default: `false`
Type: `Object`

**Note:** If you provide this option, `got.stream()` will be read-only.
**Note:** `body` must be a plain object. It will be converted to a query string using [`(new URLSearchParams(object)).toString()`](https://nodejs.org/api/url.html#url_constructor_new_urlsearchparams_obj).

If set to `true` and `Content-Type` header is not set, it will be set to `application/x-www-form-urlencoded`.
The form body is converted to query string using [`(new URLSearchParams(object)).toString()`](https://nodejs.org/api/url.html#url_constructor_new_urlsearchparams_obj).

###### json

Type: `boolean`<br>
Default: `false`

**Note:** If you use `got.stream()`, this option will be ignored.
**Note:** `body` must be a plain object or array and will be stringified.

If set to `true` and `Content-Type` header is not set, it will be set to `application/json`.

Parse response body with `JSON.parse` and set `accept` header to `application/json`. If used in conjunction with the `form` option, the `body` will the stringified as querystring and the response parsed as JSON.
If set to `true` and `Content-Type` header is not set, it will be set to `application/x-www-form-urlencoded`.

###### searchParams

Expand Down Expand Up @@ -364,19 +382,17 @@ Called with plain [request options](#options), right before their normalization.

See the [Request migration guide](migration-guides.md#breaking-changes) for an example.

**Note**: This hook must be synchronous!
**Note:** This hook must be synchronous!

###### hooks.beforeRequest

Type: `Function[]`<br>
Default: `[]`

Called with [normalized](source/normalize-arguments.js) [request options](#options). Got will make no further changes to the request before it is sent. This is especially useful in conjunction with [`got.extend()`](#instances) and [`got.create()`](advanced-creation.md) when you want to create an API client that, for example, uses HMAC-signing.
Called with [normalized](source/normalize-arguments.js) [request options](#options). Got will make no further changes to the request before it is sent (except the body serialization). This is especially useful in conjunction with [`got.extend()`](#instances) and [`got.create()`](advanced-creation.md) when you want to create an API client that, for example, uses HMAC-signing.

See the [AWS section](#aws) for an example.

**Note:** If you modify the `body` you will need to modify the `content-length` header too, because it has already been computed and assigned.

###### hooks.beforeRedirect

Type: `Function[]`<br>
Expand Down Expand Up @@ -469,7 +485,7 @@ Default: `[]`

Called with an `Error` instance. The error is passed to the hook right before it's thrown. This is especially useful when you want to have more detailed errors.

**Note**: Errors thrown while normalizing input options are thrown directly and not part of this hook.
**Note:** Errors thrown while normalizing input options are thrown directly and not part of this hook.

```js
const got = require('got');
Expand Down Expand Up @@ -505,7 +521,7 @@ Type: `Object`

##### body

Type: `string` `Object` *(depending on `options.json`)*
Type: `string` `Object` `Buffer` *(depending on `options.responseType`)*

The result of the request.

Expand Down Expand Up @@ -666,11 +682,11 @@ client.get('/demo');
'x-foo': 'bar'
}
});
const {headers} = (await client.get('/headers', {json: true})).body;
const {headers} = (await client.get('/headers').json()).body;
//=> headers['x-foo'] === 'bar'
const jsonClient = client.extend({
json: true,
responseType: 'json',
headers: {
'x-baz': 'qux'
}
Expand Down Expand Up @@ -731,7 +747,7 @@ When reading from response stream fails.

#### got.ParseError

When `json` option is enabled, server response code is 2xx, and `JSON.parse` fails. Includes `statusCode` and `statusMessage` properties.
When server response code is 2xx, and parsing body fails. Includes `body`, `statusCode` and `statusMessage` properties.

#### got.HTTPError

Expand Down Expand Up @@ -917,7 +933,7 @@ const url = 'https://api.twitter.com/1.1/statuses/home_timeline.json';
got(url, {
headers: oauth.toHeader(oauth.authorize({url, method: 'GET'}, token)),
json: true
responseType: 'json'
});
```

Expand Down Expand Up @@ -1008,6 +1024,43 @@ const createTestServer = require('create-test-server');

## Tips

### JSON mode

By default, if you pass an object to the `body` option it will be stringified using `JSON.stringify`. Example:

```js
const got = require('got');
(async () => {
const response = await got('httpbin.org/anything', {
body: {
hello: 'world'
},
responseType: 'json'
});
console.log(response.body.data);
//=> '{"hello":"world"}'
})();
```

To receive a JSON body you can either set `responseType` option to `json` or use `promise.json()`. Example:

```js
const got = require('got');
(async () => {
const {body} = await got('httpbin.org/anything', {
body: {
hello: 'world'
}
}).json();
console.log(body);
//=> {...}
})();
```

### User Agent

It's a good idea to set the `'user-agent'` header so the provider can more easily see how their resource is used. By default, it's the URL to this repo. You can omit this header by setting it to `null`.
Expand Down Expand Up @@ -1045,7 +1098,7 @@ const pkg = require('./package.json');
const custom = got.extend({
baseUrl: 'example.com',
json: true,
responseType: 'json',
headers: {
'user-agent': `my-package/${pkg.version} (https://github.com/username/my-package)`
}
Expand Down Expand Up @@ -1094,7 +1147,7 @@ const h2got = got.extend({request});
| Advanced timeouts | 鉁 | 鉁 | 鉁 | 鉁 | 鉁 |
| Timings | 鉁 | 鉁 | 鉁 | 鉁 | 鉁 |
| Errors with metadata | 鉁 | 鉁 | 鉁 | 鉁 | 鉁 |
| JSON mode | 鉁 | 鉁 | | 鉁 | 鉁 |
| JSON mode | 鉁 | 鉁 | | 鉁 | 鉁 |
| Custom defaults | 鉁 | 鉁 | 鉁 | 鉁 | 鉁 |
| Composable | 鉁 | 鉁 | 鉁 | 鉁 | 鉁 |
| Hooks | 鉁 | 鉁 | 鉁 | 鉁 | 鉁 |
Expand Down
36 changes: 32 additions & 4 deletions source/as-promise.js
Expand Up @@ -11,6 +11,16 @@ const {reNormalize} = require('./normalize-arguments');
const asPromise = options => {
const proxy = new EventEmitter();

const parseBody = response => {
if (options.responseType === 'json') {
response.body = JSON.parse(response.body);
} else if (options.responseType === 'buffer') {
response.body = Buffer.from(response.body);
} else if (options.responseType !== 'text' && !is.falsy(options.responseType)) {
throw new Error(`Failed to parse body of type '${options.responseType}'`);
}
};

const promise = new PCancelable((resolve, reject, onCancel) => {
const emitter = requestAsEventEmitter(options);

Expand Down Expand Up @@ -57,9 +67,9 @@ const asPromise = options => {

const {statusCode} = response;

if (options.json && response.body) {
if (response.body) {
try {
response.body = JSON.parse(response.body);
parseBody(response);
} catch (error) {
if (statusCode >= 200 && statusCode < 300) {
const parseError = new ParseError(error, statusCode, options, data);
Expand All @@ -79,13 +89,13 @@ const asPromise = options => {
return;
}

resolve(response);
resolve(options.resolveBodyOnly ? response.body : response);
}

return;
}

resolve(response);
resolve(options.resolveBodyOnly ? response.body : response);
});

emitter.once('error', reject);
Expand All @@ -102,6 +112,24 @@ const asPromise = options => {
return promise;
};

promise.json = () => {
options.responseType = 'json';
options.resolveBodyOnly = true;
return promise;
};

promise.buffer = () => {
options.responseType = 'buffer';
options.resolveBodyOnly = true;
return promise;
};

promise.text = () => {
options.responseType = 'text';
options.resolveBodyOnly = true;
return promise;
};

return promise;
};

Expand Down
3 changes: 2 additions & 1 deletion source/errors.js
Expand Up @@ -52,8 +52,9 @@ module.exports.ReadError = class extends GotError {

module.exports.ParseError = class extends GotError {
constructor(error, statusCode, options, data) {
super(`${error.message} in "${urlLib.format(options)}": \n${data.slice(0, 77)}...`, error, options);
super(`${error.message} in "${urlLib.format(options)}"`, error, options);
this.name = 'ParseError';
this.body = data;
this.statusCode = statusCode;
this.statusMessage = http.STATUS_CODES[this.statusCode];
}
Expand Down
6 changes: 3 additions & 3 deletions source/index.js
Expand Up @@ -47,10 +47,10 @@ const defaults = {
throwHttpErrors: true,
followRedirect: true,
stream: false,
form: false,
json: false,
cache: false,
useElectronNet: false
useElectronNet: false,
responseType: 'text',
resolveBodyOnly: false
},
mutableDefaults: false
};
Expand Down

0 comments on commit a6a7d5a

Please sign in to comment.