Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Request bodies of URLSearchParams can not be read #1365

Closed
4 tasks done
markdon opened this issue Aug 17, 2022 · 16 comments · Fixed by #1436
Closed
4 tasks done

Request bodies of URLSearchParams can not be read #1365

markdon opened this issue Aug 17, 2022 · 16 comments · Fixed by #1436
Labels
bug Something isn't working needs:triage Issues that have not been investigated yet. scope:node Related to MSW running in Node

Comments

@markdon
Copy link

markdon commented Aug 17, 2022

Prerequisites

Environment check

  • I'm using the latest msw version
  • I'm using Node.js version 14 or higher

Node.js version

v16.13.0

Reproduction repository

https://github.com/markdon/cra-msw

Reproduction steps

clone repo
npm i
DEBUG=* npm run test -- --watchAll=false

Current behavior

I think this is related to #1318

When handling a fetch request where the request body is a URLSearchParameters object, the body can not be read in the handler. This was previously available through request.body (v0.27.1). request.body is now deprecated.

I would expect to be able to just use the newer request.text(), however this seems to throw an error parsing the body.

Debug logs. See ERR_INVALID_ARG_TYPE
➜  cra-msw git:(main) DEBUG=* npm run test -- --watchAll=false

cra-msw@0.1.0 test
react-scripts test "--watchAll=false"

babel:config:loading:files:plugins Loaded preset '/Users/mark/projects/cra-msw/node_modules/babel-preset-react-app/index.js' from '/Users/mark/projects/cra-msw'. +0ms
babel:config:loading:files:plugins Loaded preset '/Users/mark/projects/cra-msw/node_modules/babel-preset-jest/index.js' from '/Users/mark/projects/cra-msw'. +2ms
http constructing the interceptor... +0ms
xhr constructing the interceptor... +0ms
setup-server constructing the interceptor... +0ms
http:on adding "request" event listener: setupServerListener +0ms
async-event-emitter:on adding "request" listener... +0ms
xhr:on adding "request" event listener: setupServerListener +0ms
async-event-emitter:on adding "request" listener... +0ms
http:on adding "response" event listener: +0ms
async-event-emitter:on adding "response" listener... +0ms
xhr:on adding "response" event listener: +0ms
async-event-emitter:on adding "response" listener... +0ms
setup-server:apply applying the interceptor... +0ms
async-event-emitter:activate set state to: ACTIVE +0ms
setup-server:apply activated the emiter! ACTIVE +0ms
setup-server retrieved global instance: undefined +4ms
setup-server:apply no running instance found, setting up a new instance... +1ms
setup-server:setup applying all 2 interceptors... +0ms
setup-server:setup applying "ClientRequestInterceptor" interceptor... +0ms
http:apply applying the interceptor... +0ms
async-event-emitter:activate set state to: ACTIVE +0ms
http:apply activated the emiter! ACTIVE +0ms
http retrieved global instance: undefined +4ms
http:apply no running instance found, setting up a new instance... +0ms
http:setup native "http" module patched! +0ms
http:setup native "https" module patched! +0ms
http set global instance! http +0ms
setup-server:setup adding interceptor dispose subscription +1ms
setup-server:setup applying "XMLHttpRequestInterceptor" interceptor... +0ms
xhr:apply applying the interceptor... +0ms
async-event-emitter:activate set state to: ACTIVE +0ms
xhr:apply activated the emiter! ACTIVE +0ms
xhr retrieved global instance: undefined +5ms
xhr:apply no running instance found, setting up a new instance... +0ms
xhr:setup patching "XMLHttpRequest" module... +0ms
xhr:setup native "XMLHttpRequest" module patched! XMLHttpRequestOverride +0ms
xhr set global instance! xhr +0ms
setup-server:setup adding interceptor dispose subscription +0ms
setup-server set global instance! setup-server +1ms
xhr:request POST /login open {
method: 'POST',
url: '/login',
async: true,
user: undefined,
password: undefined
} +0ms
xhr:request POST /login reset +1ms
xhr:request POST /login readyState change 0 -> 1 +1ms
xhr:request POST /login triggering readystate change... +0ms
xhr:request POST /login trigger "readystatechange" (1) +0ms
xhr:request POST /login resolve listener for event "readystatechange" +0ms
xhr:request POST /login set request header "content-type" to "application/x-www-form-urlencoded;charset=UTF-8" +0ms
xhr:request POST /login send POST /login +0ms
xhr:request POST /login request headers HeadersPolyfill {
[Symbol(normalizedHeaders)]: { 'content-type': 'application/x-www-form-urlencoded;charset=UTF-8' },
[Symbol(rawHeaderNames)]: Map(1) { 'content-type' => 'content-type' }
} +1ms
xhr:request POST /login emitting the "request" event for 1 listener(s)... +0ms
async-event-emitter:emit emitting "request" event... +0ms
async-event-emitter:openListenerQueue opening "request" listeners queue... +0ms
async-event-emitter:openListenerQueue no queue found, creating one... +0ms
async-event-emitter:emit appending a one-time cleanup "request" listener... +0ms
async-event-emitter:openListenerQueue opening "request" listeners queue... +0ms
async-event-emitter:openListenerQueue returning an exising queue: [] +0ms
async-event-emitter:on awaiting the "request" listener... +11ms
xhr:request POST /login awaiting mocked response... +2ms
async-event-emitter:on "TypeError [ERR_INVALID_ARG_TYPE]: The "input" argument must be an instance of ArrayBuffer or ArrayBufferView. Received an instance of URLSearchParams" listener has rejected! +13ms
xhr:request POST /login middleware function threw an exception! TypeError: The "input" argument must be an instance of ArrayBuffer or ArrayBufferView. Received an instance of URLSearchParams
at new NodeError (node:internal/errors:371:5)
at TextDecoder.decode (node:internal/encoding:413:15)
at Object.decodeBuffer (/Users/mark/projects/cra-msw/node_modules/@mswjs/interceptors/src/utils/bufferUtils.ts:11:18)
at RestRequest. (/Users/mark/projects/cra-msw/node_modules/@mswjs/interceptors/src/IsomorphicRequest.ts:59:12)
at step (/Users/mark/projects/cra-msw/node_modules/@mswjs/interceptors/lib/IsomorphicRequest.js:33:23)
at Object.next (/Users/mark/projects/cra-msw/node_modules/@mswjs/interceptors/lib/IsomorphicRequest.js:14:53)
at /Users/mark/projects/cra-msw/node_modules/@mswjs/interceptors/lib/IsomorphicRequest.js:8:71
at new Promise ()
at Object..__awaiter (/Users/mark/projects/cra-msw/node_modules/@mswjs/interceptors/lib/IsomorphicRequest.js:4:12)
at RestRequest.Object..IsomorphicRequest.text (/Users/mark/projects/cra-msw/node_modules/@mswjs/interceptors/lib/IsomorphicRequest.js:73:16) {
code: 'ERR_INVALID_ARG_TYPE'
} +12ms
xhr:request POST /login trigger "error" (1) +8ms
xhr:request POST /login resolve listener for event "error" +1ms
xhr:request POST /login abort +0ms
xhr:request POST /login readyState change 1 -> 0 +0ms
xhr:request POST /login trigger "abort" (0) +0ms
xhr:request POST /login resolve listener for event "abort" +0ms
async-event-emitter:emit cleaned up "request" listeners queue! +27ms
setup-server:dispose disposing the interceptor... +0ms
setup-server retrieved global instance: BatchInterceptor +49ms
setup-server cleared global instance! setup-server +0ms
setup-server:dispose global symbol deleted: undefined +0ms
setup-server:dispose disposing of 2 subscriptions... +0ms
http:dispose disposing the interceptor... +0ms
http retrieved global instance: ClientRequestInterceptor +51ms
http cleared global instance! http +0ms
http:dispose global symbol deleted: undefined +0ms
http:dispose disposing of 2 subscriptions... +0ms
http:setup native "http" module restored! +51ms
http:setup native "https" module restored! +0ms
http:dispose disposed of all subscriptions! 0 +0ms
async-event-emitter:deactivate removing all listeners... +0ms
async-event-emitter:removeAllListeners event: undefined +0ms
async-event-emitter:removeAllListeners cleared the listeners queue! Map(0) {} +0ms
async-event-emitter:deactivate set state to: DEACTIVATED +0ms
http:dispose destroyed the listener! +0ms
xhr:dispose disposing the interceptor... +0ms
xhr retrieved global instance: XMLHttpRequestInterceptor +50ms
xhr cleared global instance! xhr +0ms
xhr:dispose global symbol deleted: undefined +0ms
xhr:dispose disposing of 1 subscriptions... +0ms
xhr:setup native "XMLHttpRequest" module restored! XMLHttpRequest +51ms
xhr:dispose disposed of all subscriptions! 0 +1ms
async-event-emitter:deactivate removing all listeners... +0ms
async-event-emitter:removeAllListeners event: undefined +0ms
async-event-emitter:removeAllListeners cleared the listeners queue! Map(0) {} +0ms
async-event-emitter:deactivate set state to: DEACTIVATED +0ms
xhr:dispose destroyed the listener! +0ms
setup-server:dispose disposed of all subscriptions! 0 +2ms
async-event-emitter:deactivate removing all listeners... +0ms
async-event-emitter:removeAllListeners event: undefined +0ms
async-event-emitter:removeAllListeners cleared the listeners queue! Map(0) {} +0ms
async-event-emitter:deactivate set state to: DEACTIVATED +0ms
setup-server:dispose destroyed the listener! +0ms
FAIL src/Login.test.ts
fetch
✕ can login (35 ms)

● fetch › can login

expect(received).resolves.toBeInstanceOf()

Received promise rejected instead of resolved
Rejected to value: [TypeError: Network request failed]

  2 | describe('fetch', ()=>{
  3 |   it('can login', async ()=>{
> 4 |     await expect(fetch('/login', {
    |           ^
  5 |       method: 'POST',
  6 |       body: new URLSearchParams('username=admin&password=admin')
  7 |     })).resolves.toBeInstanceOf(Response);

  at expect (node_modules/expect/build/index.js:178:15)
  at Object.<anonymous> (src/Login.test.ts:4:11)
  at TestScheduler.scheduleTests (node_modules/@jest/core/build/TestScheduler.js:333:13)
  at runJest (node_modules/@jest/core/build/runJest.js:404:19)
  at _run10000 (node_modules/@jest/core/build/cli/index.js:320:7)
  at runCLI (node_modules/@jest/core/build/cli/index.js:173:3)

Test Suites: 1 failed, 1 total
Tests: 1 failed, 1 total
Snapshots: 0 total
Time: 0.834 s, estimated 1 s
Ran all test suites.

Expected behavior

A request body of type URLSearchParameters should be able to be read in a handler. I suggest via request.text(), which would return the value of URLSearchParameters.toString().

@markdon markdon added bug Something isn't working needs:triage Issues that have not been investigated yet. scope:node Related to MSW running in Node labels Aug 17, 2022
@kettanaito
Copy link
Member

Hey, @markdon. Thanks for reporting this.

We are working on improving how request bodies are read, and this use case is going to the list of things we need to account for. Meanwhile, you can still use req.body if that used to work previously. The logic there should remain the same.

@markdon
Copy link
Author

markdon commented Aug 17, 2022

Actually, I only spent the time investigating this after req.body.toString() started throwing a very similar error :( Just accessing req.body at all throws the same error.

I can refactor the few of my tests this affected to move to latest msw.

Debug log snippet
xhr:request POST /login middleware function threw an exception! TypeError: The "input" argument must be an instance of ArrayBuffer or ArrayBufferView. Received an instance of URLSearchParams
    at new NodeError (node:internal/errors:371:5)
    at TextDecoder.decode (node:internal/encoding:413:15)
    at decodeBuffer (/Users/mark/projects/cra-msw/node_modules/@mswjs/interceptors/src/utils/bufferUtils.ts:11:18)
    at RestRequest.body (/Users/mark/projects/cra-msw/node_modules/msw/src/utils/request/MockedRequest.ts:111:18)
    at /Users/mark/projects/cra-msw/src/setupTests.ts:8:44
    at /Users/mark/projects/cra-msw/node_modules/msw/src/handlers/RequestHandler.ts:232:55
    at RestHandler.run (/Users/mark/projects/cra-msw/node_modules/msw/src/handlers/RequestHandler.ts:215:34)
    at /Users/mark/projects/cra-msw/node_modules/msw/src/utils/getResponse.ts:50:34
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at getResponse (/Users/mark/projects/cra-msw/node_modules/msw/src/utils/getResponse.ts:41:18) {
  code: 'ERR_INVALID_ARG_TYPE'
} +12ms

@kettanaito
Copy link
Member

Oh, so the reason is not to read the body but to construct it. He'd have to support that.

We're using something called IsomorphicRequest abstract that allows us to handle different request body types in the same manner. That abstraction only accepts ArrayBuffer as request body (which we want to keep) but the consumer (MSW in this case) can transform the body however they need to eventually provide the buffer.

So, I'd imagine this support to be:

  1. Check when the body is URLSearchParams.
  2. Serialize that somehow.
  3. Turn it into an ArrayBuffer.
  4. Make sure it's de-serializable as well.

Contributions are welcome!

@markdon
Copy link
Author

markdon commented Aug 21, 2022

Sounds pretty straight forward to implement! I would love to contribute, if I hadn't just signed up to a project with a tight timeline for the next few months.

I will check back later!

@ianldgs
Copy link

ianldgs commented Sep 2, 2022

My workaround, which was actually for FormData, is the following:

jest.spyOn(global, "FormData").mockImplementation(mockFormDataForMSW);
jest.mock("util", mockNodeUtilForMSW);

function mockFormDataForMSW() {
  return new FormDataFakeArrayBuffer(0) as unknown as FormData;
}

function mockNodeUtilForMSW() {
  const real = jest.requireActual("util") as typeof import("util");
  return {
    ...real,
    TextDecoder: class extends real.TextDecoder {
      decode(
        input?: NodeJS.ArrayBufferView | ArrayBuffer | null,
        options?: { stream?: boolean | undefined },
      ): string {
        if (input instanceof FormDataFakeArrayBuffer) return input.toString();

        return super.decode(input, options);
      }
    },
  };
}

class FormDataFakeArrayBuffer extends ArrayBuffer {
  private params = new URLSearchParams();

  append(param: string, value: string) {
    return this.params.append(param, value);
  }

  toString() {
    return this.params.toString();
  }
}

For reference/SEO, the error was:

The input argument must be an instance of ArrayBuffer or ArrayBufferView. Received an instance of FormData

or

The input argument must be an instance of ArrayBuffer or ArrayBufferView. Received an instance of URLSearchParams

@markdon
Copy link
Author

markdon commented Sep 4, 2022

I'm not sure why I didn't consider this before but surely the easiest workaround (for URLSearchParams) is to create the request with a string instead of URLSearchParams object. e.g. use URLSearchParams.toString() for the body when calling fetch.

The MSW handler will always need to parse the text body with new URLSearchParams(await body.text()) as there is no Request method that gets the body as a URLSearchParams object.

@ddolcimascolo
Copy link

Same issue as mine in #1327 but with FormData bodies. msw just plain broke reading request bodies in v0.44.0.

New features keep getting added in the 0.x line so that it's totally OK to break stuff from a semver perspective. And bugs are not fixed.

#sadness

@chawax
Copy link

chawax commented Oct 7, 2022

Same problem here.

@chawax
Copy link

chawax commented Oct 7, 2022

I used Axios instead of Fetch and it works !

@kettanaito
Copy link
Member

kettanaito commented Nov 18, 2022

@ddolcimascolo, a true sadness is having a 7M/week downloaded library used by FAANG and having to develop it over weekends instead of spending time with one's family (which I've promised myself not to do anymore, thus you will have issues that I don't have time/desire to work on). But hey, you can take a step towards resolving that sadness, here you go.

This is going to be fixed by #1436 so I'm not going to do any work around this until that API lands. If you're blocked by this, consider contributing, I'd do my best to review your code and release it.

@ddolcimascolo
Copy link

Yeah. My message above was itself sad, sorry about that, probably written while closing yet another upgrade merge request on MSW that I can't merge because of this blocker. You do a great job in maintaining OSS projects, keep up the great work!

Since my message above I've contributed a fix, even though you implemented #1436 in the end which is a really nice addition.

Cheers,
David

@markdon
Copy link
Author

markdon commented Nov 28, 2022

I will check back later!

It looks like a lot of progress has been made towards the Fetch API update that replaces the related code and fixes this issue. There's a beta available to try.

I had a quick look at making a fix for the current release of msw but @mswjs/interceptors has already moved on from the related IsomorphicRequest class and the Fetch API update looks pretty close to done anyway.

@DusanPausly
Copy link

DusanPausly commented Jan 25, 2023

Hey, just little question from me. I am using latest msw, node and react-query@^3.39.2.
And I am trying to connect with a Keycloak (newest stable version) for fresh token before running tests. I have to use URLSearchParams formatting in body to construct fetch and I am getting this mentioned error from msw:

"The "input" argument must be an instance of ArrayBuffer or ArrayBufferView. Received an instance of URLSearchParams."

I guess, #1436 is fixing it also for this case, right? Otherwise what is the status of completition @kettanaito ?

@erperejildo
Copy link

https://stackoverflow.com/questions/75535744/cant-get-body-using-msw-and-redux-toolkit/75543115#75543115

@kettanaito
Copy link
Member

@DusanPausly, correct. The current version of MSW doesn't support URLSearchParams as the request body. The Fetch API feature will add that support.

@kettanaito
Copy link
Member

Released: v2.0.0 🎉

This has been released in v2.0.0!

Make sure to always update to the latest version (npm i msw@latest) to get the newest features and bug fixes.


Predictable release automation by @ossjs/release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working needs:triage Issues that have not been investigated yet. scope:node Related to MSW running in Node
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants