Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

parser.readMore() when fetching external url throttled #2345

Closed
iiAku opened this issue Oct 13, 2023 · 0 comments · Fixed by #2346
Closed

parser.readMore() when fetching external url throttled #2345

iiAku opened this issue Oct 13, 2023 · 0 comments · Fixed by #2346
Labels
bug Something isn't working

Comments

@iiAku
Copy link
Contributor

iiAku commented Oct 13, 2023

Bug Description

Hello,

In our app, we fetch multiple time an external endpoint, to do that smoothly everything is throttled on our side. I've double check If not throttled and concurrency is high we can fetch let say 400 external call without any issue. However when delaying those same request and throtthling this error happens.

I was able to repro this from my codebase, but I am unable to repro that isolated.

Also I slightly modified the undici package line 1073 and just skip the statement if kParser is not defined and it just work. It seems undici is smart enough to reconnect the client if a disconnection occur as well further down the line. So it might be a fix.

if anyone is able to repro what I'm describing glad to have a repro link.

I'm also going to open a pr for that, let me know if it meet your requirements.

In our app we are doing something very similar to that:

const client = new Pool('BASE_URL');
const {statusCode} = await client.request({
    path: 'ROUTE_PATH',
    method: 'POST',
    body: {some: 'body'}
})

Expected Behavior

Undici should not throw an error if kParser is not defined when calling the readMore() function OR should prevent calling the parent function when kParser is presumably not defined

Logs & Screenshots

/usr/src/app/node_modules/undici/lib/client.js:1073
  parser.readMore()
         ^

TypeError: Cannot read properties of undefined (reading 'readMore')
    at TLSSocket.onSocketReadable (/usr/src/app/node_modules/undici/lib/client.js:1073:10)
    at TLSSocket.emit (node:events:514:28)
    at emitReadable_ (node:internal/streams/readable:590:12)
    at onEofChunk (node:internal/streams/readable:568:5)
    at readableAddChunk (node:internal/streams/readable:275:5)
    at Readable.push (node:internal/streams/readable:234:10)
    at TLSWrap.onStreamRead (node:internal/stream_base_commons:232:12)

Environment

node: v18.17.0
OS: mac OS 13.4.1 (c) (22F770820d), but also happen in deployed linux environment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant