Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implement sync formdata parser #2911

Merged
merged 11 commits into from Mar 5, 2024
Merged

Conversation

KhafraDev
Copy link
Member

@KhafraDev KhafraDev commented Mar 3, 2024

Fixes #2890

Based on https://andreubotella.github.io/multipart-form-data/

import { Request } from 'undici'

const request = new Request('http://localhost', {
  method: 'POST',
  headers: {
    'Content-Type': 'multipart/form-data; boundary=----formdata-undici-0.6204674738279623'
  },
  body:
    '------formdata-undici-0.6204674738279623\r\n' +
    'Content-Disposition: form-data; name="fiŝo"\r\n' +
    '\r\n' +
    'value1\r\n' +
    '------formdata-undici-0.6204674738279623--'
})

const fd = await request.formData()
// > FormData { [Symbol(state)]: [ { name: 'fiŝo', value: 'value1' } ] }

lib/web/fetch/formdata-parser.js Fixed Show fixed Hide fixed
lib/web/fetch/formdata-parser.js Fixed Show fixed Hide fixed
lib/web/fetch/formdata-parser.js Fixed Show fixed Hide fixed
@codecov-commenter
Copy link

codecov-commenter commented Mar 3, 2024

Codecov Report

Attention: Patch coverage is 83.51449% with 91 lines in your changes are missing coverage. Please review.

Project coverage is 93.62%. Comparing base (fc4e766) to head (2f69f5d).
Report is 10 commits behind head on main.

Files Patch % Lines
lib/web/fetch/formdata-parser.js 80.67% 91 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2911      +/-   ##
==========================================
- Coverage   93.86%   93.62%   -0.25%     
==========================================
  Files          85       86       +1     
  Lines       23418    23861     +443     
==========================================
+ Hits        21981    22339     +358     
- Misses       1437     1522      +85     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@KhafraDev
Copy link
Member Author

Alternatively we could just take Deno's parser.

lib/web/fetch/formdata-parser.js Fixed Show fixed Hide fixed
lib/web/fetch/formdata-parser.js Fixed Show fixed Hide fixed
lib/web/fetch/formdata-parser.js Fixed Show fixed Hide fixed
@KhafraDev KhafraDev marked this pull request as ready for review March 3, 2024 15:53
package.json Show resolved Hide resolved
Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollina
Copy link
Member

mcollina commented Mar 3, 2024

Does this increase our wpt conpliance?

@KhafraDev
Copy link
Member Author

KhafraDev commented Mar 3, 2024

Does this increase our wpt conpliance?

Yes, there are 4 (out of 14 tests wrt .formData) that now pass and there are some untested behaviors/behaviors we had to workaround that are now correct:

  • We actually follow the steps in the spec.
  • .formData() is now a function that returns a Promise instead of being async (I asked in the WHATWG matrix chat and this is the expected behavior)
  • We were able to get rid of hacks in the code for a WPT: "Fetch aborted & connection closed when aborted after calling response.formData()" in fetch/api/abort/general.any.js
        // Wait a tick before checking if the request has been aborted.
        // Otherwise, a TypeError can be thrown when an AbortError should.
        await Promise.resolve()

@Uzlopak

This comment was marked as off-topic.

@KhafraDev
Copy link
Member Author

@Uzlopak ptal

Copy link
Contributor

@Uzlopak Uzlopak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@KhafraDev
Copy link
Member Author

We can either land this as-is or I can spend the next few days/week (its midterm week for me so it might take longer) porting the busboy tests and fixing edge cases.

@Uzlopak
Copy link
Contributor

Uzlopak commented Mar 5, 2024

how about a new branch and port the tests, and then when they are ported we merge it into main?

Can you port a test from busboy so that i can assess how much work it is?

@KhafraDev
Copy link
Member Author

KhafraDev commented Mar 5, 2024

Adding the tests is very simple for the sole reason that they don't use any test runner and rely completely on node:assert. I ported the tests in my old formdata parser branch: https://github.com/KhafraDev/undici/tree/undici-formdata-parser/test/busboy

However, I don't think there's much benefit in porting something without also fixing bugs.

@KhafraDev
Copy link
Member Author

It might be a little harder, given that we don't implement any limits, nor does the parser extend EventEmitter much less emit events, but it shouldn't take more than an hour?

@Uzlopak
Copy link
Contributor

Uzlopak commented Mar 5, 2024

ah you mean the original busboy.

@KhafraDev
Copy link
Member Author

I assumed that @fastify/busboy would have the same tests, guess I was wrong. Busboy's tests seem more relevant here since they test the actual parsing/output, where as fastify/busboy seems to test more so the actual api (which obviously we'd fail).

@Uzlopak
Copy link
Contributor

Uzlopak commented Mar 5, 2024

actually busboy (original) mixed dicer and co into it. @fastify/busboy vendored them.

Anyhow i think that our @fastify/busboy tests were kind of more reasonable.

What about these tests?

https://github.com/fastify/busboy/blob/master/test/types-multipart.test.js

@KhafraDev
Copy link
Member Author

No preference, but I don't like how expected parts removed the objects for arrays, now I have no idea what each index means.

Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link
Contributor

@gurgunday gurgunday left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me

@mcollina mcollina merged commit 5bb84ba into nodejs:main Mar 5, 2024
19 checks passed
@KhafraDev KhafraDev deleted the formdata-parsing-x2 branch March 5, 2024 17:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

undici parses FormData bodies incorrectly
8 participants