Test failures on Node v19 (and likely v20

[mcollina]

We have a few tests failures on Node v19 which we should work to iron out before the v20 release.

[targos]

Recent CITGM run (v19.8.0): https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/3139/nodes=fedora-latest-x64/testReport/junit/(root)/citgm/undici_v5_21_0/

[KhafraDev]

This is the failing test https://github.com/nodejs/undici/blob/main/test/issue-1903.js

[RishabhKodes]

Failure logs https://github.com/nodejs/undici/actions/runs/4477206842/jobs/7888223614#step:14:7115

# Subtest: test/issue-1903.js
    # Subtest: should parse content-disposition consistencely
        1..5
        not ok 1 - Invalid character in header content ["content-disposition"]
          ---
          stack: |
            Server.<anonymous> (test/issue-1903.js:27:9)
          at:
            line: 572
            column: 5
            file: node:_http_outgoing
            function: storeHeader
          type: TypeError
          code: ERR_INVALID_CHAR
          tapCaught: uncaughtException
          test: should parse content-disposition consistently
                    ...
        
    1..0 # no tests found
        # test count(1) != plan(5)
        # failed 1 test
    1..0 # no tests found
not ok 44 - test/issue-1903.js # time=60011.236ms

[climba03003]

Related to nodejs/node#46528
Seems like either side turns out provide or parsing the header wrongly.

[mcollina]

cc @marco-ippolito

[marco-ippolito]

With nodejs/node#46528 if there is explicit content-lenght, the content-disposition header is encoded in latin1 as per https://www.rfc-editor.org/rfc/rfc6266#section-4.3.
Probably there is a mismatch in the implementation, I'll have a look

[marco-ippolito]

I dont think this was ever supposed to work:

undici/test/issue-1903.js

Lines 26 to 30 in f0271d4

	const server = createServer((req, res) => {
	res.writeHead(200, {
	'content-length': 2,
	'content-disposition': "attachment; filename='år.pdf'"
	})

a header value with non-ascii characters is passed without encoding it.
nodejs/node#46528 this PR solved the behaviour mismatch in encoding when using content length.
@ShogunPanda

[KhafraDev]

Even once the header is encoded it still fails. I noticed that undici always encodes the content-disposition header to latin1, even if there is no content-length header too, but fixing that still doesn't pass the test either ☹️.

[KhafraDev]

I'm mostly sure this is a bug in node.

The test is triggering this condition and throwing an error:
https://github.com/nodejs/node/blob/57caf1356229bd24c6129574863fa414ff6862b2/lib/_http_outgoing.js#L627
where value is a Buffer:

buffer = Buffer.from(`66 69 6c 65 6e 61 6d 65 3d 27 fd 72 2e 70 64 66 27`.split(' ').map(v => parseInt(v, 16)))

so the regex is being called with a buffer, stringifying it to utf8 (as is the default encoding)

/[^\t\x20-\x7e\x80-\xff]/.exec(buffer) !== null // true

/[^\t\x20-\x7e\x80-\xff]/.exec(buffer.toString()) !== null // true

/[^\t\x20-\x7e\x80-\xff]/.exec(buffer.toString('latin1')) !== null // false, correct case

if I'm missing something please feel free to correct me 😄

---

adding on, it makes sense for the value to be a buffer because of the line here
https://github.com/nodejs/node/pull/46528/files#diff-48d21edbddb6e855d1ee5716c49bcdc0d913c11ee8a24a98ea7dbc60cd253556R583

[ronag]

Yea I also think this is a Node bug.

   res.writeHead(200, { 
     'content-length': 2, 
     'content-disposition': "attachment; filename='år.pdf'" 
   }) 

Should work and has nothing to do with undici

[mcollina]

@marco-ippolito wdyt?

[ronag]

I suggest, to get our tests working we disable this test for node 19+ until it's been fixed in Node. @mcollina wdyt?

[KhafraDev]

I have a PR that disables it & matches the parsing of content-disposition with node

[marco-ippolito]

yes this is a bug, å should be a valid character