url, i18n: update IDNA handling #13362

TimothyGu · 2017-06-01T06:38:02Z

The PR is still somewhat of a WIP, as a proof-of-concept that the proposed changes to the URL Standard work (whatwg/url#309).

Refs: whatwg/url#309
Refs: web-platform-tests/wpt#5976

/cc @domenic @zimbabao

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
commit message follows commit guidelines

Affected core subsystem(s)

url

refack · 2017-06-01T13:06:31Z

src/node_i18n.cc

+  if (mode == IDNA_STRICT) {
+    options |= UIDNA_USE_STD3_RULES;  // UseSTD3ASCIIRules = beStrict
+                                      // VerifyDnsLength = beStrict;
+                                      //   handled later


nit: align white space in comment?

It was intended to be a continuation of the previous line due to the line length lint rule, thus indented.

refack

LGTM % nit and a non-blocking request

refack · 2017-06-01T13:08:11Z

src/node_i18n.cc

-  // names in the wild. As such, in the current UTS #46 draft (rev. 18) these
-  // checks are made optional depending on the CheckHyphens flag, which will be
-  // disabled in WHATWG URL's "domain to ASCII" algorithm soon.
+  // In UTS #46 which specifies ToASCII, certain error conditions are


👍 love the comment.

refack · 2017-06-01T13:20:10Z

src/node_i18n.cc


  MaybeStackBuffer<char> buf;
-  int32_t len = ToASCII(&buf, *val, val.length(), lenient);
+  int32_t len = ToASCII(&buf, *val, val.length(), mode);


[question] AFAICT this is the only call to the workhorse ToASCII, so why implement IDNA_STRICT? Future proofing?

So since it's not exposed maybe add a test case for IDNA_STRICT in test/cctest/test_url.cc

The strict mode corresponds to beStrict variable in the algorithm. I'll make a mention of this in the code.

[follow up]so in the current implementation IDNA_STRICT exists only in the sense that !IDNA_STRICT <=> IDNA_DEFAULT || IDNA_LENIENT?

zimbabao · 2017-06-01T16:50:48Z

LGTM

watilde

LGTM so far. Let's wait till upstream gets merge the PRs :)

TimothyGu · 2017-06-03T03:26:59Z

Rebased, @refack's comment addressed. The upstream PRs were landed so this should be good to go.

CI: https://ci.nodejs.org/job/node-test-pull-request/8450/

TimothyGu · 2017-06-04T05:18:35Z

Redundant test cases removed. One last CI: ~~https://ci.nodejs.org/job/node-test-pull-request/8466/~~ https://ci.nodejs.org/job/node-test-pull-request/8525/

Remove custom tests for invalid IDNA domains in url-idna.js in favor of the more comprehensive official set. Refs: whatwg/url#309 Refs: web-platform-tests/wpt#5976

TimothyGu · 2017-06-07T06:36:04Z

Landed in 91a1bbe.

Remove custom tests for invalid IDNA domains in url-idna.js in favor of the more comprehensive official set. PR-URL: nodejs#13362 Refs: whatwg/url#309 Refs: web-platform-tests/wpt#5976 Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Daijiro Wachi <daijiro.wachi@gmail.com>

Remove custom tests for invalid IDNA domains in url-idna.js in favor of the more comprehensive official set. PR-URL: #13362 Refs: whatwg/url#309 Refs: web-platform-tests/wpt#5976 Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Daijiro Wachi <daijiro.wachi@gmail.com>

MylesBorins · 2017-07-17T17:51:30Z

This does not land cleanly in LTS. Please feel free to manually backport. Please also feel free to replace do-not-land if it is being backported

TimothyGu · 2017-07-18T00:44:07Z

dont-land is okay for this one, as the WHATWG URL parser is not in v6.x.

Remove custom tests for invalid IDNA domains in url-idna.js in favor of the more comprehensive official set. PR-URL: nodejs#13362 Refs: whatwg/url#309 Refs: web-platform-tests/wpt#5976 Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Daijiro Wachi <daijiro.wachi@gmail.com>

Remove custom tests for invalid IDNA domains in url-idna.js in favor of the more comprehensive official set. Backport-PR-URL: #17365 PR-URL: #13362 Refs: whatwg/url#309 Refs: web-platform-tests/wpt#5976 Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Daijiro Wachi <daijiro.wachi@gmail.com>

TimothyGu requested review from jasnell, srl295 and watilde June 1, 2017 06:38

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. i18n-api Issues and PRs related to the i18n implementation. labels Jun 1, 2017

TimothyGu mentioned this pull request Jun 1, 2017

Address several IDNA issues whatwg/url#309

Merged

refack reviewed Jun 1, 2017

View reviewed changes

refack approved these changes Jun 1, 2017

View reviewed changes

watilde approved these changes Jun 1, 2017

View reviewed changes

jasnell approved these changes Jun 1, 2017

View reviewed changes

TimothyGu force-pushed the url-idna branch from 8ff7b39 to bfd903e Compare June 3, 2017 03:25

TimothyGu force-pushed the url-idna branch from bfd903e to 6f75b9c Compare June 4, 2017 05:18

url: update IDNA handling

48dc28e

Remove custom tests for invalid IDNA domains in url-idna.js in favor of the more comprehensive official set. Refs: whatwg/url#309 Refs: web-platform-tests/wpt#5976

TimothyGu force-pushed the url-idna branch from 6f75b9c to 48dc28e Compare June 7, 2017 05:09

TimothyGu closed this Jun 7, 2017

TimothyGu deleted the url-idna branch June 7, 2017 06:36

gibfahn mentioned this pull request Jun 15, 2017

Auditing for 6.11.1 nodejs/Release#230

Closed

3 tasks

MylesBorins added the dont-land-on-v6.x label Jul 17, 2017

TimothyGu mentioned this pull request Nov 28, 2017

Backport WHATWG URL to 6.x #17365

Closed

4 tasks

MylesBorins mentioned this pull request Jan 24, 2018

v6.13.0 proposal #18342

Merged

aliscco mentioned this pull request Apr 27, 2023

[Snyk] Fix for 30 vulnerabilities aliscco/alisco-node#279

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

url, i18n: update IDNA handling #13362

url, i18n: update IDNA handling #13362

TimothyGu commented Jun 1, 2017

refack Jun 1, 2017

TimothyGu Jun 3, 2017

refack Jun 3, 2017

refack left a comment

refack Jun 1, 2017

refack Jun 1, 2017

refack Jun 1, 2017

TimothyGu Jun 3, 2017

refack Jun 3, 2017

zimbabao commented Jun 1, 2017

watilde left a comment

TimothyGu commented Jun 3, 2017

TimothyGu commented Jun 4, 2017 •

edited

Loading

TimothyGu commented Jun 7, 2017

MylesBorins commented Jul 17, 2017

TimothyGu commented Jul 18, 2017

url, i18n: update IDNA handling #13362

url, i18n: update IDNA handling #13362

Conversation

TimothyGu commented Jun 1, 2017

Checklist

Affected core subsystem(s)

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

refack left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zimbabao commented Jun 1, 2017

watilde left a comment

Choose a reason for hiding this comment

TimothyGu commented Jun 3, 2017

TimothyGu commented Jun 4, 2017 • edited Loading

TimothyGu commented Jun 7, 2017

MylesBorins commented Jul 17, 2017

TimothyGu commented Jul 18, 2017

TimothyGu commented Jun 4, 2017 •

edited

Loading