New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fs: allow WHATWG URL and file: URLs as paths #10739

Closed
wants to merge 1 commit into
base: master
from

Conversation

@jasnell
Member

jasnell commented Jan 11, 2017

Updates the fs module APIs to allow 'file://' URL strings and WHATWG URL objects using 'file:' protocol to be passed as the path.

For example:

const URL = require('url').URL;
const myURL = new URL('file:///C:/path/to/file');
fs.readFile(myURL, (err, data) => {});

// or
// EDIT(addaleax): this was removed from this PR
fs.readFile('file:///C:/path/to/file', (err, data) => {});

On Windows, file: URLs with a hostname convert to UNC paths, while file: URLs with drive letters convert to local absolute paths:

file://hostname/a/b/c => \\hostname\a\b\c
file:///c:/a/b/c => c:\a\b\c

On all other platforms, file: URLs with a hostname are unsupported and will result in a throw:

file://hostname/a/b/c => throw!
file:///a/b/c => /a/b/c

The documentation for the fs API is intentionally not updated in this commit because the URL API is still considered experimental and is not officially documented at this time

Note that file: URLs are required by spec to always be absolute paths from the file system root.

This is a semver-major commit because it changes error handling on the fs APIs.

Refs: #10703

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines
Affected core subsystem(s)

fs, url

@jasnell

This comment has been minimized.

Show comment
Hide comment
@joyeecheung

This comment has been minimized.

Show comment
Hide comment
@joyeecheung

joyeecheung Jan 11, 2017

Member

I think the url label can be removed here? (Also, can you take a look at nodejs/github-bot#115 so that the bot will stop labeling these PRs with url?)

Member

joyeecheung commented Jan 11, 2017

I think the url label can be removed here? (Also, can you take a look at nodejs/github-bot#115 so that the bot will stop labeling these PRs with url?)

@jasnell

This comment has been minimized.

Show comment
Hide comment
@jasnell

jasnell Jan 11, 2017

Member

heh, yeah, the bot seems to be a bit aggressive about labeling things these days

Member

jasnell commented Jan 11, 2017

heh, yeah, the bot seems to be a bit aggressive about labeling things these days

@jasnell

This comment has been minimized.

Show comment
Hide comment
@jasnell

jasnell Jan 11, 2017

Member

New CI: https://ci.nodejs.org/job/node-test-pull-request/5811/
(previous one had an error on windows)

Member

jasnell commented Jan 11, 2017

New CI: https://ci.nodejs.org/job/node-test-pull-request/5811/
(previous one had an error on windows)

Show outdated Hide outdated lib/fs.js
@@ -211,6 +213,7 @@ fs.access = function(path, mode, callback) {
throw new TypeError('"callback" argument must be a function');
}
path = getPathFromURL(path);

This comment has been minimized.

@joyeecheung

joyeecheung Jan 11, 2017

Member

The WHATWG URL parsing could escape the null byte if it's not in the hostname ('\u0000' -> '%00', e.g. new URL('file://hostname/a/b/c\u0000')), thus bypassing the null check, so might worth add a test for it. Is it possible for getPathFromURL to put null bytes in a path without one? If not, maybe the nullCheck can be performed before getPathFromURL.

@joyeecheung

joyeecheung Jan 11, 2017

Member

The WHATWG URL parsing could escape the null byte if it's not in the hostname ('\u0000' -> '%00', e.g. new URL('file://hostname/a/b/c\u0000')), thus bypassing the null check, so might worth add a test for it. Is it possible for getPathFromURL to put null bytes in a path without one? If not, maybe the nullCheck can be performed before getPathFromURL.

This comment has been minimized.

@jasnell

jasnell Jan 11, 2017

Member

good point. it's not just about null bytes, the path could contain any number of pct-encoded characters. This should run the path through a decode before returning... will fix that

@jasnell

jasnell Jan 11, 2017

Member

good point. it's not just about null bytes, the path could contain any number of pct-encoded characters. This should run the path through a decode before returning... will fix that

This comment has been minimized.

@joyeecheung

joyeecheung Jan 11, 2017

Member

Yes, that can be a problem for files with non-ASCII names...might worth add a test for something like 'file://hostname/文件.txt'.

@joyeecheung

joyeecheung Jan 11, 2017

Member

Yes, that can be a problem for files with non-ASCII names...might worth add a test for something like 'file://hostname/文件.txt'.

This comment has been minimized.

@joyeecheung

joyeecheung Jan 11, 2017

Member

Also, the hostname can be escaped by domain to ASCII too. Like 'file://你好你好/a/b'

@joyeecheung

joyeecheung Jan 11, 2017

Member

Also, the hostname can be escaped by domain to ASCII too. Like 'file://你好你好/a/b'

Show outdated Hide outdated lib/internal/url.js
function isFileUrl(path) {
if (typeof path !== 'string') return false;
if (path.length < 7) return false;
if ((path[0].codePointAt(0) | 0x20) !== 102) return false; // f F

This comment has been minimized.

@joyeecheung

joyeecheung Jan 11, 2017

Member

Why not path.codePointAt(0)? Or path.startsWith('file://')?

@joyeecheung

joyeecheung Jan 11, 2017

Member

Why not path.codePointAt(0)? Or path.startsWith('file://')?

Show outdated Hide outdated lib/internal/url.js
} else {
return path;
}
if (url.protocol !== 'file:')

This comment has been minimized.

@joyeecheung

joyeecheung Jan 11, 2017

Member

Maybe this can be moved into the else if (path instanceof URL) block, because a path that starts with file://(isFileUrl) will have the file protocol.

@joyeecheung

joyeecheung Jan 11, 2017

Member

Maybe this can be moved into the else if (path instanceof URL) block, because a path that starts with file://(isFileUrl) will have the file protocol.

@jasnell

This comment has been minimized.

Show comment
Hide comment
@jasnell

This comment has been minimized.

Show comment
Hide comment
@jasnell

jasnell Jan 12, 2017

Member

sigh... lol... obvious that last run was less than successful ;-) ... will investigate later on tonight

Member

jasnell commented Jan 12, 2017

sigh... lol... obvious that last run was less than successful ;-) ... will investigate later on tonight

@jasnell

This comment has been minimized.

Show comment
Hide comment
Show outdated Hide outdated foobar
@@ -0,0 +1 @@
foobar

This comment has been minimized.

@addaleax

addaleax Jan 12, 2017

Member

Is checking in this file intentional?

@addaleax

addaleax Jan 12, 2017

Member

Is checking in this file intentional?

This comment has been minimized.

@jasnell

jasnell Jan 12, 2017

Member

sigh

@jasnell
Show outdated Hide outdated lib/internal/url.js
url = new URL(path);
} else if (path instanceof URL) {
if (path.protocol !== 'file:')
return new Error('Only file: URLs are supported');

This comment has been minimized.

@addaleax

addaleax Jan 12, 2017

Member

I’d wrap file: in backticks to make this a bit more readable

@addaleax

addaleax Jan 12, 2017

Member

I’d wrap file: in backticks to make this a bit more readable

Show outdated Hide outdated lib/internal/url.js
}
}
function getPathFromURLOther(url) {

This comment has been minimized.

@addaleax

addaleax Jan 12, 2017

Member

getPathFromURLPosix might be a good name, that would mirror path.win32 and path.posix

@addaleax

addaleax Jan 12, 2017

Member

getPathFromURLPosix might be a good name, that would mirror path.win32 and path.posix

Show outdated Hide outdated lib/internal/url.js
// Otherwise, it's a local path that requires a drive letter
var letter = pathname.codePointAt(1) | 0x20;
var sep = pathname[2];
if (letter < 96 || letter > 122 ||

This comment has been minimized.

@addaleax

addaleax Jan 12, 2017

Member

letter < 97? And maybe add comments for a and z here fore clarity, too.

@addaleax

addaleax Jan 12, 2017

Member

letter < 97? And maybe add comments for a and z here fore clarity, too.

This comment has been minimized.

@jasnell

jasnell Jan 12, 2017

Member

doh... silly typo

@jasnell

jasnell Jan 12, 2017

Member

doh... silly typo

Show outdated Hide outdated lib/internal/url.js
function getPathFromURL(path) {
var url;
if (isFileUrl(path)) {
url = new URL(path);

This comment has been minimized.

@addaleax

addaleax Jan 12, 2017

Member

Are we sure this is safe? It feels like the kind of thing that could mess up any currently existing checks that userland applications run on the pathnames… Also, technically file:///asdf is a perfectly valid relative path on POSIX, so that alone makes this a breaking change.
I’d feel a lot better if @nodejs/security could give this a look.

And if we make decisions based on isFileUrl(path), we should probably make that function accessible from userland, too.

@addaleax

addaleax Jan 12, 2017

Member

Are we sure this is safe? It feels like the kind of thing that could mess up any currently existing checks that userland applications run on the pathnames… Also, technically file:///asdf is a perfectly valid relative path on POSIX, so that alone makes this a breaking change.
I’d feel a lot better if @nodejs/security could give this a look.

And if we make decisions based on isFileUrl(path), we should probably make that function accessible from userland, too.

This comment has been minimized.

@jasnell

jasnell Jan 12, 2017

Member

Yep, checking for file:// in the string input is a breaking change. We could limit this PR to only handle URL instances passed in and not do checking/parsing of the string. That would make it perform a bit better also. I'm ok with pulling this part back out.

@jasnell

jasnell Jan 12, 2017

Member

Yep, checking for file:// in the string input is a breaking change. We could limit this PR to only handle URL instances passed in and not do checking/parsing of the string. That would make it perform a bit better also. I'm ok with pulling this part back out.

This comment has been minimized.

@jasnell

jasnell Jan 12, 2017

Member

I'm going to go ahead and pull out the file:// check and parsing so that the string input is not touched at all.

@jasnell

jasnell Jan 12, 2017

Member

I'm going to go ahead and pull out the file:// check and parsing so that the string input is not touched at all.

@jasnell

This comment has been minimized.

Show comment
Hide comment
@jasnell

jasnell Jan 12, 2017

Member

@addaleax ... updated!

Member

jasnell commented Jan 12, 2017

@addaleax ... updated!

@addaleax

LGTM

Should this come with docs changes? Also, this looks like it can pass as semver-minor now to me. :)

Show outdated Hide outdated lib/internal/url.js
var hostname = url.hostname;
var pathname = url.pathname;
if (hostname !== '') {
// If hostname is set, then we have a UNC path

This comment has been minimized.

@addaleax

addaleax Jan 12, 2017

Member

… where decoding of percent escapes is handled by the OS? If yes, it might be nice to have the comment extended by that. :)

@addaleax

addaleax Jan 12, 2017

Member

… where decoding of percent escapes is handled by the OS? If yes, it might be nice to have the comment extended by that. :)

This comment has been minimized.

@jasnell

jasnell Jan 12, 2017

Member

I'm not sure if we should doc the changes yet or not. I still haven't even been able to land the docs for the URL object itself yet.

And good point about the handling of pct-decoding in the hostname. We should likely be doing that here instead of relying on the OS to do it.

@jasnell

jasnell Jan 12, 2017

Member

I'm not sure if we should doc the changes yet or not. I still haven't even been able to land the docs for the URL object itself yet.

And good point about the handling of pct-decoding in the hostname. We should likely be doing that here instead of relying on the OS to do it.

This comment has been minimized.

@jasnell

jasnell Jan 13, 2017

Member

Actually... nevermind about the pct escapes... for hostnames, that occurs while the URL is being parsed. There is a chance, however, that the hostname would be punycode encoded so I'll need to adjust for that... although I'm not entirely sure about the current support for extended characters in UNC hostnames.

@jasnell

jasnell Jan 13, 2017

Member

Actually... nevermind about the pct escapes... for hostnames, that occurs while the URL is being parsed. There is a chance, however, that the hostname would be punycode encoded so I'll need to adjust for that... although I'm not entirely sure about the current support for extended characters in UNC hostnames.

Show outdated Hide outdated lib/internal/url.js
}
pathname = pathname.slice(1);
for (var n = 0; n < pathname.length; n++) {
if (pathname[n] === '%') {

This comment has been minimized.

@joyeecheung

joyeecheung Jan 13, 2017

Member

Maybe a && n + 2 < pathname.length bound check here?

@joyeecheung

joyeecheung Jan 13, 2017

Member

Maybe a && n + 2 < pathname.length bound check here?

This comment has been minimized.

@jasnell

jasnell Jan 13, 2017

Member

There's no need for the extra check. If n >= pathname.length, the pathname[n] will return undefined, which causes this check to fail.

@jasnell

jasnell Jan 13, 2017

Member

There's no need for the extra check. If n >= pathname.length, the pathname[n] will return undefined, which causes this check to fail.

This comment has been minimized.

@joyeecheung

joyeecheung Jan 13, 2017

Member

But when n >= pathname.length - 2, pathname.codePointAt(n + 2) (and pathname[n+1] later) would be out of bound, no?

@joyeecheung

joyeecheung Jan 13, 2017

Member

But when n >= pathname.length - 2, pathname.codePointAt(n + 2) (and pathname[n+1] later) would be out of bound, no?

This comment has been minimized.

@jasnell

jasnell Jan 13, 2017

Member

Technically yes, but because the indexes are out of bound, each safely return undefined:

> var m = '';
undefined
> m.codePointAt(2);
undefined
> m[2]
undefined
>
@jasnell

jasnell Jan 13, 2017

Member

Technically yes, but because the indexes are out of bound, each safely return undefined:

> var m = '';
undefined
> m.codePointAt(2);
undefined
> m[2]
undefined
>
Show outdated Hide outdated lib/internal/url.js
}
var pathname = url.pathname;
for (var n = 0; n < pathname.length; n++) {
if (pathname[n] === '%') {

This comment has been minimized.

@joyeecheung

joyeecheung Jan 13, 2017

Member

Same as above.

@joyeecheung

joyeecheung Jan 13, 2017

Member

Same as above.

Show outdated Hide outdated lib/internal/url.js
(sep !== ':' && sep !== '|')) {
return new Error('File URLs must specify absolute paths');
}
pathname = pathname.slice(1);

This comment has been minimized.

@joyeecheung

joyeecheung Jan 13, 2017

Member

The first three character is known here so maybe we can start from 3?

@joyeecheung

joyeecheung Jan 13, 2017

Member

The first three character is known here so maybe we can start from 3?

This comment has been minimized.

@jasnell

jasnell Jan 13, 2017

Member

Checking the first three characters is to verify that the path begins with a proper windows drive letter. Once we do that, we simply strip the leading / off. For instance, in the file URL file:///c:/foo, the pathname component will be /c:/foo, so once we verify that c: is a proper drive letter we use slice to drop the leading / making it c:/foo.

@jasnell

jasnell Jan 13, 2017

Member

Checking the first three characters is to verify that the path begins with a proper windows drive letter. Once we do that, we simply strip the leading / off. For instance, in the file URL file:///c:/foo, the pathname component will be /c:/foo, so once we verify that c: is a proper drive letter we use slice to drop the leading / making it c:/foo.

This comment has been minimized.

@joyeecheung

joyeecheung Jan 13, 2017

Member

Ah yes, I didn't realize the pathname is passed as the .slice(1) version in the decodeURIComponent() below.

@joyeecheung

joyeecheung Jan 13, 2017

Member

Ah yes, I didn't realize the pathname is passed as the .slice(1) version in the decodeURIComponent() below.

Show outdated Hide outdated lib/internal/url.js
pathname = pathname.slice(1);
for (var n = 0; n < pathname.length; n++) {
if (pathname[n] === '%') {
var third = pathname.codePointAt(n + 2) | 0x20;

This comment has been minimized.

@joyeecheung

joyeecheung Jan 13, 2017

Member

Is there a reason for mixing [] and .codePointAt together?

@joyeecheung

joyeecheung Jan 13, 2017

Member

Is there a reason for mixing [] and .codePointAt together?

This comment has been minimized.

@jasnell

jasnell Jan 13, 2017

Member

codePointAt() is not a completely free operation performance-wise. I'm only using it where necessary.

@jasnell

jasnell Jan 13, 2017

Member

codePointAt() is not a completely free operation performance-wise. I'm only using it where necessary.

Show outdated Hide outdated lib/internal/url.js
}
}
}
return decodeURIComponent(pathname);

This comment has been minimized.

@joyeecheung

joyeecheung Jan 13, 2017

Member

Maybe we can decode first and then search for / and \?

@joyeecheung

joyeecheung Jan 13, 2017

Member

Maybe we can decode first and then search for / and \?

This comment has been minimized.

@jasnell

jasnell Jan 13, 2017

Member

We cannot. Consider the following URL: file:///c:/%2f%5c/foo. Here the pathname component is /c:/%2f%5c/, if we decode the pathname first, it would become /c://\/, making it impossible to distinguish the decoded / from the legitimate / path separators that already exist. The only approach that works is to scan for the %2f and %5c patterns first and then decode.

@jasnell

jasnell Jan 13, 2017

Member

We cannot. Consider the following URL: file:///c:/%2f%5c/foo. Here the pathname component is /c:/%2f%5c/, if we decode the pathname first, it would become /c://\/, making it impossible to distinguish the decoded / from the legitimate / path separators that already exist. The only approach that works is to scan for the %2f and %5c patterns first and then decode.

const Buffer = require('buffer').Buffer;
function pathToFileURL(p) {
if (!path.isAbsolute(p))

This comment has been minimized.

@joyeecheung

joyeecheung Jan 13, 2017

Member

If we only pass the result of path.resolve here then is this check necessary?

@joyeecheung

joyeecheung Jan 13, 2017

Member

If we only pass the result of path.resolve here then is this check necessary?

This comment has been minimized.

@jasnell

jasnell Jan 13, 2017

Member

Not entirely necessary but it's a good fallback check just to be safe. If carries a slight performance hit but in a test that is harmless.

@jasnell

jasnell Jan 13, 2017

Member

Not entirely necessary but it's a good fallback check just to be safe. If carries a slight performance hit but in a test that is harmless.

@jasnell

This comment has been minimized.

Show comment
Hide comment
@jasnell

jasnell Jan 13, 2017

Member

@addaleax ... updated! I fixed the pct encoding for paths and the punycode decoding for the hostname in the win32 variant. Thank you very much for catching that. Should be good to go now. PTAL

Member

jasnell commented Jan 13, 2017

@addaleax ... updated! I fixed the pct encoding for paths and the punycode decoding for the hostname in the win32 variant. Thank you very much for catching that. Should be good to go now. PTAL

@TimothyGu TimothyGu moved this from Spec Irrelavent / De-facto to Done in WHATWG URL implementation Feb 14, 2017

italoacasas added a commit to italoacasas/node that referenced this pull request Feb 14, 2017

fs: allow WHATWG URL and file: URLs as paths
Updates the fs module APIs to allow 'file://' URL objects
to be passed as the path.

For example:

```js
const URL = require('url').URL;
const myURL = new URL('file:///C:/path/to/file');
fs.readFile(myURL, (err, data) => {});
```

On Windows, file: URLs with a hostname convert to UNC paths,
while file: URLs with drive letters convert to local absolute
paths:

```
file://hostname/a/b/c => \\hostname\a\b\c
file:///c:/a/b/c => c:\a\b\c
```

On all other platforms, file: URLs with a hostname are unsupported
and will result in a throw:

```
file://hostname/a/b/c => throw!
file:///a/b/c => /a/b/c
```

The documentation for the fs API is intentionally not updated in
this commit because the URL API is still considered experimental
and is not officially documented *at this time*

Note that file: URLs are *required* by spec to always be absolute
paths from the file system root.

This is a semver-major commit because it changes error handling
on the fs APIs.

PR-URL: nodejs#10739
Ref: nodejs#10703
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>

italoacasas added a commit that referenced this pull request Feb 16, 2017

2017-02-21, Version 7.6.0 (Current)
Notable changes:

* deps:
    * update V8 to 5.5 (Michaël Zasso) [#11029](#11029)
    * upgrade libuv to 1.11.0 (cjihrig) [#11094](#11094)
    * add node-inspect 1.10.2 (Jan Krems) [#10187](#10187)
* lib: build `node inspect` into `node` (Anna Henningsen) [#10187](#10187)
* crypto: Remove expired certs from CNNIC whitelist (Shigeki Ohtsu) [#9469](#9469)
* inspector: add --inspect-brk (Josh Gavant) [#11149](#11149)
* fs: allow WHATWG URL and file: URLs as paths (James M Snell) [#10739](#10739)
* src: support UTF-8 in compiled-in JS source files (Ben Noordhuis) [#11129](#11129)
* url: extend url.format to support WHATWG URL (James M Snell) [#10857](#10857)

PR-URL: #11185

addaleax added a commit that referenced this pull request Feb 21, 2017

2017-02-21, Version 7.6.0 (Current)
Notable changes:

* deps:
    * update V8 to 5.5 (Michaël Zasso) [#11029](#11029)
    * upgrade libuv to 1.11.0 (cjihrig) [#11094](#11094)
    * add node-inspect 1.10.4 (Jan Krems) [#10187](#10187)
* lib: build `node inspect` into `node` (Anna Henningsen) [#10187](#10187)
* crypto: Remove expired certs from CNNIC whitelist (Shigeki Ohtsu) [#9469](#9469)
* inspector: add --inspect-brk (Josh Gavant) [#11149](#11149)
* fs: allow WHATWG URL objects as paths (James M Snell) [#10739](#10739)
* src: support UTF-8 in compiled-in JS source files (Ben Noordhuis) [#11129](#11129)
* url: extend url.format to support WHATWG URL (James M Snell) [#10857](#10857)

PR-URL: #11185

italoacasas added a commit to italoacasas/node that referenced this pull request Feb 21, 2017

2017-02-21, Version 7.6.0 (Current)
Notable changes:

* deps:
    * update V8 to 5.5 (Michaël Zasso) [#11029](nodejs#11029)
    * upgrade libuv to 1.11.0 (cjihrig) [#11094](nodejs#11094)
    * add node-inspect 1.10.4 (Jan Krems) [#10187](nodejs#10187)
    * upgrade zlib to 1.2.11 (Sam Roberts) [#10980](nodejs#10980)
* lib: build `node inspect` into `node` (Anna Henningsen) [#10187](nodejs#10187)
* crypto: Remove expired certs from CNNIC whitelist (Shigeki Ohtsu) [#9469](nodejs#9469)
* inspector: add --inspect-brk (Josh Gavant) [#11149](nodejs#11149)
* fs: allow WHATWG URL objects as paths (James M Snell) [#10739](nodejs#10739)
* src: support UTF-8 in compiled-in JS source files (Ben Noordhuis) [#11129](nodejs#11129)
* url: extend url.format to support WHATWG URL (James M Snell) [#10857](nodejs#10857)

PR-URL: nodejs#11185

italoacasas added a commit to italoacasas/node that referenced this pull request Feb 22, 2017

2017-02-21, Version 7.6.0 (Current)
Notable changes:

* deps:
    * update V8 to 5.5 (Michaël Zasso) [#11029](nodejs#11029)
    * upgrade libuv to 1.11.0 (cjihrig) [#11094](nodejs#11094)
    * add node-inspect 1.10.4 (Jan Krems) [#10187](nodejs#10187)
    * upgrade zlib to 1.2.11 (Sam Roberts) [#10980](nodejs#10980)
* lib: build `node inspect` into `node` (Anna Henningsen) [#10187](nodejs#10187)
* crypto: Remove expired certs from CNNIC whitelist (Shigeki Ohtsu) [#9469](nodejs#9469)
* inspector: add --inspect-brk (Josh Gavant) [#11149](nodejs#11149)
* fs: allow WHATWG URL objects as paths (James M Snell) [#10739](nodejs#10739)
* src: support UTF-8 in compiled-in JS source files (Ben Noordhuis) [#11129](nodejs#11129)
* url: extend url.format to support WHATWG URL (James M Snell) [#10857](nodejs#10857)

PR-URL: nodejs#11185

KryDos added a commit to KryDos/node that referenced this pull request Feb 25, 2017

fs: allow WHATWG URL and file: URLs as paths
Updates the fs module APIs to allow 'file://' URL objects
to be passed as the path.

For example:

```js
const URL = require('url').URL;
const myURL = new URL('file:///C:/path/to/file');
fs.readFile(myURL, (err, data) => {});
```

On Windows, file: URLs with a hostname convert to UNC paths,
while file: URLs with drive letters convert to local absolute
paths:

```
file://hostname/a/b/c => \\hostname\a\b\c
file:///c:/a/b/c => c:\a\b\c
```

On all other platforms, file: URLs with a hostname are unsupported
and will result in a throw:

```
file://hostname/a/b/c => throw!
file:///a/b/c => /a/b/c
```

The documentation for the fs API is intentionally not updated in
this commit because the URL API is still considered experimental
and is not officially documented *at this time*

Note that file: URLs are *required* by spec to always be absolute
paths from the file system root.

This is a semver-major commit because it changes error handling
on the fs APIs.

PR-URL: nodejs#10739
Ref: nodejs#10703
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>

KryDos added a commit to KryDos/node that referenced this pull request Feb 25, 2017

fs: allow WHATWG URL and file: URLs as paths
Updates the fs module APIs to allow 'file://' URL objects
to be passed as the path.

For example:

```js
const URL = require('url').URL;
const myURL = new URL('file:///C:/path/to/file');
fs.readFile(myURL, (err, data) => {});
```

On Windows, file: URLs with a hostname convert to UNC paths,
while file: URLs with drive letters convert to local absolute
paths:

```
file://hostname/a/b/c => \\hostname\a\b\c
file:///c:/a/b/c => c:\a\b\c
```

On all other platforms, file: URLs with a hostname are unsupported
and will result in a throw:

```
file://hostname/a/b/c => throw!
file:///a/b/c => /a/b/c
```

The documentation for the fs API is intentionally not updated in
this commit because the URL API is still considered experimental
and is not officially documented *at this time*

Note that file: URLs are *required* by spec to always be absolute
paths from the file system root.

This is a semver-major commit because it changes error handling
on the fs APIs.

PR-URL: nodejs#10739
Ref: nodejs#10703
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>

imyller added a commit to imyller/meta-nodejs that referenced this pull request Mar 2, 2017

2017-02-21, Version 7.6.0 (Current)
    Notable changes:

    * deps:
        * update V8 to 5.5 (Michaël Zasso) [#11029](nodejs/node#11029)
        * upgrade libuv to 1.11.0 (cjihrig) [#11094](nodejs/node#11094)
        * add node-inspect 1.10.4 (Jan Krems) [#10187](nodejs/node#10187)
        * upgrade zlib to 1.2.11 (Sam Roberts) [#10980](nodejs/node#10980)
    * lib: build `node inspect` into `node` (Anna Henningsen) [#10187](nodejs/node#10187)
    * crypto: Remove expired certs from CNNIC whitelist (Shigeki Ohtsu) [#9469](nodejs/node#9469)
    * inspector: add --inspect-brk (Josh Gavant) [#11149](nodejs/node#11149)
    * fs: allow WHATWG URL objects as paths (James M Snell) [#10739](nodejs/node#10739)
    * src: support UTF-8 in compiled-in JS source files (Ben Noordhuis) [#11129](nodejs/node#11129)
    * url: extend url.format to support WHATWG URL (James M Snell) [#10857](nodejs/node#10857)

    PR-URL: nodejs/node#11185

Signed-off-by: Ilkka Myller <ilkka.myller@nodefield.com>
@sam-github

This comment has been minimized.

Show comment
Hide comment
@sam-github

sam-github Apr 11, 2017

Member

@addaleax said:

Should this come with docs changes?

but I didn't see any follow up comments. Why is this not documented? I just found it while browsing the fs source.

Member

sam-github commented Apr 11, 2017

@addaleax said:

Should this come with docs changes?

but I didn't see any follow up comments. Why is this not documented? I just found it while browsing the fs source.

@addaleax

This comment has been minimized.

Show comment
Hide comment
@addaleax

addaleax Apr 11, 2017

Member

@sam-github Quoting the PR description (I think this had been added later):

The documentation for the fs API is intentionally not updated in this commit because the URL API is still considered experimental and is not officially documented at this time

Member

addaleax commented Apr 11, 2017

@sam-github Quoting the PR description (I think this had been added later):

The documentation for the fs API is intentionally not updated in this commit because the URL API is still considered experimental and is not officially documented at this time

@sam-github

This comment has been minimized.

Show comment
Hide comment
@sam-github

sam-github Apr 11, 2017

Member

I see. We can't document that fs APIs take a url.URL object when url.URL itself is undocumented. Fair enough.

Member

sam-github commented Apr 11, 2017

I see. We can't document that fs APIs take a url.URL object when url.URL itself is undocumented. Fair enough.

@addaleax

This comment has been minimized.

Show comment
Hide comment
@addaleax

addaleax Apr 11, 2017

Member

To be fair, url.URL has been documented since this PR landed: https://nodejs.org/api/url.html#url_the_whatwg_url_api. I’ll open a good first contribution issue asking for docs for this feature.

Member

addaleax commented Apr 11, 2017

To be fair, url.URL has been documented since this PR landed: https://nodejs.org/api/url.html#url_the_whatwg_url_api. I’ll open a good first contribution issue asking for docs for this feature.

@gibfahn gibfahn referenced this pull request Apr 24, 2017

Closed

Potential Semver Minor Backports #188

28 of 31 tasks complete

@anchnk anchnk referenced this pull request Apr 26, 2017

Closed

doc: add WHATWG file URLs in fs module #12670

2 of 2 tasks complete

anchnk added a commit to anchnk/node that referenced this pull request May 4, 2017

doc: add WHATWG file URLs in fs module
Update fs module documentation adding WHATWG file URLS support for
relevant fs functions/classes.

Fixes: nodejs#12341
Refs: nodejs#10739

jasnell added a commit that referenced this pull request May 4, 2017

doc: add WHATWG file URLs in fs module
Update fs module documentation adding WHATWG file URLS support for
relevant fs functions/classes.

PR-URL: #12670
Fixes: #12341
Ref: #10739
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Timothy Gu <timothygu99@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

anchnk added a commit to anchnk/node that referenced this pull request May 6, 2017

doc: add WHATWG file URLs in fs module
Update fs module documentation adding WHATWG file URLS support for
relevant fs functions/classes.

Fixes: nodejs#12341
Refs: nodejs#10739

@TimothyGu TimothyGu referenced this pull request May 8, 2017

Closed

file:// URLs #75

dobriai pushed a commit to dobriai/electron-oauth2-3legged that referenced this pull request Apr 21, 2018

Ivan Dobrianov
Path fix for windows - leading slash
Looks like there is some junk with Windows paths. To put it simply, the
following call is an Identity operation under Linux, but leaves a
leading slash under Windows:

url.parse( url.format( { pathname: path.join(process.cwd(), 'got_access_token.html'), protocol: 'elif:', slashes: true } )).pathname

On windows you get '/E:/some/path/...' - note the leading slash.

Some discussions here:
  nodejs/node#10703
  nodejs/node#10739

Dunno! I just fixed it by hand.

@addaleax addaleax referenced this pull request May 24, 2018

Closed

fs: allow file: URL strings as paths #20944

3 of 4 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment