Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Symlink support for packager #9009

Closed
wants to merge 10 commits into from

Conversation

@Kureev
Copy link
Collaborator

commented Jul 25, 2016

I saw we have quite a few user requests for symlink support...

Test plan (required)

  1. Create a symlink in node_modules (for instance use npm link)
  2. Run npm start
  3. Profit!

Code formatting

Look around. Match the style of the rest of the codebase. See also the simple style guide.

For more info, see the "Pull Requests" section of our "Contributing" guidelines.

@ghost

This comment has been minimized.

Copy link

commented Jul 25, 2016

By analyzing the blame information on this pull request, we identified @philikon and @christopherdro to be potential reviewers.

@Kureev Kureev changed the title Feature/watch symlinks Symlink support for packager Jul 25, 2016

@ghost ghost added the CLA Signed label Jul 25, 2016

@bestander

This comment has been minimized.

Copy link
Contributor

commented Jul 25, 2016

Cc @matryoshcow, our new Packager man in RN team

@ghost ghost added the CLA Signed label Jul 25, 2016

@ghost ghost added the CLA Signed label Jul 25, 2016

@bestander

This comment has been minimized.

Copy link
Contributor

commented Jul 25, 2016

FYI, big changes will be coming to packager, so all changes to current one need to be considered

@matryoshcow

This comment has been minimized.

Copy link

commented Jul 27, 2016

@Kureev, this is pretty neat! A couple of thoughts:

  1. as it stands, this would be enabled by default, even though it's clearly targeted at specific class of users; it might be a good idea to make symlink support optional
  2. to the extent possible, we're using async I/O; could your solution be modified in this manner?

bonus nitpick: one-letter variable names are the reason why we can't have nice things :)

@ghost ghost added the CLA Signed label Jul 27, 2016

@Kureev

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 27, 2016

Hi @matryoshcow!

About your notes:

  1. Yeah, it's definitely not for everybody, but on the other hand the overhead you have is just about fs.readdir + fs.lstat. You can already specify --projectRoots and it'll work for you. My intention was to make it work "by default"
  2. Sure!

@ghost ghost added the CLA Signed label Jul 27, 2016

@mkonicek

This comment has been minimized.

Copy link
Contributor

commented Jul 27, 2016

@matryoshcow If I have no symlinks in node_modules, this feature is disabled. This way the feature is only enabled for people who want to use it.

It might happen that people have symlinks in node_modules without knowing about it though, which could be confusing. Printing info about the found symlinks on packager startup could help.

Making this optional with a packager argument would work too, but people would have to learn about the extra argument (Google -> StackOverflow etc., figuring out if info is up to date which is extra work everyone has to do), and what if people launch the packager using Xcode? There are several different ways people launch the packager which makes it more work to make the feature super easy to use.

@mkonicek

This comment has been minimized.

Copy link
Contributor

commented Jul 27, 2016

I'll request changes just for adding some debugging output so people know they're using this feature.

Agree with @matryoshcow's nit that some variable names could be made a little bit clearer, for example:

.filter(...)
.map(symlink => path.resolve(process.cwd(), fs.readlinkSync(symlink)));
@mkonicek

This comment has been minimized.

Copy link
Contributor

commented Jul 27, 2016

In short I agree with @Kureev that making this "just work" is a nice developer experience, compared to having to figure out how to enable the feature.

@bestander

This comment has been minimized.

Copy link
Contributor

commented Jul 27, 2016

A concern that an extra walk through the node_modules folder slows down
packager start.
Any extra IO adds up especially in this case when it is doing "native" node
fs operations that are prone to throw ENOENT.

On Wed, Jul 27, 2016 at 1:05 PM, Martin Konicek notifications@github.com
wrote:

@matryoshcow https://github.com/matryoshcow If I have no symlinks in
node_modules, this feature is disabled. This way the feature is only
enabled for people who want to use it.

It might happen that people have symlinks in node_modules without knowing
about it though, which could be confusing. Printing info about the found
symlinks on packager startup could help.

Making this optional with a packager argument would work too, but people
would have to learn about the extra argument (Google -> StackOverflow etc.,
figuring out if info is up to date, extra work), and what if people launch
the packager using Xcode? There are several different ways people launch
the packager which makes it more work to make the feature super easy to use.


You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#9009 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACBdWA6BpFv9672eJzKu2-VmZWV09cEkks5qZ0mBgaJpZM4JUblt
.

@ghost ghost added the CLA Signed label Jul 27, 2016

@@ -74,6 +75,9 @@ function _server(argv, config, resolve, reject) {
? argToArray(args.projectRoots)
: config.getProjectRoots();

args.projectRoots = args.projectRoots
.concat(findSymlinksPaths(path.resolve(process.cwd(), 'node_modules')));

This comment has been minimized.

Copy link
@bestander

bestander Jul 27, 2016

Contributor

This means 750 sync lstat operations during packager start.

@ghost ghost added the CLA Signed label Jul 27, 2016

@matryoshcow

This comment has been minimized.

Copy link

commented Jul 27, 2016

@mkonicek hmm... agreed - making this "just work" is a good idea, as long as it doesn't slow down the startup time. Since we're only talking about node_modules here the overhead may be negligible even for us, but let's confirm this before we commit to patching it in.

@ghost ghost added the CLA Signed label Jul 27, 2016

@matryoshcow

This comment has been minimized.

Copy link

commented Jul 27, 2016

@Kureev let's add this under a flag to begin with and perhaps log the symlink resolution time; we can enable it by default once we've observed it for a bit.

@ghost ghost added the CLA Signed label Jul 27, 2016

@mkonicek

This comment has been minimized.

Copy link
Contributor

commented Jul 28, 2016

Right, readdirSync is not recursive so this should be fast but agree adding the feature behind a flag first is a good way to introduce it. Good idea with logging the time!

EDIT: Oh wait, internally at fb there are a lot of folders in node_modules (when starting the packager from the RN folder). Good point @bestander about this being 750 sync checks. With npm 2 I only have two folders in my app's node_modules: react and react-native, I was thinking of that use case :D

ghost pushed a commit that referenced this pull request Sep 15, 2016

Support symlinks-to-symlinks
Summary:
In response to [this comment](#9009 (comment)).

I could be wrong here, but I think Watchman can't handle symlinks, so we need to make sure symlinks-to-symlinks are handled up front.
Closes #9792

Differential Revision: D3858349

Pulled By: bestander

fbshipit-source-id: f3a34dae90ed9a7004a03158288db5e1932bfc69
@ryanlinnane

This comment has been minimized.

Copy link

commented Sep 20, 2016

What's the current status of this? I found that via using npm link, watchman broke because it referenced a symlink to a symlink. fs.realpathSync did the trick. Moreover, I found that the path construction sometimes was a directory off, so changing the base folder to lookupFolder fixed up a few issues.

Code:

module.exports = function findSymlinksPaths(lookupFolder) {
  const timeStart = Date.now();
  const folders = fs.readdirSync(lookupFolder);
  const resolvedSymlinks = folders.map(folder => path.resolve(lookupFolder, folder))
    .filter(folderPath => fs.lstatSync(folderPath).isSymbolicLink())
    .map(symlink => fs.realpathSync(path.resolve(lookupFolder, fs.readlinkSync(symlink))));
  const timeEnd = Date.now();
  console.log(`Scanning ${folders.length} folders for symlinks in ${lookupFolder} (${timeEnd - timeStart}ms)`);
  return resolvedSymlinks;
};
@aleclarson

This comment has been minimized.

Copy link
Contributor

commented Sep 20, 2016

@ryanlinnane Check out #9792 and let me know if you still have issues.

@cpojer

This comment has been minimized.

Copy link
Contributor

commented Oct 26, 2016

I agree we should figure out a path forward to support this both in RNP and in Jest. Give us some time to get our house in order please, as @bestander pointed out we are planning some pretty significant changes.

@cpojer cpojer self-assigned this Oct 26, 2016

facebook-github-bot added a commit that referenced this pull request Jan 31, 2017

Add symlinks under node_modules as part of projectRoots
Summary:
Support symlinks under `node_modules` for all local-cli commands. PR #9009 only adds symlink support to the packager.

But other cli commands like `react-native bundle` creates its own instance of packager that doesn't have symlinks as part of its project roots, which results in the bundler breaking since it cannot find modules that you have symlinked.

This change ensures all `local-cli` commands add symlinks to its project roots.

Test plan (required)

1.  Create a symlink in node_modules (for instance use npm/yarn link)
2. Run `react-native bundle`.
Closes #11810

Differential Revision: D4487741

fbshipit-source-id: 87fe44194134d086dca4eaca99ee5742d6eadb69

edmofro pushed a commit to edmofro/react-native that referenced this pull request Feb 6, 2017

Add symlinks under node_modules as part of projectRoots
Summary:
Support symlinks under `node_modules` for all local-cli commands. PR facebook#9009 only adds symlink support to the packager.

But other cli commands like `react-native bundle` creates its own instance of packager that doesn't have symlinks as part of its project roots, which results in the bundler breaking since it cannot find modules that you have symlinked.

This change ensures all `local-cli` commands add symlinks to its project roots.

Test plan (required)

1.  Create a symlink in node_modules (for instance use npm/yarn link)
2. Run `react-native bundle`.
Closes facebook#11810

Differential Revision: D4487741

fbshipit-source-id: 87fe44194134d086dca4eaca99ee5742d6eadb69

nicktate added a commit to nicktate/react-native that referenced this pull request Feb 7, 2017

Add symlinks under node_modules as part of projectRoots
Summary:
Support symlinks under `node_modules` for all local-cli commands. PR facebook#9009 only adds symlink support to the packager.

But other cli commands like `react-native bundle` creates its own instance of packager that doesn't have symlinks as part of its project roots, which results in the bundler breaking since it cannot find modules that you have symlinked.

This change ensures all `local-cli` commands add symlinks to its project roots.

Test plan (required)

1.  Create a symlink in node_modules (for instance use npm/yarn link)
2. Run `react-native bundle`.
Closes facebook#11810

Differential Revision: D4487741

fbshipit-source-id: 87fe44194134d086dca4eaca99ee5742d6eadb69

normanjoyner added a commit to nicktate/react-native that referenced this pull request Feb 9, 2017

Add symlinks under node_modules as part of projectRoots
Summary:
Support symlinks under `node_modules` for all local-cli commands. PR facebook#9009 only adds symlink support to the packager.

But other cli commands like `react-native bundle` creates its own instance of packager that doesn't have symlinks as part of its project roots, which results in the bundler breaking since it cannot find modules that you have symlinked.

This change ensures all `local-cli` commands add symlinks to its project roots.

Test plan (required)

1.  Create a symlink in node_modules (for instance use npm/yarn link)
2. Run `react-native bundle`.
Closes facebook#11810

Differential Revision: D4487741

fbshipit-source-id: 87fe44194134d086dca4eaca99ee5742d6eadb69

nicktate added a commit to nicktate/react-native that referenced this pull request Feb 9, 2017

Add symlinks under node_modules as part of projectRoots
Summary:
Support symlinks under `node_modules` for all local-cli commands. PR facebook#9009 only adds symlink support to the packager.

But other cli commands like `react-native bundle` creates its own instance of packager that doesn't have symlinks as part of its project roots, which results in the bundler breaking since it cannot find modules that you have symlinked.

This change ensures all `local-cli` commands add symlinks to its project roots.

Test plan (required)

1.  Create a symlink in node_modules (for instance use npm/yarn link)
2. Run `react-native bundle`.
Closes facebook#11810

Differential Revision: D4487741

fbshipit-source-id: 87fe44194134d086dca4eaca99ee5742d6eadb69

nicktate added a commit to nicktate/react-native that referenced this pull request Feb 9, 2017

Add symlinks under node_modules as part of projectRoots
Summary:
Support symlinks under `node_modules` for all local-cli commands. PR facebook#9009 only adds symlink support to the packager.

But other cli commands like `react-native bundle` creates its own instance of packager that doesn't have symlinks as part of its project roots, which results in the bundler breaking since it cannot find modules that you have symlinked.

This change ensures all `local-cli` commands add symlinks to its project roots.

Test plan (required)

1.  Create a symlink in node_modules (for instance use npm/yarn link)
2. Run `react-native bundle`.
Closes facebook#11810

Differential Revision: D4487741

fbshipit-source-id: 87fe44194134d086dca4eaca99ee5742d6eadb69

nicktate added a commit to nicktate/react-native that referenced this pull request Feb 9, 2017

Add symlinks under node_modules as part of projectRoots
Summary:
Support symlinks under `node_modules` for all local-cli commands. PR facebook#9009 only adds symlink support to the packager.

But other cli commands like `react-native bundle` creates its own instance of packager that doesn't have symlinks as part of its project roots, which results in the bundler breaking since it cannot find modules that you have symlinked.

This change ensures all `local-cli` commands add symlinks to its project roots.

Test plan (required)

1.  Create a symlink in node_modules (for instance use npm/yarn link)
2. Run `react-native bundle`.
Closes facebook#11810

Differential Revision: D4487741

fbshipit-source-id: 87fe44194134d086dca4eaca99ee5742d6eadb69

nicktate added a commit to nicktate/react-native that referenced this pull request Feb 9, 2017

Add symlinks under node_modules as part of projectRoots
Summary:
Support symlinks under `node_modules` for all local-cli commands. PR facebook#9009 only adds symlink support to the packager.

But other cli commands like `react-native bundle` creates its own instance of packager that doesn't have symlinks as part of its project roots, which results in the bundler breaking since it cannot find modules that you have symlinked.

This change ensures all `local-cli` commands add symlinks to its project roots.

Test plan (required)

1.  Create a symlink in node_modules (for instance use npm/yarn link)
2. Run `react-native bundle`.
Closes facebook#11810

Differential Revision: D4487741

fbshipit-source-id: 87fe44194134d086dca4eaca99ee5742d6eadb69

nicktate added a commit to nicktate/react-native that referenced this pull request Feb 9, 2017

Add symlinks under node_modules as part of projectRoots
Summary:
Support symlinks under `node_modules` for all local-cli commands. PR facebook#9009 only adds symlink support to the packager.

But other cli commands like `react-native bundle` creates its own instance of packager that doesn't have symlinks as part of its project roots, which results in the bundler breaking since it cannot find modules that you have symlinked.

This change ensures all `local-cli` commands add symlinks to its project roots.

Test plan (required)

1.  Create a symlink in node_modules (for instance use npm/yarn link)
2. Run `react-native bundle`.
Closes facebook#11810

Differential Revision: D4487741

fbshipit-source-id: 87fe44194134d086dca4eaca99ee5742d6eadb69

nicktate added a commit to nicktate/react-native that referenced this pull request Feb 9, 2017

Add symlinks under node_modules as part of projectRoots
Summary:
Support symlinks under `node_modules` for all local-cli commands. PR facebook#9009 only adds symlink support to the packager.

But other cli commands like `react-native bundle` creates its own instance of packager that doesn't have symlinks as part of its project roots, which results in the bundler breaking since it cannot find modules that you have symlinked.

This change ensures all `local-cli` commands add symlinks to its project roots.

Test plan (required)

1.  Create a symlink in node_modules (for instance use npm/yarn link)
2. Run `react-native bundle`.
Closes facebook#11810

Differential Revision: D4487741

fbshipit-source-id: 87fe44194134d086dca4eaca99ee5742d6eadb69

nicktate added a commit to nicktate/react-native that referenced this pull request Feb 9, 2017

Add symlinks under node_modules as part of projectRoots
Summary:
Support symlinks under `node_modules` for all local-cli commands. PR facebook#9009 only adds symlink support to the packager.

But other cli commands like `react-native bundle` creates its own instance of packager that doesn't have symlinks as part of its project roots, which results in the bundler breaking since it cannot find modules that you have symlinked.

This change ensures all `local-cli` commands add symlinks to its project roots.

Test plan (required)

1.  Create a symlink in node_modules (for instance use npm/yarn link)
2. Run `react-native bundle`.
Closes facebook#11810

Differential Revision: D4487741

fbshipit-source-id: 87fe44194134d086dca4eaca99ee5742d6eadb69

nicktate added a commit to nicktate/react-native that referenced this pull request Feb 9, 2017

Add symlinks under node_modules as part of projectRoots
Summary:
Support symlinks under `node_modules` for all local-cli commands. PR facebook#9009 only adds symlink support to the packager.

But other cli commands like `react-native bundle` creates its own instance of packager that doesn't have symlinks as part of its project roots, which results in the bundler breaking since it cannot find modules that you have symlinked.

This change ensures all `local-cli` commands add symlinks to its project roots.

Test plan (required)

1.  Create a symlink in node_modules (for instance use npm/yarn link)
2. Run `react-native bundle`.
Closes facebook#11810

Differential Revision: D4487741

fbshipit-source-id: 87fe44194134d086dca4eaca99ee5742d6eadb69

nicktate added a commit to nicktate/react-native that referenced this pull request Feb 9, 2017

Add symlinks under node_modules as part of projectRoots
Summary:
Support symlinks under `node_modules` for all local-cli commands. PR facebook#9009 only adds symlink support to the packager.

But other cli commands like `react-native bundle` creates its own instance of packager that doesn't have symlinks as part of its project roots, which results in the bundler breaking since it cannot find modules that you have symlinked.

This change ensures all `local-cli` commands add symlinks to its project roots.

Test plan (required)

1.  Create a symlink in node_modules (for instance use npm/yarn link)
2. Run `react-native bundle`.
Closes facebook#11810

Differential Revision: D4487741

fbshipit-source-id: 87fe44194134d086dca4eaca99ee5742d6eadb69

nicktate added a commit to nicktate/react-native that referenced this pull request Feb 9, 2017

Add symlinks under node_modules as part of projectRoots
Summary:
Support symlinks under `node_modules` for all local-cli commands. PR facebook#9009 only adds symlink support to the packager.

But other cli commands like `react-native bundle` creates its own instance of packager that doesn't have symlinks as part of its project roots, which results in the bundler breaking since it cannot find modules that you have symlinked.

This change ensures all `local-cli` commands add symlinks to its project roots.

Test plan (required)

1.  Create a symlink in node_modules (for instance use npm/yarn link)
2. Run `react-native bundle`.
Closes facebook#11810

Differential Revision: D4487741

fbshipit-source-id: 87fe44194134d086dca4eaca99ee5742d6eadb69

nicktate added a commit to nicktate/react-native that referenced this pull request Feb 9, 2017

Add symlinks under node_modules as part of projectRoots
Summary:
Support symlinks under `node_modules` for all local-cli commands. PR facebook#9009 only adds symlink support to the packager.

But other cli commands like `react-native bundle` creates its own instance of packager that doesn't have symlinks as part of its project roots, which results in the bundler breaking since it cannot find modules that you have symlinked.

This change ensures all `local-cli` commands add symlinks to its project roots.

Test plan (required)

1.  Create a symlink in node_modules (for instance use npm/yarn link)
2. Run `react-native bundle`.
Closes facebook#11810

Differential Revision: D4487741

fbshipit-source-id: 87fe44194134d086dca4eaca99ee5742d6eadb69

nicktate added a commit to nicktate/react-native that referenced this pull request Feb 9, 2017

Add symlinks under node_modules as part of projectRoots
Summary:
Support symlinks under `node_modules` for all local-cli commands. PR facebook#9009 only adds symlink support to the packager.

But other cli commands like `react-native bundle` creates its own instance of packager that doesn't have symlinks as part of its project roots, which results in the bundler breaking since it cannot find modules that you have symlinked.

This change ensures all `local-cli` commands add symlinks to its project roots.

Test plan (required)

1.  Create a symlink in node_modules (for instance use npm/yarn link)
2. Run `react-native bundle`.
Closes facebook#11810

Differential Revision: D4487741

fbshipit-source-id: 87fe44194134d086dca4eaca99ee5742d6eadb69

nicktate added a commit to nicktate/react-native that referenced this pull request Feb 9, 2017

Add symlinks under node_modules as part of projectRoots
Summary:
Support symlinks under `node_modules` for all local-cli commands. PR facebook#9009 only adds symlink support to the packager.

But other cli commands like `react-native bundle` creates its own instance of packager that doesn't have symlinks as part of its project roots, which results in the bundler breaking since it cannot find modules that you have symlinked.

This change ensures all `local-cli` commands add symlinks to its project roots.

Test plan (required)

1.  Create a symlink in node_modules (for instance use npm/yarn link)
2. Run `react-native bundle`.
Closes facebook#11810

Differential Revision: D4487741

fbshipit-source-id: 87fe44194134d086dca4eaca99ee5742d6eadb69

nicktate added a commit to nicktate/react-native that referenced this pull request Feb 9, 2017

Add symlinks under node_modules as part of projectRoots
Summary:
Support symlinks under `node_modules` for all local-cli commands. PR facebook#9009 only adds symlink support to the packager.

But other cli commands like `react-native bundle` creates its own instance of packager that doesn't have symlinks as part of its project roots, which results in the bundler breaking since it cannot find modules that you have symlinked.

This change ensures all `local-cli` commands add symlinks to its project roots.

Test plan (required)

1.  Create a symlink in node_modules (for instance use npm/yarn link)
2. Run `react-native bundle`.
Closes facebook#11810

Differential Revision: D4487741

fbshipit-source-id: 87fe44194134d086dca4eaca99ee5742d6eadb69

grabbou pushed a commit to react-native-community/cli that referenced this pull request Sep 26, 2018

Kureev Alexey Facebook Github Bot 9
Symlink support for packager
Summary:
I saw we have quite a few user requests for symlink support...

**Test plan (required)**

1. Create a symlink in `node_modules` (for instance use `npm link`)
2. Run `npm start`
3. Profit!

**Code formatting**

Look around. Match the style of the rest of the codebase. See also the simple [style guide](https://github.com/facebook/react-native/blob/master/CONTRIBUTING.md#style-guide).

For more info, see the ["Pull Requests" section of our "Contributing" guidelines](https://github.com/facebook/react-native/blob/master/CONTRIBUTING.md#pull-requests).
Closes facebook/react-native#9009

Differential Revision: D3648828

Pulled By: matryoshcow

fbshipit-source-id: 99cf313bfa70324ca904fa6919ef112180974e9e

grabbou pushed a commit to react-native-community/cli that referenced this pull request Sep 26, 2018

Support symlinks-to-symlinks
Summary:
In response to [this comment](facebook/react-native#9009 (comment)).

I could be wrong here, but I think Watchman can't handle symlinks, so we need to make sure symlinks-to-symlinks are handled up front.
Closes facebook/react-native#9792

Differential Revision: D3858349

Pulled By: bestander

fbshipit-source-id: f3a34dae90ed9a7004a03158288db5e1932bfc69

grabbou pushed a commit to react-native-community/cli that referenced this pull request Sep 26, 2018

Add symlinks under node_modules as part of projectRoots
Summary:
Support symlinks under `node_modules` for all local-cli commands. PR facebook/react-native#9009 only adds symlink support to the packager.

But other cli commands like `react-native bundle` creates its own instance of packager that doesn't have symlinks as part of its project roots, which results in the bundler breaking since it cannot find modules that you have symlinked.

This change ensures all `local-cli` commands add symlinks to its project roots.

Test plan (required)

1.  Create a symlink in node_modules (for instance use npm/yarn link)
2. Run `react-native bundle`.
Closes facebook/react-native#11810

Differential Revision: D4487741

fbshipit-source-id: 87fe44194134d086dca4eaca99ee5742d6eadb69
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
10 participants
You can’t perform that action at this time.