Enable NPM as package manager for NativeScript projects

[ligaz]

A common use case for NativeScript developers is the ability to install pure JavaScript modules from NPM. Currently this process is cumbersome and it is not straightforward how to require modules that have dependencies due to the nesting of node_modules folders. The end goal is that npm install ... should work out of the box.

Relying on 'standards'

In order to support this a couple of things have to be changed in the CLI. Currently we create a project file .tnsproject that stores the app id (bundle identifier) along with the currently used version of each runtime. We plan to replace this file with package.json that will be compatible with NPM package json format. Here is the structure of a simple app following this rule:

my-app/
|-- app/
|-- |-- App_Resources/
|-- |-- tns_modules/
|-- |-- ...
|-- platforms/
|-- node_modules/
`-- package.json

Preparing for a better world

Assuming we have package.json in the root of the project will allow us to support npm install lodash --save which will install lodash into node_modules folder and save it as dependency in the package.json. All good but how lodash will end-up in the tns_modules folder of the app? This is where the second significant change will come into place. We will extend the prepare command to recursively transform the node_modules folder contents into tns_modules in the native platform project. This transformation should exclude all modules defined in the devDependencies section of the package.json.

Open questions:

What should be the way to add NativeScript specific properties to the project.json?

• prefixed with tns_ or nativescript_ Example:

{
  "name": "myapp",
  "version": "10.3.1",
  "tns_id": "org.nativescript.myapp",
  "tns_platforms": {
    "ios" : "0.9.2",
    "android" : "0.9.0"
  }
}

• in a designated nativescript key. Example:

{
  "name": "myapp",
  "version": "10.3.1",
  "nativescript": {
    "id" : "org.nativescript.myapp",
    "platforms": {
        "ios" : "0.9.2",
        "android" : "0.9.0"
    }
  }
}

Should we require NativeScript tailored modules published to NPM to have nativescript as an engine?

Example:

{ "engines" : { "nativescript" : ">=0.9 <0.10" } }

or

{  
  "engines" : { 
    "nativescript-android" : "0.9.0",
    "nativescript-ios" : "0.9.3" 
  }   
}

This will give us a couple of advantages:

• We can warn (or error) the user if the module is not compatible with currently used runtime versions.
• (Future) We can implement a Node compatibility layer in the runtimes that will be able to load Node dependent modules from node_modules folder by providing shims for all Node APIs.

---

Tasks

• Remove .tnsproject and introduce package.json
• Move project properties into nativescript key in the package.json
• Extend prepare to copy the modules from node_modules into platform native project 'tns_modules'
  • Flatten nested dependencies
  • Do not prepare devDependencies.
  • Make sure we do not degrade the performance during prepare when copying the modules. We can consider things like symlinks or broccoli.js kind of infrastructure.
• Guard the new code with tests
• Implement an upgrade procedure for the old projects

[ErjanGavalji]

Kudos!

Regarding the nativescript-specific properties, a separate key looks like the cleanest solution.

As per having nativescript "engine" prerequisite, though cumbersome in the beginning, it is the cleanest approach when having to avoid unsupported nodejs modules. Shim creation could get easy by using the original NodeJS module as a git submodule of the shim repo.

[ErjanGavalji]

Additionally, the root node_modules folder might hold preprocessing-related packages (i.e. grunt, gulp, plugins for them, etc.) that should not get copied to the NativeScript application folder...

[ligaz]

☝️ Good point about devDependencies I will update the issue.

[tjvantoll]

First of all, this look spectacular. I'm a big fan of this setup.

What should be the way to add NativeScript specific properties to the project.json?

I'm also a fan of the single “nativescript” key.

Should we require NativeScript tailored modules published to NPM to have nativescript as an engine?

This will be a nuisance for maintenance but I still think it's a good idea. We might want to make it a warning rather than a hard requirement. I believe that's what Cordova does for their <engines>.

Out of curiosity is the plan to have the cross-platform modules follow these same conventions? As we add modules that require certain permissions (e.g. the camera) I could see dog fooding these same ideas being pretty handy.

I've already had to manually upgrade tns_modules in my apps several times and it's a nuisance at the moment. It'd be great if I could just depend on “nativescript-core” (or whatever) in my package.json and use that to update my cross-platform-modules files. Core modules that have certain permission or other config requirements could become their own npm modules—“nativescript-camera”, “nativescript-map”, etc.

[RangerMauve]

I +1 the idea of having a property called nativescript in the package.json. The alternative looks a lot more messy.

Also, publishing modules to npm will definately increase the speed at which people adopt Nativescript since it'd be trivial to publish re-usable packages that use native code.

[JpCapdevila]

I think that supporting NPM is a must. People is already writing cross-platform modules (I am), and publishing to NPM will accelerate the usage of Nativescript.

@tjvantoll idea of having a "nativescript-core" module that is upgradeable via NPM sounds like heaven.

[burkeholland]

+1 to being able to take a dependency on the cross-platform-modules. Makes for an easy upgrade path.

[gngeorgiev]

👍

[RangerMauve]

Has there been any progress on this?

[ligaz]

@RangerMauve Yes we are currently working on this. You can check the tasks we have defined in the issue description.

[valentinstoychev]

This issue is related to Node.js API support feature.

[dead-claudia]

Mind adding Node.js/io.js API support to the checklist? With same-named modules having different APIs, and the same module system being used, this will inevitably trip up a lot of people in a hurry.

[moll]

Shouldn't the package.json be inside the app folder as the rest of app-specific code is there? Currently I believe there are two package.jsons for some reason. Or, not require the app folder at all and let things be at the root.

[moll]

Having thought about it for a little, I realized preparing JavaScript for a mobile app is very much like preparing it for the browser. You'd want to ensure the minimal amount of code gets included, process it if you're doing something clever and perhaps even minify it. Assuming the root node_modules is wanted in app would interfere with such an approach. As a developer, you'd instead consider the current app folder to be your build output folder.

[RangerMauve]

I totally agree with what @moll is saying. Maybe it'd be useful to integrate something like browserify or webpack for building up the dependencies into a single JS file in the build step.

[tjvantoll]

@moll Just as an fyi I've requested that the package.json within app be removed in #577. Currently it's only needed to specify the "main", and personally I think it should go away. Here's an example of a minimal app/package.json.

[moll]

@tjvantoll: Great. Thanks.

Wow, just switched to using Browserify as a preprocessor, and after running tns build ios, I see not only my dependencies in platforms/ios/$app_name/app/tns_modules, but all of Browserify et alii. Any quick way to stop prevent that now? Modules contain a lot of superfluous things not suitable for shipping and the flattening breaks modules depending on the same package with different versions.

[tjvantoll]

Make sure you install Browserify with --save-dev so that it's saved as a "devDependency" and not a "dependency". The NativeScript CLI is smart enough to exclude devDependency modules, and those modules' dependencies.

For the superfluous things I think that's just something you might have to take up with the individual modules; they really should be using an .npmignore file to exclude any non-executable files. I do agree that it would be nice if the NativeScript CLI exposed some sort of hook so you had the ability to manually manipulate files on their way from app and node_modules into platforms/.... The CLI might already have some mechanism for doing that; @ligaz would know.

[moll]

Yep, used devDeps there and still had them copied to tns_modules.

I am one of those module authors who intentionally puts superfluous things in their release tarball. Things like tests and documentation. Vital for development, but definitely not meant to be sent down browsers or up mobile apps. :)

I don't think copying node_modules is necessarily a path NativeScript CLI should go down on. Perhaps for development it'd be nice if one could symlink the whole app dir to platforms' app dir and have things resolve through node_modules, but for any production release I'd say preprocessing JS is the way to go. It's a magnitude faster than using tns prepare, allows doing all kinds of funky things we already do for browsers and, well, takes care of minimal builds by following requires and not some human edited ignore files.

[tjvantoll]

I tend to agree; you have good points. I'm curious what @ligaz's thoughts on this are.

[ligaz]

We do plan to provide hooks as extensibility point (they are already there in the common lib, but not enabled in NativeScript CLI) at some near future as part of our transpilers support #493.

Currently we prepare a couple of things:

• Copy the app folder to the respective platform
• Copy and process native resource files (App_Resources)
• Smart copy node_modules into platform's tns_modules.
• Rename platform specific files to strip the platform from the file name (*.ios.js, *.android.css, etc.)

We will think about how we can abstract our current default prepare functionality to a hook that can be tweaked.

[rclai]

Hey guys, I'm using the Telerik platform (browser-based), and I used the GUI to install an NPM package, this one, and I could not require it. Since I can't npm install one folder down to require it absolutely inside the Telerik platform GUI, what can I do?

[Vitaly-Mashkov]

Hey all! I'm using Telerik AppBuilder, trying to import nativescript-background-http module in my app and get error Module "nativescript-background-http" not found. When i'm looking for it in Properties => Dependencies, it is present in installed NPM Packages. Can i make smth with it?

[didiepetrova]

Hey Gertus88,
Would you please submit a support case under product category AppBuilder through the ticketing system here and provide more details on the case including the exact steps you follow? It will be very helpful if you can capture a small video illustrating the step-by-step actions you perform at you end, so that we can verify the same locally.

We will be glad to assist with resolving the issue once we can reproduce it.

[augmen]

I guess am using telrik platform . The platform already configured the nom packages into nod_module. do i need to any config to run the native script apps using NPM packages ?