New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable NPM as package manager for NativeScript projects #362

Closed
ligaz opened this Issue Mar 21, 2015 · 25 comments

Comments

Projects
None yet
@ligaz
Copy link

ligaz commented Mar 21, 2015

A common use case for NativeScript developers is the ability to install pure JavaScript modules from NPM. Currently this process is cumbersome and it is not straightforward how to require modules that have dependencies due to the nesting of node_modules folders. The end goal is that npm install ... should work out of the box.

Relying on 'standards'

In order to support this a couple of things have to be changed in the CLI. Currently we create a project file .tnsproject that stores the app id (bundle identifier) along with the currently used version of each runtime. We plan to replace this file with package.json that will be compatible with NPM package json format. Here is the structure of a simple app following this rule:

my-app/
|-- app/
|-- |-- App_Resources/
|-- |-- tns_modules/
|-- |-- ...
|-- platforms/
|-- node_modules/
`-- package.json

Preparing for a better world

Assuming we have package.json in the root of the project will allow us to support npm install lodash --save which will install lodash into node_modules folder and save it as dependency in the package.json. All good but how lodash will end-up in the tns_modules folder of the app? This is where the second significant change will come into place. We will extend the prepare command to recursively transform the node_modules folder contents into tns_modules in the native platform project. This transformation should exclude all modules defined in the devDependencies section of the package.json.

Open questions:

What should be the way to add NativeScript specific properties to the project.json?

  • prefixed with tns_ or nativescript_ Example:
{
  "name": "myapp",
  "version": "10.3.1",
  "tns_id": "org.nativescript.myapp",
  "tns_platforms": {
    "ios" : "0.9.2",
    "android" : "0.9.0"
  }
}
  • in a designated nativescript key. Example:
{
  "name": "myapp",
  "version": "10.3.1",
  "nativescript": {
    "id" : "org.nativescript.myapp",
    "platforms": {
        "ios" : "0.9.2",
        "android" : "0.9.0"
    }
  }
}

Should we require NativeScript tailored modules published to NPM to have nativescript as an engine?

Example:

{ "engines" : { "nativescript" : ">=0.9 <0.10" } }

or

{  
  "engines" : { 
    "nativescript-android" : "0.9.0",
    "nativescript-ios" : "0.9.3" 
  }   
}

This will give us a couple of advantages:

  • We can warn (or error) the user if the module is not compatible with currently used runtime versions.
  • (Future) We can implement a Node compatibility layer in the runtimes that will be able to load Node dependent modules from node_modules folder by providing shims for all Node APIs.

Tasks

  • Remove .tnsproject and introduce package.json
  • Move project properties into nativescript key in the package.json
  • Extend prepare to copy the modules from node_modules into platform native project 'tns_modules'
    • Flatten nested dependencies
    • Do not prepare devDependencies.
    • Make sure we do not degrade the performance during prepare when copying the modules. We can consider things like symlinks or broccoli.js kind of infrastructure.
  • Guard the new code with tests
  • Implement an upgrade procedure for the old projects
@ErjanGavalji

This comment has been minimized.

Copy link
Contributor

ErjanGavalji commented Mar 21, 2015

Kudos!

Regarding the nativescript-specific properties, a separate key looks like the cleanest solution.

As per having nativescript "engine" prerequisite, though cumbersome in the beginning, it is the cleanest approach when having to avoid unsupported nodejs modules. Shim creation could get easy by using the original NodeJS module as a git submodule of the shim repo.

@ErjanGavalji

This comment has been minimized.

Copy link
Contributor

ErjanGavalji commented Mar 21, 2015

Additionally, the root node_modules folder might hold preprocessing-related packages (i.e. grunt, gulp, plugins for them, etc.) that should not get copied to the NativeScript application folder...

@ligaz

This comment has been minimized.

Copy link

ligaz commented Mar 21, 2015

☝️ Good point about devDependencies I will update the issue.

@tjvantoll

This comment has been minimized.

Copy link
Contributor

tjvantoll commented Mar 21, 2015

First of all, this look spectacular. I'm a big fan of this setup.

What should be the way to add NativeScript specific properties to the project.json?

I'm also a fan of the single “nativescript” key.

Should we require NativeScript tailored modules published to NPM to have nativescript as an engine?

This will be a nuisance for maintenance but I still think it's a good idea. We might want to make it a warning rather than a hard requirement. I believe that's what Cordova does for their <engines>.

Out of curiosity is the plan to have the cross-platform modules follow these same conventions? As we add modules that require certain permissions (e.g. the camera) I could see dog fooding these same ideas being pretty handy.

I've already had to manually upgrade tns_modules in my apps several times and it's a nuisance at the moment. It'd be great if I could just depend on “nativescript-core” (or whatever) in my package.json and use that to update my cross-platform-modules files. Core modules that have certain permission or other config requirements could become their own npm modules—“nativescript-camera”, “nativescript-map”, etc.

@RangerMauve

This comment has been minimized.

Copy link

RangerMauve commented Mar 25, 2015

I +1 the idea of having a property called nativescript in the package.json. The alternative looks a lot more messy.

Also, publishing modules to npm will definately increase the speed at which people adopt Nativescript since it'd be trivial to publish re-usable packages that use native code.

@JpCapdevila

This comment has been minimized.

Copy link

JpCapdevila commented Mar 26, 2015

I think that supporting NPM is a must. People is already writing cross-platform modules (I am), and publishing to NPM will accelerate the usage of Nativescript.

@tjvantoll idea of having a "nativescript-core" module that is upgradeable via NPM sounds like heaven.

@burkeholland

This comment has been minimized.

Copy link

burkeholland commented Apr 1, 2015

+1 to being able to take a dependency on the cross-platform-modules. Makes for an easy upgrade path.

@gngeorgiev

This comment has been minimized.

Copy link

gngeorgiev commented May 4, 2015

👍

@RangerMauve

This comment has been minimized.

Copy link

RangerMauve commented May 6, 2015

Has there been any progress on this?

@ligaz

This comment has been minimized.

Copy link

ligaz commented May 8, 2015

@RangerMauve Yes we are currently working on this. You can check the tasks we have defined in the issue description.

@valentinstoychev

This comment has been minimized.

Copy link

valentinstoychev commented May 20, 2015

This issue is related to Node.js API support feature.

@isiahmeadows

This comment has been minimized.

Copy link

isiahmeadows commented Jun 28, 2015

Mind adding Node.js/io.js API support to the checklist? With same-named modules having different APIs, and the same module system being used, this will inevitably trip up a lot of people in a hurry.

@moll

This comment has been minimized.

Copy link

moll commented Jul 30, 2015

Shouldn't the package.json be inside the app folder as the rest of app-specific code is there? Currently I believe there are two package.jsons for some reason. Or, not require the app folder at all and let things be at the root.

@moll

This comment has been minimized.

Copy link

moll commented Jul 30, 2015

Having thought about it for a little, I realized preparing JavaScript for a mobile app is very much like preparing it for the browser. You'd want to ensure the minimal amount of code gets included, process it if you're doing something clever and perhaps even minify it. Assuming the root node_modules is wanted in app would interfere with such an approach. As a developer, you'd instead consider the current app folder to be your build output folder.

@RangerMauve

This comment has been minimized.

Copy link

RangerMauve commented Jul 30, 2015

I totally agree with what @moll is saying. Maybe it'd be useful to integrate something like browserify or webpack for building up the dependencies into a single JS file in the build step.

@tjvantoll

This comment has been minimized.

Copy link
Contributor

tjvantoll commented Jul 30, 2015

@moll Just as an fyi I've requested that the package.json within app be removed in #577. Currently it's only needed to specify the "main", and personally I think it should go away. Here's an example of a minimal app/package.json.

@moll

This comment has been minimized.

Copy link

moll commented Jul 31, 2015

@tjvantoll: Great. Thanks.

Wow, just switched to using Browserify as a preprocessor, and after running tns build ios, I see not only my dependencies in platforms/ios/$app_name/app/tns_modules, but all of Browserify et alii. Any quick way to stop prevent that now? Modules contain a lot of superfluous things not suitable for shipping and the flattening breaks modules depending on the same package with different versions.

@tjvantoll

This comment has been minimized.

Copy link
Contributor

tjvantoll commented Jul 31, 2015

Make sure you install Browserify with --save-dev so that it's saved as a "devDependency" and not a "dependency". The NativeScript CLI is smart enough to exclude devDependency modules, and those modules' dependencies.

For the superfluous things I think that's just something you might have to take up with the individual modules; they really should be using an .npmignore file to exclude any non-executable files. I do agree that it would be nice if the NativeScript CLI exposed some sort of hook so you had the ability to manually manipulate files on their way from app and node_modules into platforms/.... The CLI might already have some mechanism for doing that; @ligaz would know.

@moll

This comment has been minimized.

Copy link

moll commented Jul 31, 2015

Yep, used devDeps there and still had them copied to tns_modules.

I am one of those module authors who intentionally puts superfluous things in their release tarball. Things like tests and documentation. Vital for development, but definitely not meant to be sent down browsers or up mobile apps. :)

I don't think copying node_modules is necessarily a path NativeScript CLI should go down on. Perhaps for development it'd be nice if one could symlink the whole app dir to platforms' app dir and have things resolve through node_modules, but for any production release I'd say preprocessing JS is the way to go. It's a magnitude faster than using tns prepare, allows doing all kinds of funky things we already do for browsers and, well, takes care of minimal builds by following requires and not some human edited ignore files.

@tjvantoll

This comment has been minimized.

Copy link
Contributor

tjvantoll commented Jul 31, 2015

I tend to agree; you have good points. I'm curious what @ligaz's thoughts on this are.

@ligaz

This comment has been minimized.

Copy link

ligaz commented Aug 1, 2015

We do plan to provide hooks as extensibility point (they are already there in the common lib, but not enabled in NativeScript CLI) at some near future as part of our transpilers support #493.

Currently we prepare a couple of things:

  • Copy the app folder to the respective platform
  • Copy and process native resource files (App_Resources)
  • Smart copy node_modules into platform's tns_modules.
  • Rename platform specific files to strip the platform from the file name (*.ios.js, *.android.css, etc.)

We will think about how we can abstract our current default prepare functionality to a hook that can be tweaked.

@rclai

This comment has been minimized.

Copy link

rclai commented Nov 16, 2015

Hey guys, I'm using the Telerik platform (browser-based), and I used the GUI to install an NPM package, this one, and I could not require it. Since I can't npm install one folder down to require it absolutely inside the Telerik platform GUI, what can I do?

@Gertus88

This comment has been minimized.

Copy link

Gertus88 commented Nov 19, 2015

Hey all! I'm using Telerik AppBuilder, trying to import nativescript-background-http module in my app and get error Module "nativescript-background-http" not found. When i'm looking for it in Properties => Dependencies, it is present in installed NPM Packages. Can i make smth with it?

@didiepetrova

This comment has been minimized.

Copy link

didiepetrova commented Nov 19, 2015

Hey Gertus88,
Would you please submit a support case under product category AppBuilder through the ticketing system here and provide more details on the case including the exact steps you follow? It will be very helpful if you can capture a small video illustrating the step-by-step actions you perform at you end, so that we can verify the same locally.

We will be glad to assist with resolving the issue once we can reproduce it.

@augmen

This comment has been minimized.

Copy link

augmen commented Jun 13, 2017

I guess am using telrik platform . The platform already configured the nom packages into nod_module. do i need to any config to run the native script apps using NPM packages ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment