unpack will fail if there is anywhere in the parent path of bitbake a node_modules folder

[ephraim]

Guess bitbake is install under /home/user/Projects/bitbake and /home/user/node_modules is existing.

npm-fetch will download the module to /home/user/node_modules rather than to sources/downloads/npm/.....

The download will succeed but the unpack will fail then, as the folder below sources/downloads/npm/... is empty.

[imyller]

Thank you for reporting this issue.

I can not duplicate it in my setup, so could you please provide some log outputs?

To my knowledge npm should always prefer the current directory it is run from as a target for node_modules (unless --global parameters is used). The bitbake-npm expects this. If this is not the case I will look into options to forcing npm install path.

[ephraim]

I removed some exports at the Running output. Basically export of ssh auth sock, proxies, home path and path variable.

log.do_fetch:
DEBUG: Executing python function do_fetch
DEBUG: Executing python function base_do_fetch
DEBUG: Trying PREMIRRORS
DEBUG: For url ['npm', '', 'bcryptjs', '', '', {'version': '2.1.0'}] comparing ['cvs', '.', '/.', '', '', {}] to ['http', 'sources.angstrom-distribution.org', '/v2013.06/', '', '', {}]
DEBUG: For url ['npm', '', 'bcryptjs', '', '', {'version': '2.1.0'}] comparing ['svn', '.', '/.', '', '', {}] to ['http', 'sources.angstrom-distribution.org', '/v2013.06/', '', '', {}]
DEBUG: For url ['npm', '', 'bcryptjs', '', '', {'version': '2.1.0'}] comparing ['git', '.', '/.', '', '', {}] to ['http', 'sources.angstrom-distribution.org', '/v2013.06/', '', '', {}]
DEBUG: For url ['npm', '', 'bcryptjs', '', '', {'version': '2.1.0'}] comparing ['hg', '.', '/.', '', '', {}] to ['http', 'sources.angstrom-distribution.org', '/v2013.06/', '', '', {}]
DEBUG: For url ['npm', '', 'bcryptjs', '', '', {'version': '2.1.0'}] comparing ['bzr', '.', '/.', '', '', {}] to ['http', 'sources.angstrom-distribution.org', '/v2013.06/', '', '', {}]
DEBUG: For url ['npm', '', 'bcryptjs', '', '', {'version': '2.1.0'}] comparing ['osc', '.', '/.', '', '', {}] to ['http', 'sources.angstrom-distribution.org', '/v2013.06/', '', '', {}]
DEBUG: For url ['npm', '', 'bcryptjs', '', '', {'version': '2.1.0'}] comparing ['p4', '.', '/.', '', '', {}] to ['http', 'sources.angstrom-distribution.org', '/v2013.06/', '', '', {}]
DEBUG: For url ['npm', '', 'bcryptjs', '', '', {'version': '2.1.0'}] comparing ['svk', '.', '/.', '', '', {}] to ['http', 'sources.angstrom-distribution.org', '/v2013.06/', '', '', {}]
DEBUG: Trying Upstream
NOTE: npm install npm://bcryptjs;version=2.1.0
DEBUG: executing npm --arch=arm --target_arch=arm install bcryptjs@2.1.0 --force
DEBUG: Fetcher accessed the network with the command npm --arch=arm --target_arch=arm install bcryptjs@2.1.0 --force
DEBUG: Running REMOVEDEXPORTS; npm --arch=arm --target_arch=arm install bcryptjs@2.1.0 --force
DEBUG: Python function base_do_fetch finished
DEBUG: Python function do_fetch finished

log.do_unpack:
DEBUG: Executing python function do_unpack
DEBUG: Executing python function base_do_unpack
DEBUG: Running REMOVEDEXPORTS; cp -r /home/cestonarot/Projects/magna/setup-scripts/sources/downloads/npm/bcryptjs_2.1.0/node_modules/bcryptjs /home/cestonarot/Projects/magna/setup-scripts/build/tmp-magna_2013_x-eglibc/work/armv7a-magna-linux-gnueabi/bcryptjs/2.1.0-r0/npm//
DEBUG: Python function base_do_unpack finished
DEBUG: Python function do_unpack finished
ERROR: Function failed: Fetcher failure: Fetch command failed with exit code 1, output:
cp: cannot stat '/home/cestonarot/Projects/magna/setup-scripts/sources/downloads/npm/bcryptjs_2.1.0/node_modules/bcryptjs': No such file or directory

[ephraim]

I did no special settings or so. :(

for bcryptjs I used this recipe stuff:

bcryptjs_2.1.0.bb

SUMMARY = "Optimized bcrypt in plain JavaScript with zero dependencies. Compatible to 'bcrypt'."
SECTION = "nodejs/module"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://LICENSE;md5=d99a4f4253fea793a1256aef626aa6e5"

PR = "r0"

require node.inc

---

node.inc

NODEVERSION ?= "${PV}"
NODENAME ?= "${PN}"
SRC_URI = "npm://${NODENAME};version=${NODEVERSION}"
S = "${WORKDIR}/npm"

inherit npm-install npm-fetch

do_install() {
install -d ${D}${libdir}/node_modules/${NODENAME}
for i in $(find ${S} -type f -a ( -name '.json' -or -name '.js' -or -name '.html' -or -name '.node' -or -name '.png' -or -name '.conf' -or -name '*.css' -or -executable ) | grep -v test) ; do
rel=$(echo "$i" | sed "s#${S}/##")
if [ ! -d ${D}${libdir}/node_modules/${NODENAME}/$(dirname $rel) ] ; then
mkdir -p ${D}${libdir}/node_modules/${NODENAME}/$(dirname $rel)
fi
cp ${S}/$rel ${D}${libdir}/node_modules/${NODENAME}/$(dirname $rel)
done
}

FILES_${PN} += "${libdir}/node_modules"

[imyller]

Thanks for the details. Looks like unpack works as expected, but somehow npm does not install to the current ./node_modules after changing the directory to /downloads/npm/....

Other explanation is that fetch somehow does not actually change to the correct directory under sources/downloads. I think I'll have to add some debug logging to display the current path before running npm install.

[ephraim]

when I create an empty node_modules folder in my home before running bitbake, the bcryptjs stuff goes into this folder when running bitbake bcryptjs -c fetch .... so I guess npm is not correct.

[imyller]

Just checked npm folder docs at https://docs.npmjs.com/files/folders

It says that all local install should go to ./node_modules no matter what. Obviously this is not the case here.

Have to look at the npm source to see if there is some special handling for node_modules at the homedir.

[imyller]

What is the npm version you are using npm -v .. or are you running with bitbake compiled nodejs-native or iojs-native in native tools rootfs. If not it falls back to your local npm installation.

[ephraim]

nodejs-native was built, so I guess the bitbake's npm is used which has the version 1.4.28.

Can I somehow figure out which npm bitbake-npm is using?

[imyller]

I'll have to add some debug logging for npm install phase. Just testing various configurations myself.

Any reason why you are not running more recent node/npm? See https://github.com/imyller/meta-nodejs for recipe layer with latest node.js releases or https://github.com/imyller/meta-iojs for io.js.

[ephraim]

yeah ... v8 c++ gets much more complex then :)

[imyller]

Use nan to standardize with C++ ABI :)

[ephraim]

thanks for the hint .... will have a look :)

[imyller]

Can you please test with changes made in master or 1.26 branches.

[ephraim]

Sorry for the delay! Fetch is working now. Thanks for fixing it!

[imyller]

Good to hear. Thank you for reporting and testing. I'm closing this issue as resolved.