Node-RED addon package installer appears to not handle updates to Git URL dependencies

[jpravetz]

Problem/Motivation

Difficulty installing and updating to specific or latest versions of a package, using Git URLs

Expected behavior

Adding a gitURL that specifies a tag or version should cause the package to be updated to that version. Or there should be a way for users to direct the add-on to get the latest version of a package.

Actual behavior

Package remains at a previous install level. package.lock.json continues to refer to the older release.

Steps to reproduce

After already having a older version of the below example package installed, for example by adding the following to your add-on config:

npm_packages:
  - https://github.com/jpravetz/hassio-node-red-utils.git#v0.8.0

Update the reference to a later version of the package:

npm_packages:
  - https://github.com/jpravetz/hassio-node-red-utils.git#v0.8.1 

Proposed changes

The package should update to the specified version.
Even without #0.8.1 appended, the installer should get the latest version,
or we should have a way to specify that we need the latest version installed, rather than having to specify tags.

Baring the ability to have the add-on manage the full life-cycle of packages, we should have direct access to run yarn
to install, update, uninstall and manage this life-cycle on our own. On my ODroid N+ with Home Assistant pre-installed, I can't find yarn installed anywhere, but yet it must be there? So must be npm and node. I have a fear that if I install these again, I will break something.

Also: On an independently installed instance of Node-RED (e.g. on my Mac, not tied to Home Assistant), I am able to manage the life cycle of my package dependencies using npm. In other words, this limitation is unique to the Home Assistant add-on where we must go thru the config in order to add packages.

[sinclairpaul]

Briefly looking at the npm documentation, # refers to a branch or commit id, where as @ refers to a release/tag

You can find the code for npm install at

addon-node-red/node-red/rootfs/etc/s6-overlay/s6-rc.d/init-customizations/run

Lines 22 to 35 in 48ed99d

	if bashio::config.has_value 'npm_packages'; then
	cd /opt || bashio::exit.nok "Could not change directory to Node-RED"
	bashio::log.info "Starting installation of custom NPM/Node-RED packages..."
	for package in $(bashio::config 'npm_packages'); do
	npmlist+=("$package")
	done
	npm install \
	--no-optional \
	--only=production \
	"${npmlist[@]}" \
	|| bashio::exit.nok "Failed to install a specified npm package"
	fi

but in all honesty if you want this level of customisation I don't think running the addon is for you and you may be better running it elsewhere.

[tjorim]

Actually I think it should be installing a specific version using semantic versioning.

To install a package version based on a specific tag or tag range, you can specify a semver expression as follows:

$ npm install <url>#semver:<semver>

Which will make npm look for any tags matching that range in the remote repository, as it would for a package published on the npm registry.

For example:

npm_packages:
  - https://github.com/jpravetz/hassio-node-red-utils#semver:v0.8.1 

[jpravetz]

Oh goodness, did I really get the # wrong? I will experiment more. I had earlier tried the semver syntax with a # and using the exact syntax you wrote, and thought it hadn't been working at all, which is why I started tagging.

Stay tuned

[jpravetz]

I think this might be an SSH auth problem. Although I entered an https URL, the code is transcoding the URL and using ssh to fetch the repo? I have a private key and the pubkey is up on my own github account, but to no avail.

npm ERR! code 128
npm ERR! An unknown git error occurred
npm ERR! command git --no-replace-objects ls-remote ssh://git@github.com/jpravetz/epdoc-node-red-utils.git
npm ERR! Warning: Permanently added 'github.com' (ED25519) to the list of known hosts.
npm ERR! git@github.com: Permission denied (publickey).
npm ERR! fatal: Could not read from remote repository.
npm ERR! 
npm ERR! Please make sure you have the correct access rights
npm ERR! and the repository exists.
npm ERR! A complete log of this run can be found in: /root/.npm/_logs/2023-09-14T21_35_46_690Z-debug-0.log

Also it confuses me that Advanced SSH & Web Terminal shows more entries in the .ssh folder than my Studio Code Server. I think this has something to do with protection mode. But maybe that's related to this problem, as I switch between the two often.

[sinclairpaul]

Try prefixing it as per the docs? git+https://isaacs@github.com/npm/cli.git

Addons are containers, you won't see the same behaviour between them.

[jpravetz]

Containers, well no bloody wonder. This onion that is home automation is getting more layers to it all the time.

I think I am getting somewhere with your suggestion, @sinclairpaul

[jpravetz]

So far, I find that git+https://jpravetz@github.com/jpravetz/epdoc-node-red-utils works to install.

But git+https://jpravetz@github.com/jpravetz/epdoc-node-red-utils#semver:0.9.1 and git+https://jpravetz@github.com/jpravetz/epdoc-node-red-utils#semver:^0.9.1 don't work to update.

npm ERR! notarget No matching version found for undefined@^0.9.1.

So no solution still for updating to latest.

[github-actions]

There hasn't been any activity on this issue recently, so we clean up some of the older and inactive issues.
Please make sure to update to the latest version and check if that solves the issue. Let us know if that works for you by leaving a comment 👍
This issue has now been marked as stale and will be closed if no further activity occurs. Thanks!

[jpravetz]

It's potentially still an issue for me, but I've gone off on too many tangents, starting a dive into node-red node development, that resulted in several other tangents, including bun. By the time I come up for air I can raise this again if need be. Thus I would close it.

[jpravetz]

but in all honesty if you want this level of customisation I don't think running the addon is for you and you may be better running it elsewhere.

To close out this issue with things I've discovered (for future readers):

• I agree with @sinclairpaul about running Node-Red elsewhere. And it's quite easy to get it running and connect it to home assistant.
• The easiest no-mystery way to deal with dependency install is to directly edit your dependencies in package.json, delete the corresponding dependency folder from under node_modules, and restart Node-Red via it's page at Settings > Add-ons > Node-RED. The add-on will install any missing dependencies when it launches.
• Writing libraries in TypeScript is totally doable. This helps a lot with eliminating bugs. Just† use tsc to transpile.
• Stay away from bun for now (as of v1.0.7)
• I like adding my dependencies to functionGlobalContext in settings.json.
• I also like using Function Node startup scripts, triggered by an Inject Node, to do any necessary initialization
• And by the time you read this far, you'll probably just† be packaging your code into custom Nodes.

† Beware all uses of the word "just".