cordova@8.0.0 -> cordova@9.0.0 breaks plugin installs using git repos or local directories #87

shankari · 2020-07-14T05:18:43Z

Bug Report

Problem

in cordova@8.0.0, installing plugins from git repos works
in cordova@9.0.0, installing plugins from the same identical git repos fails

What is expected to happen?

$ npx cordova plugin add https://github.com/phonegap/phonegap-plugin-contentsync.git

should succeed since "url to a git repository containing a plugin.xml" is a valid plugin-spec.

What does actually happen?

the plugin add succeeds on cordova@8.0.0
the plugin add fails on cordova@9.0.0

Information

on cordova@8.0.0, plugins can be added using either the registry or a git URL

$ npx cordova --version
8.0.0

$ npx cordova plugin add https://github.com/phonegap/phonegap-plugin-contentsync.git
Installing "phonegap-plugin-contentsync" for android
Installing "phonegap-plugin-contentsync" for ios
Adding phonegap-plugin-contentsync to package.json
Saved plugin info for "phonegap-plugin-contentsync" to config.xml

$ npx cordova plugin rm phonegap-plugin-contentsync
Uninstalling phonegap-plugin-contentsync from android
Uninstalling phonegap-plugin-contentsync from ios
Removing "phonegap-plugin-contentsync"
Removing plugin phonegap-plugin-contentsync from config.xml file...
Removing phonegap-plugin-contentsync from package.json

$ npx cordova plugin add phonegap-plugin-contentsync
Installing "phonegap-plugin-contentsync" for android
Installing "phonegap-plugin-contentsync" for ios
Adding phonegap-plugin-contentsync to package.json
Saved plugin info for "phonegap-plugin-contentsync" to config.xml

$ npx cordova plugin rm phonegap-plugin-contentsync
Uninstalling phonegap-plugin-contentsync from android
Uninstalling phonegap-plugin-contentsync from ios
Removing "phonegap-plugin-contentsync"
Removing plugin phonegap-plugin-contentsync from config.xml file...
Removing phonegap-plugin-contentsync from package.json

In the same directory, with cordova@9.0.0, plugins can only be added using the registry, and only intermittently

$ npm install cordova@9.0.0

+ cordova@9.0.0
added 161 packages from 112 contributors, removed 73 packages, updated 53 packages and audited 1707 packages in 19.859s

Note that, in the case of the registry add, the first run of the command fails, and the second run of the identical command succeeds

$ npx cordova plugin add phonegap-plugin-contentsync
Failed to fetch plugin phonegap-plugin-contentsync via registry.
Probably this is either a connection problem, or plugin spec is incorrect.
Check your connection and plugin name/version/URL.
Could not determine package name from output:
phonegap-plugin-contentsync@1.4.2 node_modules/phonegap-plugin-contentsync

$ npx cordova plugin add phonegap-plugin-contentsync
Installing "phonegap-plugin-contentsync" for android
Installing "phonegap-plugin-contentsync" for ios
Adding phonegap-plugin-contentsync to package.json

However, the git add always fails.

$ npx cordova plugin add https://github.com/phonegap/phonegap-plugin-contentsync.git
Failed to fetch plugin https://github.com/phonegap/phonegap-plugin-contentsync.git via registry.
Probably this is either a connection problem, or plugin spec is incorrect.
Check your connection and plugin name/version/URL.
Could not determine package name from output:
phonegap-plugin-contentsync@1.4.2 node_modules/phonegap-plugin-contentsync

$ npx cordova plugin add https://github.com/phonegap/phonegap-plugin-contentsync.git
Failed to fetch plugin https://github.com/phonegap/phonegap-plugin-contentsync.git via registry.
Probably this is either a connection problem, or plugin spec is incorrect.
Check your connection and plugin name/version/URL.
Could not determine package name from output:
phonegap-plugin-contentsync@1.4.2 node_modules/phonegap-plugin-contentsync

$ npx cordova plugin add https://github.com/phonegap/phonegap-plugin-contentsync.git
Failed to fetch plugin https://github.com/phonegap/phonegap-plugin-contentsync.git via registry.
Probably this is either a connection problem, or plugin spec is incorrect.
Check your connection and plugin name/version/URL.
Could not determine package name from output:
phonegap-plugin-contentsync@1.4.2 node_modules/phonegap-plugin-contentsync

Command or Code

$ cordova plugin add <git URL>

OR

$ cordova plugin add <local URL>

Environment, Platform, Device

Mac OSX Catalina 10.15.5

Version information

For cordova and ionic:

$ npx cordova --version
9.0.0 (cordova-lib@9.0.1)
$ npx ionic --version
6.10.1

For other cordova dependencies, I have attached my package.json
package.json.gz

For the rest of the environment:

OSX 10.15.5
Android Studio 4.0
Xcode 11.5

export NVM_VERSION=0.35.3
export NODE_VERSION=14.5.0
export NPM_VERSION=6.14.5

Checklist

I searched for existing GitHub issues
I updated all Cordova tooling to most recent version
I included all the necessary information above

The text was updated successfully, but these errors were encountered:

shankari · 2020-07-14T05:27:12Z

I poked around this a little bit, and the Failed to fetch plugin ... via registry error is from cordova-lib/src/plugman/fetch.js

Code

                .catch(function (error) {
                    var message = 'Failed to fetch plugin ' + plugin_src + ' via registry.' +
                        '\nProbably this is either a connection problem, or plugin spec is incorrect.' +
                        '\nCheck your connection and plugin name/version/URL.' +
                        '\n' + error;
                    return Promise.reject(new CordovaError(message));

So the real error is Could not determine package name from output

shankari · 2020-07-14T05:30:43Z

Could not determine package name from output is from node_modules//cordova-fetch/index.js

Code

function getTargetPackageSpecFromNpmInstallOutput (npmInstallOutput) {
    const lines = npmInstallOutput.split('\n');
    for (let i = 0; i < lines.length; i++) {
        if (lines[i].startsWith('+ ')) {
            // npm >= 5
            return lines[i].slice(2);
        } else if (lines[i].startsWith('└─') || lines[i].startsWith('`-')) {
            // 3 <= npm <= 4
            return lines[i].slice(4).split(' ')[0];
        }
    }
    throw new CordovaError('Could not determine package name from output:\n' + npmInstallOutput);
}

Essentially, the code expects the npm install output to have a +, but it doesn't. The output is just phonegap-plugin-contentsync@1.4.2 node_modules/phonegap-plugin-contentsync with no +, which is why this fails.

shankari · 2020-07-14T05:40:31Z

The npm install command is launched using superspawn.spawn.

And now I come to the part that I don't understand. If I use -d, I can see the arguments passed in to npm install

Command line arguments for npm install

No scripts found for hook "before_plugin_add".
Calling plugman.fetch on plugin "https://github.com/phonegap/phonegap-plugin-contentsync.git"
fetch: Installing https://github.com/phonegap/phonegap-plugin-contentsync.git to /Users/kshankar/e-mission/upgrade_platform
Running command: npm install https://github.com/phonegap/phonegap-plugin-contentsync.git --production --save
Command finished with error code 0: npm install,https://github.com/phonegap/phonegap-plugin-contentsync.git,--production,--save
Failed to fetch plugin https://github.com/phonegap/phonegap-plugin-contentsync.git via registry.
Probably this is either a connection problem, or plugin spec is incorrect.
Check your connection and plugin name/version/URL.
Could not determine package name from output:
phonegap-plugin-contentsync@1.4.2 node_modules/phonegap-plugin-contentsync
CordovaError: Failed to fetch plugin https://github.com/phonegap/phonegap-plugin-contentsync.git via registry.
Probably this is either a connection problem, or plugin spec is incorrect.
Check your connection and plugin name/version/URL.
Could not determine package name from output:
phonegap-plugin-contentsync@1.4.2 node_modules/phonegap-plugin-contentsync
    at /Users/kshankar/e-mission/upgrade_platform/node_modules/cordova-lib/src/plugman/fetch.js:146:43
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

If I run the exact same command using superspawn using the node REPL, I get an output with the +

running same command in node REPL

$ node
Welcome to Node.js v14.5.0.
Type ".help" for more information.
> var superspawn = require('cordova-common').superspawn;
undefined
> var dest = "<redacted>"
undefined
> var args = ["install", "https://github.com/phonegap/phonegap-plugin-contentsync.git", "--production", "--save"]
undefined
> superspawn.spawn('npm', args, { cwd: dest }).then(function(output){ console.log(output); });
Promise {
  promiseDispatch: [Function (anonymous)],
  valueOf: [Function (anonymous)],
  inspect: [Function (anonymous)]
}
> (node:59879) [DEP0097] DeprecationWarning: Using a domain property in MakeCallback is deprecated. Use the async_context variant of MakeCallback or the AsyncResource class instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
+ phonegap-plugin-contentsync@1.4.2
updated 1 package and audited 1707 packages in 11.991s

47 packages are looking for funding
  run `npm fund` for details

found 173 vulnerabilities (45 low, 75 moderate, 53 high)
  run `npm audit fix` to fix them, or `npm audit` for details

But if I add the same kind of output logging in fetch.js, it returns the output without the +

e.g. if I change the code to

New code

function installPackage (target, dest, opts) {
    return isNpmInstalled()
        // Ensure that `npm` installs to `dest` and not any of its ancestors
        .then(_ => fs.ensureDir(path.join(dest, 'node_modules')))

        // Run `npm` to install requested package
        .then(_ => npmArgs(target, opts))
        .then(args => {
            events.emit('verbose', `fetch: Installing ${target} to ${dest}`);
            return superspawn.spawn('npm', args, { cwd: dest });
        })
        .then(function(output){ console.log("Output = "+output); return output; })

        // Resolve path to installed package
        .then(getTargetPackageSpecFromNpmInstallOutput)
        .then(spec => pathToInstalledPackage(spec, dest));
}

I get the following output

non-REPL usage

$ npx cordova plugin -d add https://github.com/phonegap/phonegap-plugin-contentsync.git
No scripts found for hook "before_plugin_add".
Calling plugman.fetch on plugin "https://github.com/phonegap/phonegap-plugin-contentsync.git"
fetch: Installing https://github.com/phonegap/phonegap-plugin-contentsync.git to /Users/kshankar/e-mission/upgrade_platform
Running command: npm install https://github.com/phonegap/phonegap-plugin-contentsync.git --production --save
Command finished with error code 0: npm install,https://github.com/phonegap/phonegap-plugin-contentsync.git,--production,--save
Output = phonegap-plugin-contentsync@1.4.2 node_modules/phonegap-plugin-contentsync
Failed to fetch plugin https://github.com/phonegap/phonegap-plugin-contentsync.git via registry.
Probably this is either a connection problem, or plugin spec is incorrect.
Check your connection and plugin name/version/URL.
Could not determine package name from output:
phonegap-plugin-contentsync@1.4.2 node_modules/phonegap-plugin-contentsync
CordovaError: Failed to fetch plugin https://github.com/phonegap/phonegap-plugin-contentsync.git via registry.
Probably this is either a connection problem, or plugin spec is incorrect.
Check your connection and plugin name/version/URL.
Could not determine package name from output:
phonegap-plugin-contentsync@1.4.2 node_modules/phonegap-plugin-contentsync
    at /Users/kshankar/e-mission/upgrade_platform/node_modules/cordova-lib/src/plugman/fetch.js:146:43
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

I can't figure out why superspawn behaves differently in the REPL and in the script. Any thoughts?

shankari · 2020-07-14T05:44:01Z

Finally, -d also gives us a clue as to why the registry install works (eventually). It uses plugman install instead of npm install

Logs

``` $ npx cordova plugin -d add phonegap-plugin-contentsync No scripts found for hook "before_plugin_add". No version specified for phonegap-plugin-contentsync, retrieving version from package.json Calling plugman.fetch on plugin "phonegap-plugin-contentsync@^1.4.2" Copying plugin "/Users/kshankar/e-mission/upgrade_platform/node_modules/phonegap-plugin-contentsync" => "/Users/kshankar/e-mission/upgrade_platform/plugins/phonegap-plugin-contentsync" Calling plugman.install on plugin "/Users/kshankar/e-mission/upgrade_platform/plugins/phonegap-plugin-contentsync" for platform "android Installing "phonegap-plugin-contentsync" for android Running command: /Users/kshankar/e-mission/upgrade_platform/platforms/android/cordova/version Command finished with error code 0: /Users/kshankar/e-mission/upgrade_platform/platforms/android/cordova/version Finding scripts for "before_plugin_install" hook from plugin phonegap-plugin-contentsync on android platform only. No scripts found for hook "before_plugin_install". Install start for "phonegap-plugin-contentsync" on android. PlatformApi successfully found for platform android Beginning processing of action stack for android project... Action stack processing complete. Install complete for phonegap-plugin-contentsync on android. Finding scripts for "after_plugin_install" hook from plugin phonegap-plugin-contentsync on android platform only. No scripts found for hook "after_plugin_install". Calling plugman.install on plugin "/Users/kshankar/e-mission/upgrade_platform/plugins/phonegap-plugin-contentsync" for platform "ios Installing "phonegap-plugin-contentsync" for ios Running command: /Users/kshankar/e-mission/upgrade_platform/platforms/ios/cordova/version Command finished with error code 0: /Users/kshankar/e-mission/upgrade_platform/platforms/ios/cordova/version Finding scripts for "before_plugin_install" hook from plugin phonegap-plugin-contentsync on ios platform only. No scripts found for hook "before_plugin_install". Install start for "phonegap-plugin-contentsync" on ios. PlatformApi successfully found for platform ios Beginning processing of action stack for ios project... Adding non-custom framework to project... libz.dylib -> {"customFramework":false,"embed":false,"link":true,"weak":false} Non-custom framework added to project. libz.dylib -> {"customFramework":false,"link":true,"weak":false} Action stack processing complete. pods.json found in platforms/ios Podfile found in platforms/ios Install complete for phonegap-plugin-contentsync on ios. Finding scripts for "after_plugin_install" hook from plugin phonegap-plugin-contentsync on ios platform only. No scripts found for hook "after_plugin_install". Adding phonegap-plugin-contentsync to package.json No scripts found for hook "after_plugin_add". ```

raphinesse · 2020-07-14T18:45:56Z

Thanks for reporting this and for the detailed information and debugging. It's indeed strange that the npm output differs between manual invocations and those performed by cordova-fetch.

Just to be sure that this is still an issue in the current code base, could you please try if you can reproduce this issue when using cordova@nightly?

shankari · 2020-07-14T20:38:28Z

Ok, so I poked around this some more, and I know the reason behind the difference in superspawn behavior.

After adding log statements to all instances of superspawn, it looks like this is difference is due to the differences in the versions of superspawn included in the project.

$ find node_modules -name \*superspawn.js
node_modules/cordova/node_modules/cordova-common/src/superspawn.js
node_modules/cordova-common/src/superspawn.js
node_modules/cordova-create/node_modules/cordova-common/src/superspawn.js
node_modules/cordova-fetch/node_modules/cordova-common/src/superspawn.js
node_modules/cordova-lib/node_modules/cordova-common/src/superspawn.js
node_modules/phonegap/node_modules/cordova-common/src/superspawn.js
node_modules/phonegap/node_modules/cordova-create/node_modules/cordova-common/src/superspawn.js

While using node directly, we use node_modules/cordova-common/src/superspawn.js, while using plugin add, we use node_modules/cordova-fetch/node_modules/cordova-common/src/superspawn.js. The two versions are different, although I cannot see which of the differences lead to this difference in behavior.

not_working_to_working.diff.gz

shankari · 2020-07-14T20:51:30Z

so the two versions are different because the versions of cordova-common are different.

$ find node_modules -name package.json | grep "cordova-common.package" | xargs grep version.:
node_modules/cordova/node_modules/cordova-common/package.json:  "version": "3.2.1"
node_modules/cordova-common/package.json:  "version": "4.0.2"
node_modules/cordova-create/node_modules/cordova-common/package.json:  "version": "3.2.1"
node_modules/cordova-fetch/node_modules/cordova-common/package.json:  "version": "3.2.1"
node_modules/cordova-lib/node_modules/cordova-common/package.json:  "version": "3.2.1"
node_modules/phonegap/node_modules/cordova-common/package.json:  "version": "2.1.1"
node_modules/phonegap/node_modules/cordova-create/node_modules/cordova-common/package.json:  "version": "2.2.5"

shankari · 2020-07-14T21:16:50Z

But if I use the cordova-fetch version superspawn directly in the REPL, it works correctly. Which is consistent with the diffs seeming fairly innocuous. The main difference seems to be running this in cordova.

Logs

> var superspawn = require('./node_modules/cordova-fetch/node_modules/cordova-common/src/superspawn').superspawn;

> superspawn.spawn('npm', args, { cwd: dest }).then(function(output){ console.log(output); return output;})
Promise {
  promiseDispatch: [Function (anonymous)],
  valueOf: [Function (anonymous)],
  inspect: [Function (anonymous)]
}
> + phonegap-plugin-contentsync@1.4.2
updated 1 package and audited 1707 packages in 19.092s

47 packages are looking for funding
  run `npm fund` for details

found 173 vulnerabilities (45 low, 75 moderate, 53 high)
  run `npm audit fix` to fix them, or `npm audit` for details

I will try with the latest cordova@nightly and if that still doesn't work, I'm going to try to create a MCVE to make it easier for others to debug as well.

shankari · 2020-07-14T22:43:07Z

@raphinesse Confirmed that this occurs even with cordova@nightly

Logs

$ npx cordova plugin add https://github.com/phonegap/phonegap-plugin-contentsync.git
Warning: using prerelease version 10.0.0-nightly.2020.7.14.d69f1084 (cordova-lib@10.0.0-nightly.2020.7.14.784f6b4b)
Failed to fetch plugin https://github.com/phonegap/phonegap-plugin-contentsync.git via registry.
Probably this is either a connection problem, or plugin spec is incorrect.
Check your connection and plugin name/version/URL.
CordovaError: CordovaError: Could not determine package name from output:
phonegap-plugin-contentsync@1.4.2 node_modules/phonegap-plugin-contentsync

shankari · 2020-07-15T00:04:28Z

Aha! While creating the MCVE, figured out the issue. I have both cordova and phonegap as dependencies. I was working on upgrading cordova first, and had not yet started upgrading phonegap, which was still at 7.1.1. After upgrading phonegap to 9.0.0+cordova.9.0.0, the error disappeared. I guess 7.1.1 was close enough to 8.0.0 to work, but was too far from 9.0.0

Similar to 9c705ab but for the other package.json Note that this includes phonegap and the resulting upgrade apache/cordova-fetch#87

raphinesse transferred this issue from apache/cordova-lib Jul 14, 2020

shankari closed this as completed Jul 15, 2020

shankari added a commit to shankari/e-mission-phone that referenced this issue Aug 1, 2020

Upgrade the dependencies for the serve side as well

d4a184b

Similar to 9c705ab but for the other package.json Note that this includes phonegap and the resulting upgrade apache/cordova-fetch#87

asiripanich pushed a commit to rciti/e-mission-phone that referenced this issue Aug 7, 2021

Upgrade the dependencies for the serve side as well

3099dd5

Similar to 9c705ab but for the other package.json Note that this includes phonegap and the resulting upgrade apache/cordova-fetch#87

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cordova@8.0.0 -> cordova@9.0.0 breaks plugin installs using git repos or local directories #87

cordova@8.0.0 -> cordova@9.0.0 breaks plugin installs using git repos or local directories #87

shankari commented Jul 14, 2020

shankari commented Jul 14, 2020 •

edited

Loading

shankari commented Jul 14, 2020 •

edited

Loading

shankari commented Jul 14, 2020 •

edited

Loading

shankari commented Jul 14, 2020 •

edited

Loading

raphinesse commented Jul 14, 2020

shankari commented Jul 14, 2020 •

edited

Loading

shankari commented Jul 14, 2020

shankari commented Jul 14, 2020 •

edited

Loading

shankari commented Jul 14, 2020 •

edited

Loading

shankari commented Jul 15, 2020 •

edited

Loading

cordova@8.0.0 -> cordova@9.0.0 breaks plugin installs using git repos or local directories #87

cordova@8.0.0 -> cordova@9.0.0 breaks plugin installs using git repos or local directories #87

Comments

shankari commented Jul 14, 2020

Bug Report

Problem

What is expected to happen?

What does actually happen?

Information

on cordova@8.0.0, plugins can be added using either the registry or a git URL

In the same directory, with cordova@9.0.0, plugins can only be added using the registry, and only intermittently

Command or Code

Environment, Platform, Device

Version information

Checklist

shankari commented Jul 14, 2020 • edited Loading

shankari commented Jul 14, 2020 • edited Loading

shankari commented Jul 14, 2020 • edited Loading

shankari commented Jul 14, 2020 • edited Loading

raphinesse commented Jul 14, 2020

shankari commented Jul 14, 2020 • edited Loading

shankari commented Jul 14, 2020

shankari commented Jul 14, 2020 • edited Loading

shankari commented Jul 14, 2020 • edited Loading

shankari commented Jul 15, 2020 • edited Loading

shankari commented Jul 14, 2020 •

edited

Loading

shankari commented Jul 14, 2020 •

edited

Loading

shankari commented Jul 14, 2020 •

edited

Loading

shankari commented Jul 14, 2020 •

edited

Loading

shankari commented Jul 14, 2020 •

edited

Loading

shankari commented Jul 14, 2020 •

edited

Loading

shankari commented Jul 14, 2020 •

edited

Loading

shankari commented Jul 15, 2020 •

edited

Loading