Add an option to publish a .tgz previously packed with npm pack

[asiffermann]

Hello,

I use VSTS to build and publish a NPM package.
I have a CI build which restores packages, executes the build scripts, runs npm pack and upload the .tgz file as an artifact. Then, I have a release definition which is responsible for deploying the package in VSTS Package Management, MyGet, or nmpjs public registry depending on the stability.

The npm pack command creates a .tgz file with the package name and the version, so it is not possible to predict the final file name.
However, the arguments input of the NPM task does not support minimatch patterns.
Plus, the working folder input is not necessary anymore in this mode (and should be let empty, or the task would fail with the error "TypeError: Cannot read property 'trim' of null").

I think it would be great if we could choose either to publish a folder (and let the working folder input as is) or to publish a tarball (and add an input that supports minimatch for this purpose).

As a temporary fix, I currently extract the tarball and then call the NPM publish task with the working folder set.

Thank you!

[keithrob]

@asiffermann,
I'm not sure I understand your scenario. It sounds like you're packing a project, un-tgzing it, and then publishing it. That seems odd. Feel free to respond back with clarifications.

Here are a few suggestions:

• If you "npm publish" from the same directory as your package.json NPM will correctly bundle everything up and push the resulting .tgz to your configured registry. As long as your "publish" build step is in the working directory with the package.json that defines your NPM package, the build step's "npm publish" command should just work.

• NPM pack does generate predictable package name+version based on what is in the package.json[1].

[1]

C:\temp\packtest> npm version minor ; npm pack
v1.2.0
packtest-1.2.0.tgz
C:\temp\packtest> npm version minor ; npm pack
v1.3.0
packtest-1.3.0.tgz

[asiffermann]

Hi,

The key part of my scenario is that I have two different pipelines: a build definition which creates the package (with the npm pack command), and a release definition which deploys the same package to different registries.

Build definition

Create the package

Copy it in the build artifacts

Release definition

Pipeline

I cannot use the preconfigured publish command

I cannot use the custom command

The minimatch pattern is not resolved, and I cannot select the external npm registry.

My temporary fix

So, at the moment, I can only extract the package and then publish it.
Yes, it IS odd! 😅 That's why I opened this issue to ask if you could add an option to the preconfigured publish command

Hope it's clearer now!
Thanks!

[keithrob]

@asiffermann,
Thanks for the clarification. I think it makes sense to add pattern matching to the "publish" command so that you could publish either a .tgz or package.json. I'll reach back out when we've got something in place for you.

In the near term, you should know that NPM 5+ doesn't support publishing a .tgz so be careful upgrading. In fact, I would recommend using the NPM tool installer in your release definition to ensure that you're using npm 3/4.x while the NPM issue gets resolved... hopefully.

Keith

[yrtimiD]

@asiffermann, if supplied minimatch pattern matches several packages, would you expect all of them to be published or task should fail?

@keithrob, do you have any rough ETA for this feature? I need it and considering doing it by myself. Is it worth to wait?

[asiffermann]

@yrtimiD I suppose that all the matched packages being published would make sense, however it could be confusing and would not follow the NPM command syntax, so I'll go for the task failure 😉

@keithrob I suppose you saw it, but since August the NPM issue got resolved in version 5.6.0 🎉

[keithrob]

@yrtimiD,
It isn't on the top of our backlog right now. Would be happy to look at a PR.

[cfletcher]

Just to add I've got the same issue, in our instance we're building (packing) within a docker container but it's necessary (on-top of being a good idea) to publish from TFS because we need to authenticate with the registry. I'm going to use your workaround @asiffermann

[AArnott]

Everyone with the same issue should please Thumbs Up the original message box that describes the issue so that the owners can better understand how many customers are impacted in one place.

[AArnott]

@keithrob said:

In the near term, you should know that NPM 5+ doesn't support publishing a .tgz so be careful upgrading.

Publishing tarballs is still very much a documented feature. So I think it's safe to add support for publishing .tgz files directly in the NPM task.

I would recommend using the NPM tool installer in your release definition

I don't see any NPM installer task. I see a Node tool installer task, and maybe that's loosely coupled with NPM, but it's not clear to me which version of Node.js to install in order to get a particular version of NPM.

My attempt at a workaround for this is to use a PowerShell task to find the name of the file to publish (since the version changes for each release) and set that filename to a variable. Then my next release task is the NPM task with the 'custom' command set to publish $(myvariable). This appears to almost work, in that npm publish is invoked with my tgz, but after enumerating its contents, the command ultimately fails with:

2018-09-25T04:25:53.8900942Z 30 error path C:\Users\VSSADM~1\AppData\Local\Temp\npm-4080-2c7addee\tmp\fromPackage-da780555\package.json
2018-09-25T04:25:53.8901357Z 31 error code EPERM
2018-09-25T04:25:53.8901571Z 32 error errno -4048
2018-09-25T04:25:53.8901998Z 33 error syscall unlink
2018-09-25T04:25:53.8902331Z 34 error Error: EPERM: operation not permitted, unlink 'C:\Users\VSSADM~1\AppData\Local\Temp\npm-4080-2c7addee\tmp\fromPackage-da780555\package.json'
2018-09-25T04:25:53.8902785Z 34 error  { [Error: EPERM: operation not permitted, unlink 'C:\Users\VSSADM~1\AppData\Local\Temp\npm-4080-2c7addee\tmp\fromPackage-da780555\package.json']
2018-09-25T04:25:53.8903105Z 34 error   cause:
2018-09-25T04:25:53.8903425Z 34 error    { Error: EPERM: operation not permitted, unlink 'C:\Users\VSSADM~1\AppData\Local\Temp\npm-4080-2c7addee\tmp\fromPackage-da780555\package.json'
2018-09-25T04:25:53.8903765Z 34 error      errno: -4048,
2018-09-25T04:25:53.8903976Z 34 error      code: 'EPERM',
2018-09-25T04:25:53.8904200Z 34 error      syscall: 'unlink',
2018-09-25T04:25:53.8904416Z 34 error      path:
2018-09-25T04:25:53.8904729Z 34 error       'C:\\Users\\VSSADM~1\\AppData\\Local\\Temp\\npm-4080-2c7addee\\tmp\\fromPackage-da780555\\package.json' },
2018-09-25T04:25:53.8905028Z 34 error   isOperational: true,
2018-09-25T04:25:53.8905241Z 34 error   stack:
2018-09-25T04:25:53.8905576Z 34 error    'Error: EPERM: operation not permitted, unlink \'C:\\Users\\VSSADM~1\\AppData\\Local\\Temp\\npm-4080-2c7addee\\tmp\\fromPackage-da780555\\package.json\'',
2018-09-25T04:25:53.8905911Z 34 error   errno: -4048,
2018-09-25T04:25:53.8906116Z 34 error   code: 'EPERM',
2018-09-25T04:25:53.8906336Z 34 error   syscall: 'unlink',
2018-09-25T04:25:53.8906548Z 34 error   path:
2018-09-25T04:25:53.8906836Z 34 error    'C:\\Users\\VSSADM~1\\AppData\\Local\\Temp\\npm-4080-2c7addee\\tmp\\fromPackage-da780555\\package.json' }
2018-09-25T04:25:53.8907169Z 35 error The operation was rejected by your operating system.
2018-09-25T04:25:53.8907465Z 35 error It's possible that the file was already in use (by a text editor or antivirus),
2018-09-25T04:25:53.8907769Z 35 error or that you lack permissions to access it.
2018-09-25T04:25:53.8907979Z 35 error
2018-09-25T04:25:53.8908238Z 35 error If you believe this might be a permissions issue, please double-check the
2018-09-25T04:25:53.8908542Z 35 error permissions of the file and its containing directories, or try running
2018-09-25T04:25:53.8908855Z 35 error the command again as root/Administrator (though this is not recommended).
2018-09-25T04:25:53.8909115Z 36 verbose exit [ -4048, true ]

[AArnott]

So I eventually settled on the workaround of running npm install mypackage-version.tgz and then running the NPM task's publish command, pointing at node_modules/mypackage . That worked.

[infin8x]

Closing this as WF. In the upcoming quarter, we'll be moving away from the full-featured Artifacts-related tasks and recommending calling npm directly, which will give you the flexibility to call whatever command you wish, even if you're working against an authenticated feed. See the design here for more details.