TFS/VSTS Package Management authentication #13

Megasware128 · 2017-04-10T14:17:16Z

Great to use Yarn as a build task but unlike the npm tasks available, Yarn doesn't seem to automatically authenticate for npm registries created by the Package Management feature in TFS2017/VSTS. Is there a solution for using Package Management registries in combination with the Yarn build task?

sandorfr · 2017-05-07T23:28:37Z

This is of course a great suggestion. We need to investigate how we can adapt the authentication procedure from the npm task to yarn https://github.com/Microsoft/vsts-tasks/blob/master/Tasks/Npm/npmtask.ts#L190 .

This is a priority one for us.

waterfoul · 2017-06-13T19:50:02Z

Looks like the code you want to be looking at is in https://github.com/Microsoft/vsts-tasks/blob/master/Tasks/Npm/npmregistry.ts

mihir0x69 · 2017-07-07T07:02:59Z

We can't switch to yarn on our build server because of this :(

sandorfr · 2017-07-11T04:49:08Z

Happy to accept a PR is anyone has the time to work on this.

mihir0x69 · 2017-07-11T06:22:24Z

@sandorfr Would have loved to work on this. But I lack the context.

Megasware128 · 2017-07-11T06:55:58Z

I might take a look but need to find some time

manu-unter · 2017-07-28T07:35:20Z

A manual way to set the access token using a build agent that we use at the moment is the following PowerShell script:

Add-Content <path-to-your-npmrc> "`r`n//<your-team>.pkgs.visualstudio.com/_packaging/<your-feed-name>/npm/registry/:_authToken=$(System.AccessToken)"

It will add the access token to the .npmrc so yarn can use it to access the registry. You should run this before doing yarn or yarn install. Yarn will consider any .npmrc files, even upwards in the folder tree, so just put it somewhere within the tree and it should work.

Please mind: this requires to have the following Build Definition Option to be set:

sandorfr · 2017-08-02T12:10:16Z

I adjusted the roadmap, as I'm focused on this issue right now. This is basically around the corner, but I'm running into issues similar to this yarnpkg/yarn#678 . It seems like it might be necessary to generate a yarnrc in addition to npmrc.

sandorfr · 2017-08-13T13:24:14Z

I've pushed a new major version including VSTS feed support.

Mardoxx · 2017-09-13T10:15:46Z

Broken for me. npm install works fined, yarn install yields 401 with VSTS feeds.

Also getting lots of warnings:

2017-09-13T10:15:02.0212231Z ##[warning]Resource file haven\'t set, can\'t find loc string for key: FoundLocalRegistries
2017-09-13T10:15:02.0222238Z ##[warning]Resource file haven\'t set, can\'t find loc string for key: FoundBuildCredentials
2017-09-13T10:15:03.4391547Z ##[warning]Resource file haven\'t set, can\'t find loc string for key: SavingFile

I have tried with Registries in my .npmrc file and Registry I select here, both give 401. Either of these options work fine with npm task.

Using @cryza's method works fine.

sandorfr · 2017-09-13T11:37:19Z

Can you share de debug log? Le mer. 13 sept. 2017 à 20:15, Mardoxx <notifications@github.com> a écrit :

…

Broken for me. npm install works fined, yarn install yields 401 with VSTS feeds. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#13 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACiXpqOK14MujuhZ2J-JDnzLFquj2kMXks5sh6tTgaJpZM4M41-k> .

Mardoxx · 2017-09-18T15:44:35Z

@sandorfr weird, it's working now. I was having issues with VSTS feeds locally too so perhaps this was unrelated.

Mardoxx · 2017-09-18T15:51:46Z

Ok it's still broken on my build. Here's debug log

******************************************************************************
Starting: Yarn install --verbose
******************************************************************************
==============================================================================
Task         : Yarn task
Description  : Executes Yarn
Version      : 1.1.1001
Author       : Geek Learning
Help         : [More Information](https://github.com/geeklearningio/gl-vsts-tasks-yarn/wiki/Yarn) (Version 1.1.1001)
==============================================================================
agent.workFolder=d:\a
loading inputs and endpoints
loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN
loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION
loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION
loading INPUT_ARGUMENTS
loading INPUT_CUSTOMFEED
loading INPUT_CUSTOMREGISTRY
loading INPUT_PRODUCTIONMODE
loading INPUT_PROJECTDIRECTORY
loading SECRET_NPM_ACCESSTOKEN
loaded 9
which 'yarn'
found: 'C:\NPM\Modules\yarn.cmd'
Arguments=install --verbose
ProjectDirectory=d:\a\1\s
customRegistry=useNpmrc
customFeed=c314204c-5eb4-4dbd-b660-e0c97452ddd5
customEndpoint=null
C:\NPM\Modules\yarn.cmd
Build.BuildId=608
Agent.BuildDirectory=d:\a\1
testing directory 'd:\a\1\npm'
testing directory 'd:\a\1'
mkdir 'd:\a\1\npm'
Build.BuildId=608
Agent.BuildDirectory=d:\a\1
Npm.PackagingCollectionUrl=undefined
System.TeamFoundationCollectionUri=https://{companyName}.visualstudio.com/
Resource file haven\'t set, can\'t find loc string for key: FoundLocalRegistries
Processed: ##vso[task.issue type=warning;]Resource file haven\'t set, can\'t find loc string for key: FoundLocalRegistries
FoundLocalRegistries 1
SYSTEMVSSCONNECTION exists true
SYSTEMVSSCONNECTION exists true
Resource file haven\'t set, can\'t find loc string for key: FoundBuildCredentials
Processed: ##vso[task.issue type=warning;]Resource file haven\'t set, can\'t find loc string for key: FoundBuildCredentials
FoundBuildCredentials
Npm.PackagingCollectionUrl=undefined
System.TeamFoundationCollectionUri=https://{companyName}.visualstudio.com/
SYSTEMVSSCONNECTION exists true
SYSTEMVSSCONNECTION exists true
Resource file haven\'t set, can\'t find loc string for key: FoundBuildCredentials
Processed: ##vso[task.issue type=warning;]Resource file haven\'t set, can\'t find loc string for key: FoundBuildCredentials
FoundBuildCredentials
Using registries in .npmrc
customEndpoint=null
Adding auth for registry: https://{companyName}.pkgs.visualstudio.com/_packaging/{FeedName}/npm/registry/
which 'C:\NPM\Modules\yarn.cmd'
found: 'C:\NPM\Modules\yarn.cmd'
which 'C:\NPM\Modules\yarn.cmd'
found: 'C:\NPM\Modules\yarn.cmd'
ProductionMode=false
C:\NPM\Modules\yarn.cmd arg: install --verbose
exec tool: C:\NPM\Modules\yarn.cmd
arguments:
install
--verbose
C:\Windows\system32\cmd.exe /D /S /C "C:\NPM\Modules\yarn.cmd install --verbose"
yarn install v1.0.1
verbose 0.684 Checking for configuration file "d:\\a\\1\\s\\.npmrc".
verbose 0.685 Found configuration file "d:\\a\\1\\s\\.npmrc".
verbose 0.685 Checking for configuration file "C:\\Users\\buildguest\\.npmrc".
verbose 0.686 Checking for configuration file "d:\\a\\_tool\\node\\8.5.0\\x64\\etc\\npmrc".
verbose 0.686 Checking for configuration file "d:\\a\\1\\s\\.npmrc".
verbose 0.686 Found configuration file "d:\\a\\1\\s\\.npmrc".
verbose 0.687 Checking for configuration file "d:\\a\\1\\.npmrc".
verbose 0.687 Checking for configuration file "d:\\a\\.npmrc".
verbose 0.691 Checking for configuration file "d:\\a\\1\\s\\.yarnrc".
verbose 0.691 Checking for configuration file "C:\\Users\\buildguest\\.yarnrc".
verbose 0.692 Checking for configuration file "d:\\a\\_tool\\node\\8.5.0\\x64\\etc\\yarnrc".
verbose 0.692 Checking for configuration file "d:\\a\\1\\s\\.yarnrc".
verbose 0.692 Checking for configuration file "d:\\a\\1\\.yarnrc".
verbose 0.692 Checking for configuration file "d:\\a\\.yarnrc".
verbose 0.696 current time: 2017-09-18T15:49:42.957Z
 Resolving packages...
 Fetching packages...
verbose 2.343 Performing "GET" request to "https://{companyName}.pkgs.visualstudio.com/_packaging/{FeedName}/npm/registry/axios/-/axios-0.16.2.tgz".
verbose 2.356 Performing "GET" request to "https://{companyName}.pkgs.visualstudio.com/_packaging/{FeedName}/npm/registry/follow-redirects/-/follow-redirects-1.2.4.tgz".
verbose 2.388 Performing "GET" request to "https://{companyName}.pkgs.visualstudio.com/_packaging/{FeedName}/npm/registry/is-buffer/-/is-buffer-1.1.5.tgz".
verbose 2.394 Performing "GET" request to "https://{companyName}.pkgs.visualstudio.com/_packaging/{FeedName}/npm/registry/debug/-/debug-2.6.8.tgz".
verbose 2.396 Performing "GET" request to "https://{companyName}.pkgs.visualstudio.com/_packaging/{FeedName}/npm/registry/ms/-/ms-2.0.0.tgz".
verbose 2.398 Performing "GET" request to "https://{companyName}.pkgs.visualstudio.com/_packaging/{FeedName}/npm/registry/babel-core/-/babel-core-6.26.0.tgz".
verbose 2.4 Performing "GET" request to "https://{companyName}.pkgs.visualstudio.com/_packaging/{FeedName}/npm/registry/babel-code-frame/-/babel-code-frame-6.26.0.tgz".
verbose 2.403 Performing "GET" request to "https://{companyName}.pkgs.visualstudio.com/_packaging/{FeedName}/npm/registry/babel-generator/-/babel-generator-6.26.0.tgz".
verbose 2.482 Error: https://{companyName}.pkgs.visualstudio.com/_packaging/{FeedName}/npm/registry/follow-redirects/-/follow-redirects-1.2.4.tgz: Request failed "401 Unauthorized"
    at Request.<anonymous> (C:\NPM\Modules\node_modules\yarn\lib\cli.js:57662:26)
    at emitOne (events.js:115:13)
    at Request.emit (events.js:210:7)
    at Request.module.exports.Request.onRequestResponse (C:\NPM\Modules\node_modules\yarn\lib\cli.js:118210:10)
    at emitOne (events.js:115:13)
    at ClientRequest.emit (events.js:210:7)
    at HTTPParser.parserOnIncomingClient (_http_client.js:565:21)
    at HTTPParser.parserOnHeadersComplete (_http_common.js:116:23)
    at TLSSocket.socketOnData (_http_client.js:454:20)
    at emitOne (events.js:115:13)
error An unexpected error occurred: "https://{companyName}.pkgs.visualstudio.com/_packaging/{FeedName}/npm/registry/follow-redirects/-/follow-redirects-1.2.4.tgz: Request failed \"401 Unauthorized\"".
info If you think this is a bug, please open a bug report with the information provided in "d:\\a\\1\\s\\yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
Error: C:\NPM\Modules\yarn.cmd failed with return code: 1
rc:1
success:false
Error: C:\NPM\Modules\yarn.cmd failed with return code: 1
task result: Failed
Error: C:\NPM\Modules\yarn.cmd failed with return code: 1
Processed: ##vso[task.issue type=error;]Error: C:\NPM\Modules\yarn.cmd failed with return code: 1
Processed: ##vso[task.complete result=Failed;]Error: C:\NPM\Modules\yarn.cmd failed with return code: 1
Agent.BuildDirectory=d:\a\1
rm -rf d:\a\1\npm
removing directory
******************************************************************************
Finishing: Yarn install --verbose
******************************************************************************

If I add a powershell task before it to do this (with access token as an env var):

Add-Content .npmrc "`r`n//{companyName}.pkgs.visualstudio.com/_packaging/{FeedName}/npm/registry/:_authToken=$(npm.accessToken)"
Add-Content .npmrc "`r`n//{companyName}.pkgs.visualstudio.com/_packaging/{FeedName}/npm/:_authToken=$(npm.accessToken)"

It works fine.

I notice the line
Adding auth for registry: https://{companyName}.pkgs.visualstudio.com/_packaging/{FeedName}/npm/registry/
so it should work correctly? Weird. The only difference I see here is that it's potentially (haven't looked at your code) only adding it for /npm/registry/ and not /npm/ as well, and that the scheme is provided as https://, where as I'm providing it as relative scheme.

sandorfr · 2017-09-21T01:15:15Z

can you explain to me how this templated url https://{companyName}.pkgs.visualstudio.com/_packaging/{FeedName}/npm/registry/ is working or point me to some documentation? I'm not familiar with this.

Mardoxx · 2017-09-21T09:46:29Z

It's not real, I should have said https://foo.pkgs.visualstudio.com/_packaging/bar/npm/registry/

sandorfr · 2017-09-21T10:15:00Z

Given your logs it seems you have selected the first option but I'm pretty sure you want the second one :

sandorfr · 2017-09-21T10:27:44Z

another possible cause is that you don't have the proper rights defined for your package feed :

see documentation: https://docs.microsoft.com/en-us/vsts/package/feeds/common-identities

Mardoxx · 2017-09-21T10:56:36Z

Tried both, Registries in my .npmrc file and Registry I select here neither work, stirll get 401 without extra task of putting auth token in to .npmrc 😢

Checked perms on the feed too, they're all correct!

I'd assume the way it automatically inserts auth stuff is lifted from this https://github.com/Microsoft/vsts-tasks/tree/master/Tasks/Npm so I'd imagine it should auth in an identical way. Npm task works fine with either Registries in my .npmrc file and Registry I select here, but this doesn't without the extra step. Weird!

I'd guess it's more likely some misconfiguration at my end, otherwise you'd have others reporting issues too?

It's also strange how it happens on the 10th or so package... Perhaps that's a concurrency thing though (not sure if the requests are performed in parallel?)

What I'll try when I get time, is after the yarn install task (which should modify .npmrc) I shall get it to output the contents .npmrc file and see if it's appended an appropriate token to it.

sandorfr · 2017-09-21T11:15:10Z

can you show me your .npmrc so I can try to reproduce your configuration on my test env?

Mardoxx · 2017-09-21T11:17:22Z

registry=https://foo.pkgs.visualstudio.com/_packaging/bar/npm/registry/
always-auth=true

Just copied and pasted from Connect to Feed>npm in VSTS

There is a new line at the bottom of the file, can't imagine that is what is throwing it off? Haven't got time to test without it at the moment, sorry.

sandorfr · 2017-09-21T12:41:49Z

Ok reproduced

sandorfr · 2017-10-04T22:53:22Z

Have you seen the the NpmAuthenticate Task, I'm quite tempted to remove the Authentication support from the task.

Happy to hear your feedback about it.

Mardoxx · 2017-10-05T08:35:29Z

@sandorfr No I haven't! Would make it rather more maintainable for you if auth was done externally by MS's task. I'm happy with that.

sandorfr · 2017-10-09T13:18:31Z

@Mardoxx, can you confirm that the issue is fixed for you with the latest version ?

Had a feedback from vsts team and they'd prefer us to keep builtin authentication support for security reasons (because the task cleans up credentials).

Mardoxx · 2017-10-09T14:53:38Z

@sandorfr Works perfectly! Thanks for your support 😄

Still coming up with those resource warnings, e.g. ##[warning]Resource file haven\'t set, can\'t find loc string for key: FoundLocalRegistries

kdelmonte · 2018-05-23T23:13:40Z

This is also happening to me... was a solution even found?

sandorfr · 2018-05-24T07:53:28Z

It’s long addressed and closed. If you have a problem please file a new detailed issue.

Megasware128 · 2018-05-24T08:09:03Z

@kdelmonte Try uninstalling and reinstalling the extensions

JackieNagel · 2018-05-31T12:20:35Z

I was getting the same 401 issues when trying to install from an internal packagefeed. The workaround for me, was to include a .npmrc file and use the built-in npm authenticate task.

All permissions should be alright and OAuth script access is turned on.

Are you sure it's working as intended?

PS. also tried to reinstall the extension without luck

sandorfr · 2018-05-31T12:46:23Z

As I said this has been resolved a long time ago. If you have a problem, it is probably something specific please open an issue with the proper logs.

sandorfr added the enhancement label May 7, 2017

sandorfr modified the milestone: 1.0.0 May 7, 2017

sandorfr added the PR Welcome ! ♥ label Jul 11, 2017

sandorfr mentioned this issue Jul 22, 2017

Feature/authentication #19

Merged

sandorfr removed the PR Welcome ! ♥ label Aug 2, 2017

sandorfr closed this as completed Aug 13, 2017

sandorfr reopened this Sep 18, 2017

sandorfr added the bug label Sep 30, 2017

sandorfr modified the milestones: 1.0.0, 2.1.0 Sep 30, 2017

sandorfr closed this as completed Oct 10, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TFS/VSTS Package Management authentication #13

TFS/VSTS Package Management authentication #13

Megasware128 commented Apr 10, 2017 •

edited

sandorfr commented May 7, 2017

waterfoul commented Jun 13, 2017

mihir0x69 commented Jul 7, 2017 •

edited

sandorfr commented Jul 11, 2017

mihir0x69 commented Jul 11, 2017

Megasware128 commented Jul 11, 2017

manu-unter commented Jul 28, 2017

sandorfr commented Aug 2, 2017

sandorfr commented Aug 13, 2017

Mardoxx commented Sep 13, 2017 •

edited

sandorfr commented Sep 13, 2017 via email

Mardoxx commented Sep 18, 2017

Mardoxx commented Sep 18, 2017 •

edited

sandorfr commented Sep 21, 2017

Mardoxx commented Sep 21, 2017

sandorfr commented Sep 21, 2017

sandorfr commented Sep 21, 2017

Mardoxx commented Sep 21, 2017 •

edited

sandorfr commented Sep 21, 2017

Mardoxx commented Sep 21, 2017 •

edited

sandorfr commented Sep 21, 2017

sandorfr commented Oct 4, 2017

Mardoxx commented Oct 5, 2017

sandorfr commented Oct 9, 2017

Mardoxx commented Oct 9, 2017

kdelmonte commented May 23, 2018

sandorfr commented May 24, 2018

Megasware128 commented May 24, 2018 •

edited

JackieNagel commented May 31, 2018

sandorfr commented May 31, 2018

TFS/VSTS Package Management authentication #13

TFS/VSTS Package Management authentication #13

Comments

Megasware128 commented Apr 10, 2017 • edited

sandorfr commented May 7, 2017

waterfoul commented Jun 13, 2017

mihir0x69 commented Jul 7, 2017 • edited

sandorfr commented Jul 11, 2017

mihir0x69 commented Jul 11, 2017

Megasware128 commented Jul 11, 2017

manu-unter commented Jul 28, 2017

sandorfr commented Aug 2, 2017

sandorfr commented Aug 13, 2017

Mardoxx commented Sep 13, 2017 • edited

sandorfr commented Sep 13, 2017 via email

Mardoxx commented Sep 18, 2017

Mardoxx commented Sep 18, 2017 • edited

sandorfr commented Sep 21, 2017

Mardoxx commented Sep 21, 2017

sandorfr commented Sep 21, 2017

sandorfr commented Sep 21, 2017

Mardoxx commented Sep 21, 2017 • edited

sandorfr commented Sep 21, 2017

Mardoxx commented Sep 21, 2017 • edited

sandorfr commented Sep 21, 2017

sandorfr commented Oct 4, 2017

Mardoxx commented Oct 5, 2017

sandorfr commented Oct 9, 2017

Mardoxx commented Oct 9, 2017

kdelmonte commented May 23, 2018

sandorfr commented May 24, 2018

Megasware128 commented May 24, 2018 • edited

JackieNagel commented May 31, 2018

sandorfr commented May 31, 2018

Megasware128 commented Apr 10, 2017 •

edited

mihir0x69 commented Jul 7, 2017 •

edited

Mardoxx commented Sep 13, 2017 •

edited

Mardoxx commented Sep 18, 2017 •

edited

Mardoxx commented Sep 21, 2017 •

edited

Mardoxx commented Sep 21, 2017 •

edited

Megasware128 commented May 24, 2018 •

edited