Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ALM Accelerator - BUG]: Update build pipeline variables error #6020

Closed
ryanhodgson1 opened this issue Jul 7, 2023 · 9 comments · Fixed by microsoft/coe-alm-accelerator-templates#310
Assignees
Labels
alm-accelerator ALM Accelerator Components and Apps bug Something isn't working

Comments

@ryanhodgson1
Copy link

ryanhodgson1 commented Jul 7, 2023

Describe the issue

I get these errors when doing an export specifically for 2 of my solutions which are called CCMSNonProdData and CCMSNZDataSourceAndProvider_V2:

For CCMSNonProdData i get:
image

2023-07-07T03:25:12.4265825Z https://sdo-online.visualstudio.com/Te Ara Manaaki/_apis/build/definitions/1460?api-version=6.0
2023-07-07T03:25:13.2589938Z Invoke-RestMethod : {"$id":"1","innerException":null,"message":"TF400898: An Internal Error Occurred. Activity Id:
2023-07-07T03:25:13.2591905Z e60f7bee-0763-4900-956c-09d559bc2a2a.","typeName":"Newtonsoft.Json.JsonReaderException,
2023-07-07T03:25:13.2592926Z Newtonsoft.Json","typeKey":"JsonReaderException","errorCode":0,"eventId":0}
2023-07-07T03:25:13.2594045Z At D:\a\1\PipelineUtils\PowerShell\update-deployment-settings.ps1:505 char:9
2023-07-07T03:25:13.2595133Z + Invoke-RestMethod $buildDefinitionResourceUrl -Method 'PUT' - ...
2023-07-07T03:25:13.2596046Z + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2023-07-07T03:25:13.2597186Z + CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebExc
2023-07-07T03:25:13.2598090Z eption
2023-07-07T03:25:13.2605351Z + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
2023-07-07T03:25:13.7887555Z ##[error]PowerShell exited with code '1'.

For CCMSNZDataSourceAndProvider_V2 i get:

image

2023-07-07T03:33:39.9415641Z https://sdo-online.visualstudio.com/Te Ara Manaaki/_apis/build/definitions/1533?api-version=6.0
2023-07-07T03:33:40.2672044Z Invoke-RestMethod : {"$id":"1","innerException":null,"message":"This request expects an object in the request body,
2023-07-07T03:33:40.2672511Z but the supplied data could not be
2023-07-07T03:33:40.2673165Z deserialized.","typeName":"Microsoft.TeamFoundation.Build.WebApi.RequestContentException,
2023-07-07T03:33:40.2674715Z Microsoft.TeamFoundation.Build2.WebApi","typeKey":"RequestContentException","errorCode":0,"eventId":3000}
2023-07-07T03:33:40.2675600Z At D:\a\1\PipelineUtils\PowerShell\update-deployment-settings.ps1:505 char:9
2023-07-07T03:33:40.2676184Z + Invoke-RestMethod $buildDefinitionResourceUrl -Method 'PUT' - ...
2023-07-07T03:33:40.2676822Z + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2023-07-07T03:33:40.2677482Z + CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebExc
2023-07-07T03:33:40.2677924Z eption
2023-07-07T03:33:40.2678376Z + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
2023-07-07T03:33:40.3562743Z ##[error]PowerShell exited with code '1'.

They seem to be failing on this line in the update-deployment-settings.ps1 file:

image

For some reason its struggling to set the variables on the build pipeline.

This seems related to this old ticket too: #2011

Expected Behavior

For the build pipeline variables to be set without issue

What component are you experiencing the issue with?

ALM Accelerator Pipelines

What solution version are you using?

1.0.20230606.1

Steps To Reproduce

No response

Anything else?

No response

@ryanhodgson1 ryanhodgson1 added alm-accelerator ALM Accelerator Components and Apps bug Something isn't working labels Jul 7, 2023
@mikefactorial
Copy link
Collaborator

Hey Ryan, tough to tell what the issue is here. You're more than likely correct that it has something to do with the payload having some sort of special character in it based on the error. Wonder if you can add

Write-Host $body

just before the call to Invoke-RestMethod and run it again so we can see the full body of the request. Feel free to send it over to me or @RajeevPentyala

@ryanhodgson1
Copy link
Author

ryanhodgson1 commented Jul 9, 2023

Hey @mikefactorial @RajeevPentyala

Looks like its an issue with any solution starting with N, the path doesnt include the N:

"path": "\CCMS Phase 2\Z Data Source and Provider",

"path": "\CCMS Phase 2\on Prod Data",

Which im guessing is from line 503:
image

@RajeevPentyala
Copy link
Collaborator

@ryanhodgson1 I tried with a solution named starting with 'N' and was able to export the solution without any issues. As @mikefactorial mentioned can you share the output of "Write-Host $body"?

@ryanhodgson1
Copy link
Author

ryanhodgson1 commented Jul 11, 2023

@RajeevPentyala the inital export works fine but after you setup the pipelines it looks for the build and in my case the build pipeline is imbedded in a directory so it has a \ in front of it and replaces \n with blank.

Here is the full body:
2023-07-09T21:44:39.1590072Z {
2023-07-09T21:44:39.1590414Z "triggers": [
2023-07-09T21:44:39.1590599Z {
2023-07-09T21:44:39.1591481Z "branchFilters": [
2023-07-09T21:44:39.1591774Z
2023-07-09T21:44:39.1591938Z ],
2023-07-09T21:44:39.1592078Z "pathFilters": [
2023-07-09T21:44:39.1592135Z
2023-07-09T21:44:39.1592262Z ],
2023-07-09T21:44:39.1592498Z "settingsSourceType": 2,
2023-07-09T21:44:39.1592640Z "batchChanges": false,
2023-07-09T21:44:39.1592794Z "maxConcurrentBuildsPerBranch": 1,
2023-07-09T21:44:39.1592964Z "triggerType": "continuousIntegration"
2023-07-09T21:44:39.1593098Z }
2023-07-09T21:44:39.1593210Z ],
2023-07-09T21:44:39.1593330Z "variables": {
2023-07-09T21:44:39.1593465Z "EnvironmentName": {
2023-07-09T21:44:39.1593619Z "value": "ccmscrmdevfulldev"
2023-07-09T21:44:39.1593757Z },
2023-07-09T21:44:39.1593878Z "ServiceConnection": {
2023-07-09T21:44:39.1594052Z "value": "https://ccmscrmdevfulldev.crm6.dynamics.com/"
2023-07-09T21:44:39.1594215Z },
2023-07-09T21:44:39.1594353Z "TriggerSolutionUpgrade": {
2023-07-09T21:44:39.1594498Z "value": "true"
2023-07-09T21:44:39.1594629Z },
2023-07-09T21:44:39.1594762Z "ServiceConnectionUrl": {
2023-07-09T21:44:39.1594931Z "value": "https://ccmscrmdevfulldev.crm6.dynamics.com/"
2023-07-09T21:44:39.1595088Z },
2023-07-09T21:44:39.1595232Z "environmentvariable.dia_NZPostAuthURL": {
2023-07-09T21:44:39.1595439Z "value": "https://oauth.nzpost.co.nz/as/token.oauth2?grant_type=client_credentials"
2023-07-09T21:44:39.1595621Z },
2023-07-09T21:44:39.1595875Z "environmentvariable.dia_NZPostAPI": {
2023-07-09T21:44:39.1596061Z "value": "https://api.nzpost.co.nz/addresschecker/1.0/"
2023-07-09T21:44:39.1596218Z },
2023-07-09T21:44:39.1596372Z "environmentvariable.dia_NZPostParcelAPI": {
2023-07-09T21:44:39.1596553Z "value": "https://api.nzpost.co.nz/parceladdress/2.0/"
2023-07-09T21:44:39.1596709Z },
2023-07-09T21:44:39.1596841Z "UserSettingId": {
2023-07-09T21:44:39.1597278Z "value": "7d14a548-2816-ee11-9cbe-00224891cdac"
2023-07-09T21:44:39.1597433Z }
2023-07-09T21:44:39.1597581Z },
2023-07-09T21:44:39.1597711Z "properties": {
2023-07-09T21:44:39.1597763Z
2023-07-09T21:44:39.1597869Z },
2023-07-09T21:44:39.1597983Z "tags": [
2023-07-09T21:44:39.1598031Z
2023-07-09T21:44:39.1598136Z ],
2023-07-09T21:44:39.1598260Z "_links": {
2023-07-09T21:44:39.1598376Z "self": {
2023-07-09T21:44:39.1598720Z "href": "https://sdo-online.visualstudio.com/b1047096-86f6-46e2-ba59-20ad8913e286/_apis/build/Definitions/1460?revision=4"
2023-07-09T21:44:39.1598919Z },
2023-07-09T21:44:39.1599042Z "web": {
2023-07-09T21:44:39.1599440Z "href": "https://sdo-online.visualstudio.com/b1047096-86f6-46e2-ba59-20ad8913e286/_build/definition?definitionId=1460"
2023-07-09T21:44:39.1599633Z },
2023-07-09T21:44:39.1599742Z "editor": {
2023-07-09T21:44:39.1600089Z "href": "https://sdo-online.visualstudio.com/b1047096-86f6-46e2-ba59-20ad8913e286/_build/designer?id=1460\u0026_a=edit-build-definition"
2023-07-09T21:44:39.1600300Z },
2023-07-09T21:44:39.1600417Z "badge": {
2023-07-09T21:44:39.1600733Z "href": "https://sdo-online.visualstudio.com/b1047096-86f6-46e2-ba59-20ad8913e286/_apis/build/status/1460"
2023-07-09T21:44:39.1600916Z }
2023-07-09T21:44:39.1601029Z },
2023-07-09T21:44:39.1601178Z "jobAuthorizationScope": "projectCollection",
2023-07-09T21:44:39.1601326Z "process": {
2023-07-09T21:44:39.1601623Z "yamlFilename": "/CCMSNZDataSourceAndProvider_V2/deploy-ccmscrmdevfulldev-CCMSNZDataSourceAndProvider_V2.yml",
2023-07-09T21:44:39.1601816Z "type": 2
2023-07-09T21:44:39.1601932Z },
2023-07-09T21:44:39.1602052Z "repository": {
2023-07-09T21:44:39.1602296Z "id": "2407971d-1577-46ee-89fe-3999407511b5",
2023-07-09T21:44:39.1602455Z "type": "TfsGit",
2023-07-09T21:44:39.1602679Z "name": "ccms-nz-data-source-and-provider",
2023-07-09T21:44:39.1602990Z "url": "https://sdo-online.visualstudio.com/Te%20Ara%20Manaaki/_git/ccms-nz-data-source-and-provider",
2023-07-09T21:44:39.1603197Z "defaultBranch": "refs/heads/main",
2023-07-09T21:44:39.1603330Z "clean": null,
2023-07-09T21:44:39.1603478Z "checkoutSubmodules": false
2023-07-09T21:44:39.1603613Z },
2023-07-09T21:44:39.1603739Z "quality": "definition",
2023-07-09T21:44:39.1603870Z "authoredBy": {
2023-07-09T21:44:39.1604013Z "displayName": "Ryan Hodgson",
2023-07-09T21:44:39.1604395Z "url": "https://spsprodeau1.vssps.visualstudio.com/Ae21ff30a-0192-40a2-8ed7-ce8808203b39/_apis/Identities/0de2d1d5-d861-66b6-84ff-0daec89fd822",
2023-07-09T21:44:39.1604669Z "_links": {
2023-07-09T21:44:39.1604791Z "avatar": {
2023-07-09T21:44:39.1605140Z "href": "https://sdo-online.visualstudio.com/_apis/GraphProfile/MemberAvatars/aad.MGRlMmQxZDUtZDg2MS03NmI2LTg0ZmYtMGRhZWM4OWZkODIy"
2023-07-09T21:44:39.1605349Z }
2023-07-09T21:44:39.1605465Z },
2023-07-09T21:44:39.1605700Z "id": "0de2d1d5-d861-66b6-84ff-0daec89fd822",
2023-07-09T21:44:39.1605878Z "uniqueName": "Ryan.Hodgson@dia.govt.nz",
2023-07-09T21:44:39.1606246Z "imageUrl": "https://sdo-online.visualstudio.com/_apis/GraphProfile/MemberAvatars/aad.MGRlMmQxZDUtZDg2MS03NmI2LTg0ZmYtMGRhZWM4OWZkODIy",
2023-07-09T21:44:39.1606505Z "descriptor": "aad.MGRlMmQxZDUtZDg2MS03NmI2LTg0ZmYtMGRhZWM4OWZkODIy"
2023-07-09T21:44:39.1606665Z },
2023-07-09T21:44:39.1606786Z "drafts": [
2023-07-09T21:44:39.1606837Z
2023-07-09T21:44:39.1606945Z ],
2023-07-09T21:44:39.1607060Z "queue": {
2023-07-09T21:44:39.1607185Z "_links": {
2023-07-09T21:44:39.1607303Z "self": {
2023-07-09T21:44:39.1607575Z "href": "https://sdo-online.visualstudio.com/_apis/build/Queues/318"
2023-07-09T21:44:39.1607742Z }
2023-07-09T21:44:39.1607920Z },
2023-07-09T21:44:39.1608041Z "id": 318,
2023-07-09T21:44:39.1608293Z "url": "https://sdo-online.visualstudio.com/_apis/build/Queues/318",
2023-07-09T21:44:39.1608461Z "pool": null
2023-07-09T21:44:39.1608582Z },
2023-07-09T21:44:39.1608690Z "id": 1460,
2023-07-09T21:44:39.1608934Z "name": "deploy-ccmscrmdevfulldev-CCMSNZDataSourceAndProvider_V2",
2023-07-09T21:44:39.1609292Z "url": "https://sdo-online.visualstudio.com/b1047096-86f6-46e2-ba59-20ad8913e286/_apis/build/Definitions/1460?revision=4",
2023-07-09T21:44:39.1609509Z "uri": "vstfs:///Build/Definition/1460",
2023-07-09T21:44:39.1609683Z "path": "\CCMS Phase 2\Z Data Source and Provider",
2023-07-09T21:44:39.1609839Z "type": "build",
2023-07-09T21:44:39.1609973Z "queueStatus": "enabled",
2023-07-09T21:44:39.1610105Z "revision": 4,
2023-07-09T21:44:39.1610346Z "createdDate": "2023-06-29T03:13:57.61Z",
2023-07-09T21:44:39.1610499Z "project": {
2023-07-09T21:44:39.1610731Z "id": "b1047096-86f6-46e2-ba59-20ad8913e286",
2023-07-09T21:44:39.1610891Z "name": "Te Ara Manaaki",
2023-07-09T21:44:39.1611200Z "url": "https://sdo-online.visualstudio.com/_apis/projects/b1047096-86f6-46e2-ba59-20ad8913e286",
2023-07-09T21:44:39.1611401Z "state": "wellFormed",
2023-07-09T21:44:39.1611540Z "revision": 1267,
2023-07-09T21:44:39.1611681Z "visibility": "private",
2023-07-09T21:44:39.1611918Z "lastUpdateTime": "2023-07-05T00:23:45.103Z"
2023-07-09T21:44:39.1612061Z }
2023-07-09T21:44:39.1612250Z }

@RajeevPentyala RajeevPentyala self-assigned this Jul 11, 2023
@RajeevPentyala
Copy link
Collaborator

@ryanhodgson1 Issue is reproducible when the 'Repository Name' starts with letter N. We will work on the fix. You might want to track the PR if you would like to see the fix before the release.

@ryanhodgson1
Copy link
Author

@RajeevPentyala thats odd cause my repo name starts with a c at "ccms-nz-data-source-and-provider".

For me the only time the issue happens is when the build pipeline path has \n in it and removes it:
image

@RajeevPentyala
Copy link
Collaborator

@ryanhodgson1 Fix specified in this PR : https://github.com/microsoft/coe-alm-accelerator-templates/pull/310 should fix the issue. Can you please try by updating your 'update-deployment-settings.ps1' file with the fix from the PR.

@ryanhodgson1
Copy link
Author

Thanks @RajeevPentyala

I applied that and tested it and its working for me now, thanks for your help!

@CoEStarterKitBot
Copy link
Collaborator

@ryanhodgson1 This has been fixed in the latest release. Please install the latest version of the toolkit following the instructions for installing updates. Note that if you do not remove the unmanaged layers as described there you will not receive updates from us.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
alm-accelerator ALM Accelerator Components and Apps bug Something isn't working
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

4 participants