diff --git a/.gitignore b/.gitignore index 5a17d1b3..7bf3c0dc 100644 --- a/.gitignore +++ b/.gitignore @@ -332,6 +332,8 @@ ASALocalRun/ # MFractors (Xamarin productivity tool) working folder .mfractor/ + +## Specific to this repo secrets/ src/aggregator-cli/FunctionRuntime.zip *.exe diff --git a/README.md b/README.md index c7eb93c3..daf2a8b2 100644 --- a/README.md +++ b/README.md @@ -1,32 +1,35 @@ # aggregator-cli -![](https://tfsaggregator.visualstudio.com/_apis/public/build/definitions/1cca877b-3e26-4880-b5b8-79e4b10fbfb4/16/badge) +[![Build status: master](https://dev.azure.com/TfsAggregator/Aggregator3/_apis/build/status/Aggregator3-CI?branchName=master)](https://dev.azure.com/TfsAggregator/Aggregator3/_build/latest?definitionId=16) +[![Build status](https://dev.azure.com/TfsAggregator/Aggregator3/_apis/build/status/Aggregator3-CI)](https://dev.azure.com/TfsAggregator/Aggregator3/_build/latest?definitionId=16) + This is the successor to TFS Aggregator. The current Server Plugin version (2.x) will be maintained to support TFS. The Web Service flavor will be discontinued in favor of this (its deployment and configuration was too complex for most users). -The main scenario for Aggregator (3.x) is supporting VSTS and the cloud scenario. It will work for TFS as long as it is reachable from Internet. +The main scenario for Aggregator (3.x) is supporting Azure DevOps and the cloud scenario. It will work for TFS as long as it is reachable from Internet. -> **This is an early version (alpha)**: we might change verbs and rule language before the final release! +> **This is an early version (beta)**: we might change verbs and rule language before the final release! +*Note*: The documentation is limited to this page and the content of the `doc` folder. ## Major features -- use of new REST API +- use of new Azure DevOps REST API - simple deployment via CLI tool -- similar model for Rules +- Rule object model similar to v2 ## Planned features - Support for Deployment Slots for blue/green-style deployments - OAuth support to avoid maintain access tokens -- Additional VSTS events -- Additional VSTS objects +- Additional Azure DevOps events +- Additional Azure DevOps objects ## How it works An Aggregator Instance is an Azure Function Application in its own Resource Group, -sharing the same VSTS credential. You can have only one Application per Resource Group. +sharing the same Azure DevOps credential. You can have only one Application per Resource Group. If the Resource Group does not exists, Aggregator will try to create it. *Note*: The Instance name must be **unique** amongst all Aggregator Instances in Azure! @@ -36,28 +39,28 @@ To work, it uses an Aggregator Runtime. Aggregator checks its latest GitHub Release to ensure that Aggregator Runtime is up-to-date before uploading the function. *Note*: We use [Azure Functions Runtime](https://docs.microsoft.com/en-us/azure/azure-functions/functions-versions) 2.0 for C# which is still in Preview. -An Aggregator Mapping is a VSTS Service Hook for a specific work item event that invokes an Aggregator Rule i.e. the Azure Function hosting the Rule code. VSTS saves the Azure Function Key in the Service Hook configuration. +An Aggregator Mapping is a Azure DevOps Service Hook for a specific work item event that invokes an Aggregator Rule i.e. the Azure Function hosting the Rule code. Azure DevOps saves the Azure Function Key in the Service Hook configuration. -You can deploy the same Rule in many Instances or map the same VSTS event to many Rules: it is up how to organize. +You can deploy the same Rule in many Instances or map the same Azure DevOps event to many Rules: it is up to you choosing the best way to organize. ## Authentication You must instruct Aggregator which credential to use. -To do this, run the `login.azure` and `login.vsts` commands. +To do this, run the `login.azure` and `login.ado` commands. -To create the credentials, you need an Azure Service Principal and a VSTS Personal Access Token. +To create the credentials, you need an Azure Service Principal and a Azure DevOps Personal Access Token. -These documents will guide you +These documents will guide you in creating the credentials * [Use portal to create an Azure Active Directory application and service principal that can access resources](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal) -* [Create personal access tokens to authenticate access](https://docs.microsoft.com/en-us/vsts/organizations/accounts/use-personal-access-tokens-to-authenticate?view=vsts#create-personal-access-tokens-to-authenticate-access). +* [Create personal access tokens to authenticate access](https://docs.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate). Logon credentials are stored locally and expire after 2 hours. -The PAT is stored in the Azure Function settings: **whoever has access to the Resource Group can read it!** +The PAT is also stored in the Azure Function settings: **whoever has access to the Resource Group can read it!** -The Service Principal must have Contributor permission to the Azure Subscription. -In alternative, pre-create the `aggregator-` Resource Group in Azure and give the service account Contributor permission to the Resource Group. -The `instance` parameter prefixes `aggregator-` to identify the Resource Group. +The Service Principal must have Contributor permission to the Azure Subscription or, in alternative, pre-create the Resource Group in Azure and give the service account Contributor permission to the Resource Group. +![Permission on existing Resource Group](doc/contributor-on-rg.png) +If you go this route, remember add the `--resourceGroup` to all commands requiring an instance, otherwise the `instance` parameter adds an `aggregator-` prefixe to find the Resource Group. ## Usage @@ -71,24 +74,29 @@ To run Aggregator use Verb | Use --------------------|---------------------------------------- logon.azure | Logon into Azure. -logon.vsts | Logon into Visual Studio Team Services. +logon.ado | Logon into Azure DevOps. list.instances | Lists Aggregator instances. install.instance | Creates a new Aggregator instance in Azure. uninstall.instance | Destroy an Aggregator instance in Azure. +configure.instance | Configures an existing Aggregator instance. list.rules | List the rule in existing Aggregator instance in Azure. add.rule | Add a rule to existing Aggregator instance in Azure. remove.rule | Remove a rule from existing Aggregator instance in Azure. -configure.rule | Change a rule configuration and code. -list.mappings | Lists mappings from existing VSTS Projects to Aggregator Rules. -map.rule | Maps an Aggregator Rule to existing VSTS Projects. -unmap.rule | Unmaps an Aggregator Rule from a VSTS Project. +configure.rule | Change a rule configuration. +update.rule | Update a rule code and/or runtime. +invoke.rule | Executes a rule locally or in an existing Aggregator instance. +list.mappings | Lists mappings from existing Azure DevOps Projects to Aggregator Rules. +map.rule | Maps an Aggregator Rule to existing Azure DevOps Projects. +unmap.rule | Unmaps an Aggregator Rule from a Azure DevOps Project. help | Display more information on a specific command. version | Display version information. -## Examples +You can see a few Command examples in [Sample Aggregator CLI usage](doc/command-examples.md), see also [Manual Tests](doc/test-matrix.md). + +## Rule language -You can see a few Command examples in [Sample Aggregator CLI usage](doc/command-examples.md). -You can see a few Rule examples in [Rule Examples](doc/rule-examples.md). +See [Rule Language](doc/rule-language.md) for a list of objects and properties to use. +For examples see [Rule Examples](doc/rule-examples.md). ## Troubleshooting diff --git a/doc/build-and-test.md b/doc/build-and-test.md new file mode 100644 index 00000000..9a6923bb --- /dev/null +++ b/doc/build-and-test.md @@ -0,0 +1,23 @@ +# Build +Building locally requires +- Visual Studio 2017 15.8.9 +- Azure Functions and Web Jobs Tools + +# Debug + +## Custom/development Aggregator runtime +In Visual Studio, `src\aggregator-function\Directory.Build.targets` will automatically package and copy the runtime needed by CLI. +You might have to change the version number in `src\aggregator-function\aggregator-manifest.ini` to force your local version. + +You can also use the *Pack* right-click command on the `aggregator-function` project and make sure to copy the created zip into your CLI directory so it uploads the correct one when creating an instance. + +## CLI +Set `aggregator-cli` as Start-up project +Use the Visual Studio Project properties to set the Command line arguments + +## Runtime +Set `aggregator-function` as Start-up project +Use **Postman** or similar tool to send the request at http://localhost:7071/api/name_of_rule + +# Integration tests +`git update-index --assume-unchanged src/integrationtests-cli/logon-data.json` and edit the file content diff --git a/doc/command-examples.md b/doc/command-examples.md index e48f2ac1..538ff2d7 100644 --- a/doc/command-examples.md +++ b/doc/command-examples.md @@ -5,40 +5,60 @@ Remember that the Instance name must be unique in Azure. ``` # logon logon.azure --subscription 9c********08 --client 5a********b6 --password P@assword1 --tenant 3c********1d -logon.vsts --url https://someaccount.visualstudio.com --mode PAT --token 2**************************************q +logon.ado --url https://someaccount.visualstudio.com --mode PAT --token 2**************************************q # create an Azure Function Application install.instance --verbose --name my1 --location westeurope +install.instance --name my3 --resourceGroup myRG1 --location westeurope --requiredVersion latest +# search instances in the Azure subscription list.instances +# search instances in the Azure Resource Group +list.instances --resourceGroup myRG1 -# create three Azure Functions +# create two Azure Functions add.rule --verbose --instance my1 --name test1 --file test\test1.rule add.rule --verbose --instance my1 --name test2 --file test\test2.rule -add.rule --verbose --instance my1 --name test3 --file test\test3.rule list.rules --verbose --instance my1 -# adds two Service Hook to VSTS, each invoking a different rule +# create Azure Function in specified App and Resource Group +add.rule --verbose --instance my3 --resourceGroup myRG1 --name test3 --file test\test3.rule + +# adds two Service Hook to Azure DevOps, each invoking a different rule map.rule --verbose --project SampleProject --event workitem.created --instance my1 --rule test1 map.rule --verbose --project SampleProject --event workitem.updated --instance my1 --rule test2 +map.rule --verbose --project SampleProject --event workitem.created --instance my3 --resourceGroup myRG1 --rule test3 + + list.mappings --verbose --instance my1 +list.mappings --verbose --project SampleProject +list.mappings --instance my1 --project SampleProject -# disable a rule +# disable an existing rule configure.rule --verbose --instance my1 --name test1 --disable # re-enable a rule configure.rule --verbose --instance my1 --name test1 --enable -# update the code of a rule -configure.rule --verbose --instance my1 --name test --update test.rule +# update the code and runtime of a rule +update.rule --verbose --instance my1 --name test1 --file test1.rule --requiredVersion 0.4.0 +update.rule --verbose --instance my3 --resourceGroup myRG1 --name test3 --file test\test3.rule -# updates the VSTS credential stored by the rules -configure.instance --authentication +# updates the Azure DevOps credential stored in Azure Function and used by rules to connect back +configure.instance --name my1 --location westeurope --authentication +configure.instance --name my3 --resourceGroup myRG1 --location westeurope --authentication -# remove a Service Hook from VSTS +# remove a Service Hook from Azure DevOps unmap.rule --verbose --event workitem.created --instance my1 --rule test1 +unmap.rule --verbose --event workitem.updated --project SampleProject --instance my1 --rule test2 +unmap.rule --verbose --project SampleProject --event workitem.created --instance my3 --resourceGroup myRG1 --rule test3 -# deletes two Azure Functions +# deletes an Azure Function and all Service Hooks referring to it remove.rule --verbose --instance my1 --name test1 -remove.rule --verbose --instance my1 --name test2 +remove.rule --verbose --instance my3 --resourceGroup myRG1 --name test3 + +# delete the Azure Function Application leaving the Service Hooks in place +uninstall.instance --name my1 --location westeurope --dont-remove-mappings +# delete the Azure Function Application and any Service Hooks referring to it +uninstall.instance --verbose --name my3 --resourceGroup myRG1 --location westeurope -# delete the Azure Function Application -uninstall.instance --verbose --name my1 --location westeurope +# run rule locally, no change is sent to Azure DevOps +invoke.rule --dryrun --project SampleProject --event workitem.created --workItemId 14 --local --source test\test2.rule ``` \ No newline at end of file diff --git a/doc/contributor-on-rg.png b/doc/contributor-on-rg.png new file mode 100644 index 00000000..c6a41bfe Binary files /dev/null and b/doc/contributor-on-rg.png differ diff --git a/doc/rule-examples.md b/doc/rule-examples.md index e369a01a..54f50923 100644 --- a/doc/rule-examples.md +++ b/doc/rule-examples.md @@ -12,14 +12,14 @@ $"Hello { self.WorkItemType } #{ self.Id } - { self.Title }!" This is more similar to classic TFS Aggregator. It move a parent work item to Closed state, if all children are closed. -The major difference is the navigation: `Parent` and `Children` properties do not returns work items but relation. You have to explicitly query VSTS to retrieve the referenced work items. +The major difference is the navigation: `Parent` and `Children` properties do not returns work items but relation. You have to explicitly query Azure DevOps to retrieve the referenced work items. ``` string message = ""; -if (self.Parent != null) +var parent = self.Parent; +if (parent != null) { - var parent = store.GetWorkItem(self.Parent); - var children = store.GetWorkItems(parent.Children); + var children = parent.Children; if (children.All(c => c.State == "Closed")) { parent.State = "Closed"; @@ -41,3 +41,13 @@ return message; ``` return self.PreviousRevision.PreviousRevision.Description; ``` + +# Create new Work Item +``` +var parent = self; +var newChild = store.NewWorkItem("Task"); +newChild.Title = "Brand new child"; +parent.Relations.AddChild(newChild); + +return parent.Title; +``` diff --git a/doc/rule-language.md b/doc/rule-language.md new file mode 100644 index 00000000..aa9474fa --- /dev/null +++ b/doc/rule-language.md @@ -0,0 +1,122 @@ +# Directives + +`.lang=C#` +`.language=Csharp` + + + +# WorkItem Object + +## Revisions +Navigate to previous versions of the work item. +`WorkItem PreviousRevision` +`IEnumerable Revisions` + +## Relations +Navigate to related work items. +`IEnumerable RelationLinks` +`WorkItemRelationCollection Relations` +`IEnumerable ChildrenLinks` +`IEnumerable Children` +`WorkItemRelation ParentLink` +`WorkItem Parent` + +## Links +Navigate links to non-workitem objects. +`IEnumerable RelatedLinks` +`IEnumerable Hyperlinks` +`int ExternalLinkCount` +`int HyperLinkCount` +`int RelatedLinkCount` + +## Fields +Data fields of the work item. +`WorkItemId Id` Read-only. +`int Rev` Read-only. +`string Url` Read-only. +`string WorkItemType` Read-only. +`string State` +`int AreaId` +`string AreaPath` +`IdentityRef AssignedTo` +`IdentityRef AuthorizedAs` +`IdentityRef ChangedBy` +`DateTime? ChangedDate` +`IdentityRef CreatedBy` +`DateTime? CreatedDate` +`string Description` +`string History` +`int IterationId` +`string IterationPath` +`string Reason` +`DateTime? RevisedDate` +`DateTime? AuthorizedDate` +`string TeamProject` +`string Tags` +`string Title` +`double Watermark` Read-only. +`bool IsDeleted` Read-only. +`bool IsReadOnly` Read-only, returns `true` if work item cannot be modified. +`bool IsNew` Read-only. +`bool IsDirty` Read-only, returns `true` if work item changed after retrieval. +`object this[string field]` access to non-core fields. + +## Attachments +`int AttachedFileCount` + + + +# WorkItemStore Object +Retrival, creation and removal of work items. + +`WorkItem GetWorkItem(int id)` +`WorkItem GetWorkItem(WorkItemRelation item)` + +`IList GetWorkItems(IEnumerable ids)` +`IList GetWorkItems(IEnumerable collection)` + +`WorkItemWrapper NewWorkItem(string workItemType)` + + + +# WorkItemRelationCollection +Navigate and modify related objects. + +`IEnumerator GetEnumerator()` +`Add(WorkItemRelation item)` +`AddChild(WorkItemWrapper child)` +`AddParent(WorkItemWrapper parent)` +`AddLink(string type, string url, string comment)` +`AddHyperlink(string url, string comment = null)` +`AddRelatedLink(WorkItem item, string comment = null)` +`AddRelatedLink(string url, string comment = null)` +`Clear()` +`bool Contains(WorkItemRelation item)` +`bool Remove(WorkItemRelation item)` +`int Count` +`bool IsReadOnly` + + + +# WorkItemRelation + +`string Title` +`string Rel` +`string Url` +`IDictionary Attributes` + + + +# IdentityRef +Represents a User identity. + +`string DirectoryAlias` +`string DisplayName` +`string Id` +`string ImageUrl` +`bool Inactive` +`bool IsAadIdentity` +`bool IsContainer` +`string ProfileUrl` +`string UniqueName` +`string Url` diff --git a/doc/service-principal.png b/doc/service-principal.png new file mode 100644 index 00000000..5c857791 Binary files /dev/null and b/doc/service-principal.png differ diff --git a/doc/test-matrix.md b/doc/test-matrix.md new file mode 100644 index 00000000..0bf426c8 --- /dev/null +++ b/doc/test-matrix.md @@ -0,0 +1,56 @@ +| Scenario 1 | Pass/Fail | Expected | +|---------------------------------------------------------------------------------------------------------------|:---------:|----------------------------------------------------------------------| +| logon.azure SUBSCRIPTION_OWNER | 0.4.0 | | +| logon.ado ACCOUNT_ADMIN | 0.4.0 | | +| install.instance --name my1 --location westeurope | 0.4.0 | Creates Resource Group and Function App | +| list.instances | 0.4.0 | | +| list.instances --resourceGroup aggregator-my1 | 0.4.0 | | +| add.rule --instance my1 --name test1 --file test\test1.rule | 0.4.0 | Add Function | +| add.rule --instance my1 --name test2 --file test\test2.rule | 0.4.0 | Add Function | +| list.rules --instance my1 | 0.4.0 | | +| map.rule --project SampleProject --event workitem.created --instance my1 --rule test1 | 0.4.0 | Add Subscription to AzDO | +| map.rule --project SampleProject --event workitem.updated --instance my1 --rule test2 | 0.4.0 | Add Subscription to AzDO | +| list.mappings --instance my1 | 0.4.0 | | +| list.mappings --project SampleProject | 0.4.0 | | +| list.mappings --instance my1 --project SampleProject | 0.4.0 | | +| configure.rule --instance my1 --name test1 --disable | 0.4.0 | Function is disabled | +| configure.rule --instance my1 --name test1 --enable | 0.4.0 | Function is enabled | +| update.rule --instance my1 --name test1 --file test\test1.rule --requiredVersion 99.99.99 | 0.4.0 | Must fail because version does not exists | +| update.rule --instance my1 --name test1 --file test\test1.rule --requiredVersion 0.3.3 | 0.4.0 | Uploads only the rule | +| configure.instance --name my1 --location westeurope --authentication | 0.4.0 | | +| unmap.rule --event workitem.created --instance my1 --rule test1 | 0.4.0 | Subscription removed | +| unmap.rule --event workitem.updated --project SampleProject --instance my1 --rule test2 | 0.4.0 | Subscription removed | +| remove.rule --instance my1 --name test1 | 0.4.0 | Function removed | +| uninstall.instance --name my1 --location westeurope --dont-remove-mappings | 0.4.0 | Remove the entire Resource Group leaving the Subscriptions in AzDO | + +| Scenario 2 | Pass/Fail | Expected | +|---------------------------------------------------------------------------------------------------------------|:---------:|----------------------------------------------------------------------| +| logon.azure RG_CONTRIBUTOR | 0.4.0 | | +| logon.ado ACCOUNT_ADMIN | 0.4.0 | | +| install.instance --name my3 --resourceGroup myRG1 --location westeurope --requiredVersion latest | 0.4.0 | Add Function App to existing Resource Group with most recent runtime | +| list.instances --resourceGroup myRG1 | 0.4.0 | | +| add.rule --instance my3 --resourceGroup myRG1 --name test3 --file test\test3.rule | 0.4.0 | Add Function | +| list.rules --instance my3 --resourceGroup myRG1 | 0.4.0 | | +| map.rule --project SampleProject --event workitem.created --instance my3 --resourceGroup myRG1 --rule test3 | 0.4.0 | Add Subscription to AzDO | +| list.mappings --instance my3 --resourceGroup myRG1 | 0.4.0 | | +| list.mappings --project SampleProject | 0.4.0 | | +| list.mappings --instance my3 --resourceGroup myRG1 --project SampleProject | 0.4.0 | | +| configure.rule --instance my3 --resourceGroup myRG1 --name test3 --disable | 0.4.0 | Function is disabled | +| configure.rule --instance my3 --resourceGroup myRG1 --name test3 --enable | 0.4.0 | Function is enabled | +| update.rule --instance my3 --resourceGroup myRG1 --name test3 --file test\test3b.rule | 0.4.0 | Function code has additional line | +| configure.instance --name my3 --resourceGroup myRG1 --location westeurope --authentication | 0.4.0 | | +| unmap.rule --project SampleProject --event workitem.created --instance my3 --resourceGroup myRG1 --rule test3 | 0.4.0 | Subscription removed | +| remove.rule --instance my3 --resourceGroup myRG1 --name test3 | 0.4.0 | Function removed | +| uninstall.instance --name my3 --resourceGroup myRG1 --location westeurope | 0.4.0 | Function app removed | + +| Scenario 3 | Pass/Fail | Expected | +|---------------------------------------------------------------------------------------------------------------|:---------:|----------------------------------------------------------------------| +| logon.azure RG_CONTRIBUTOR | 0.4.0 | | +| logon.ado ACCOUNT_ADMIN | 0.4.0 | | +| install.instance --name my3 --resourceGroup myRG1 --location westeurope | 0.4.0 | Add Function App to existing Resource Group | +| list.instances --resourceGroup myRG1 | 0.4.0 | | +| add.rule --instance my3 --resourceGroup myRG1 --name test3 --file test\test3.rule | 0.4.0 | Add Function | +| list.rules --instance my3 --resourceGroup myRG1 | 0.4.0 | | +| map.rule --project SampleProject --event workitem.created --instance my3 --resourceGroup myRG1 --rule test3 | 0.4.0 | Add Subscription to AzDO | +| list.mappings --instance my3 --resourceGroup myRG1 | 0.4.0 | | +| uninstall.instance --name my3 --resourceGroup myRG1 --location westeurope | 0.4.0 | Removes the Function and the Subscriptions | \ No newline at end of file diff --git a/src/aggregator-cli.sln b/src/aggregator-cli.sln index a736aa6d..4b4df8a4 100644 --- a/src/aggregator-cli.sln +++ b/src/aggregator-cli.sln @@ -3,24 +3,30 @@ Microsoft Visual Studio Solution File, Format Version 12.00 # Visual Studio 15 VisualStudioVersion = 15.0.27703.2026 MinimumVisualStudioVersion = 10.0.40219.1 -Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "aggregator-core", "aggregator-core\aggregator-core.csproj", "{B024DF56-E474-4A1A-BEA9-03A87F6D00F3}" +Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "aggregator-shared", "aggregator-shared\aggregator-shared.csproj", "{B024DF56-E474-4A1A-BEA9-03A87F6D00F3}" EndProject -Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "aggregator-cli", "aggregator-cli\aggregator-cli.csproj", "{FFB65D93-8193-4DA7-820B-3CD4D1872ABA}" +Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "aggregator-ruleng", "aggregator-ruleng\aggregator-ruleng.csproj", "{87B7E8EE-7C3B-450A-A319-18E2BA7D35D4}" EndProject -Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "unittests-core", "unittests-core\unittests-core.csproj", "{4D4361EC-F361-4E63-8CAF-6913EF42ED6D}" +Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "aggregator-cli", "aggregator-cli\aggregator-cli.csproj", "{FFB65D93-8193-4DA7-820B-3CD4D1872ABA}" EndProject -Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "unittests-cli", "unittests-cli\unittests-cli.csproj", "{2DFEB5D5-A70E-4C43-B180-586FAA52E2DD}" +Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "integrationtests-cli", "integrationtests-cli\integrationtests-cli.csproj", "{D1FB85EA-CEEC-4F02-97E9-27D6C8AAE760}" EndProject Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "aggregator-function", "aggregator-function\aggregator-function.csproj", "{6ADD44D3-3B38-4D2F-BBFF-242E5F2E787C}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "docs", "docs", "{1A84F5C1-ADA4-4DD1-A0D9-03635DF9FC56}" ProjectSection(SolutionItems) = preProject + ..\doc\build-and-test.md = ..\doc\build-and-test.md ..\doc\command-examples.md = ..\doc\command-examples.md + ..\doc\contributor-on-rg.png = ..\doc\contributor-on-rg.png ..\doc\log-streaming-from-azure-portal.png = ..\doc\log-streaming-from-azure-portal.png ..\README.md = ..\README.md ..\doc\rule-examples.md = ..\doc\rule-examples.md + ..\doc\rule-language.md = ..\doc\rule-language.md + ..\doc\test-matrix.md = ..\doc\test-matrix.md EndProjectSection EndProject +Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "unittests-ruleng", "unittests-ruleng\unittests-ruleng.csproj", "{19AE3176-D15F-4A2B-AB49-1B9DE6EA3EE4}" +EndProject Global GlobalSection(SolutionConfigurationPlatforms) = preSolution Debug|Any CPU = Debug|Any CPU @@ -31,22 +37,26 @@ Global {B024DF56-E474-4A1A-BEA9-03A87F6D00F3}.Debug|Any CPU.Build.0 = Debug|Any CPU {B024DF56-E474-4A1A-BEA9-03A87F6D00F3}.Release|Any CPU.ActiveCfg = Release|Any CPU {B024DF56-E474-4A1A-BEA9-03A87F6D00F3}.Release|Any CPU.Build.0 = Release|Any CPU + {87B7E8EE-7C3B-450A-A319-18E2BA7D35D4}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {87B7E8EE-7C3B-450A-A319-18E2BA7D35D4}.Debug|Any CPU.Build.0 = Debug|Any CPU + {87B7E8EE-7C3B-450A-A319-18E2BA7D35D4}.Release|Any CPU.ActiveCfg = Release|Any CPU + {87B7E8EE-7C3B-450A-A319-18E2BA7D35D4}.Release|Any CPU.Build.0 = Release|Any CPU {FFB65D93-8193-4DA7-820B-3CD4D1872ABA}.Debug|Any CPU.ActiveCfg = Debug|Any CPU {FFB65D93-8193-4DA7-820B-3CD4D1872ABA}.Debug|Any CPU.Build.0 = Debug|Any CPU {FFB65D93-8193-4DA7-820B-3CD4D1872ABA}.Release|Any CPU.ActiveCfg = Release|Any CPU {FFB65D93-8193-4DA7-820B-3CD4D1872ABA}.Release|Any CPU.Build.0 = Release|Any CPU - {4D4361EC-F361-4E63-8CAF-6913EF42ED6D}.Debug|Any CPU.ActiveCfg = Debug|Any CPU - {4D4361EC-F361-4E63-8CAF-6913EF42ED6D}.Debug|Any CPU.Build.0 = Debug|Any CPU - {4D4361EC-F361-4E63-8CAF-6913EF42ED6D}.Release|Any CPU.ActiveCfg = Release|Any CPU - {4D4361EC-F361-4E63-8CAF-6913EF42ED6D}.Release|Any CPU.Build.0 = Release|Any CPU - {2DFEB5D5-A70E-4C43-B180-586FAA52E2DD}.Debug|Any CPU.ActiveCfg = Debug|Any CPU - {2DFEB5D5-A70E-4C43-B180-586FAA52E2DD}.Debug|Any CPU.Build.0 = Debug|Any CPU - {2DFEB5D5-A70E-4C43-B180-586FAA52E2DD}.Release|Any CPU.ActiveCfg = Release|Any CPU - {2DFEB5D5-A70E-4C43-B180-586FAA52E2DD}.Release|Any CPU.Build.0 = Release|Any CPU + {D1FB85EA-CEEC-4F02-97E9-27D6C8AAE760}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {D1FB85EA-CEEC-4F02-97E9-27D6C8AAE760}.Debug|Any CPU.Build.0 = Debug|Any CPU + {D1FB85EA-CEEC-4F02-97E9-27D6C8AAE760}.Release|Any CPU.ActiveCfg = Release|Any CPU + {D1FB85EA-CEEC-4F02-97E9-27D6C8AAE760}.Release|Any CPU.Build.0 = Release|Any CPU {6ADD44D3-3B38-4D2F-BBFF-242E5F2E787C}.Debug|Any CPU.ActiveCfg = Debug|Any CPU {6ADD44D3-3B38-4D2F-BBFF-242E5F2E787C}.Debug|Any CPU.Build.0 = Debug|Any CPU {6ADD44D3-3B38-4D2F-BBFF-242E5F2E787C}.Release|Any CPU.ActiveCfg = Release|Any CPU {6ADD44D3-3B38-4D2F-BBFF-242E5F2E787C}.Release|Any CPU.Build.0 = Release|Any CPU + {19AE3176-D15F-4A2B-AB49-1B9DE6EA3EE4}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {19AE3176-D15F-4A2B-AB49-1B9DE6EA3EE4}.Debug|Any CPU.Build.0 = Debug|Any CPU + {19AE3176-D15F-4A2B-AB49-1B9DE6EA3EE4}.Release|Any CPU.ActiveCfg = Release|Any CPU + {19AE3176-D15F-4A2B-AB49-1B9DE6EA3EE4}.Release|Any CPU.Build.0 = Release|Any CPU EndGlobalSection GlobalSection(SolutionProperties) = preSolution HideSolutionNode = FALSE diff --git a/src/aggregator-cli/AssemblyInfo.cs b/src/aggregator-cli/AssemblyInfo.cs index f7e5afbd..fcfa99c9 100644 --- a/src/aggregator-cli/AssemblyInfo.cs +++ b/src/aggregator-cli/AssemblyInfo.cs @@ -8,8 +8,8 @@ [assembly: AssemblyConfiguration("Release")] #endif [assembly: AssemblyCopyright("TFS Aggregator Team")] -[assembly: AssemblyFileVersion("0.3.4.0")] -[assembly: AssemblyInformationalVersion("0.3.4")] +[assembly: AssemblyFileVersion("0.4.0.0")] +[assembly: AssemblyInformationalVersion("0.4.0")] [assembly: AssemblyProduct("Aggregator CLI")] [assembly: AssemblyTitle("Aggregator CLI")] -[assembly: AssemblyVersion("0.3.4.0")] +[assembly: AssemblyVersion("0.4.0.0")] diff --git a/src/aggregator-cli/ContextBuilder.cs b/src/aggregator-cli/ContextBuilder.cs index 89544384..c834536c 100644 --- a/src/aggregator-cli/ContextBuilder.cs +++ b/src/aggregator-cli/ContextBuilder.cs @@ -12,12 +12,12 @@ internal class CommandContext { internal ILogger Logger { get; private set; } internal IAzure Azure { get; private set; } - internal VssConnection Vsts { get; private set; } - internal CommandContext(ILogger logger, IAzure azure, VssConnection vsts) + internal VssConnection Devops { get; private set; } + internal CommandContext(ILogger logger, IAzure azure, VssConnection devops) { Logger = logger; Azure = azure; - Vsts = vsts; + Devops = devops; } } @@ -25,7 +25,7 @@ internal class ContextBuilder { ILogger logger; bool azureLogon = false; - bool vstsLogon = false; + bool devopsLogon = false; internal ContextBuilder(ILogger logger) => this.logger = logger; @@ -34,15 +34,15 @@ internal ContextBuilder WithAzureLogon() azureLogon = true; return this; } - internal ContextBuilder WithVstsLogon() + internal ContextBuilder WithDevOpsLogon() { - vstsLogon = true; + devopsLogon = true; return this; } internal async Task Build() { IAzure azure = null; - VssConnection vsts = null; + VssConnection devops = null; if (azureLogon) { @@ -57,20 +57,20 @@ internal async Task Build() logger.WriteInfo($"Connected to subscription {azure.SubscriptionId}"); } - if (vstsLogon) + if (devopsLogon) { - logger.WriteVerbose($"Authenticating to VSTS..."); - var (connection, reason) = VstsLogon.Load(); + logger.WriteVerbose($"Authenticating to Azure DevOps..."); + var (connection, reason) = DevOpsLogon.Load(); if (reason != LogonResult.Succeeded) { string msg = TranslateResult(reason); - throw new ApplicationException(string.Format(msg, "VSTS", "logon.vsts")); + throw new ApplicationException(string.Format(msg, "Azure DevOps", "logon.ado")); } - vsts = await connection.LogonAsync(); - logger.WriteInfo($"Connected to {vsts.Uri.Host}"); + devops = await connection.LogonAsync(); + logger.WriteInfo($"Connected to {devops.Uri.Host}"); } - return new CommandContext(logger, azure, vsts); + return new CommandContext(logger, azure, devops); } private string TranslateResult(LogonResult reason) diff --git a/src/aggregator-cli/Instances/AggregatorInstances.cs b/src/aggregator-cli/Instances/AggregatorInstances.cs index 77f33886..74e084f0 100644 --- a/src/aggregator-cli/Instances/AggregatorInstances.cs +++ b/src/aggregator-cli/Instances/AggregatorInstances.cs @@ -58,11 +58,32 @@ public async Task> ListByLocationAsync(string location) return result; } + internal async Task> ListInResourceGroupAsync(string resourceGroup) + { + var apps = await azure.AppServices.FunctionApps.ListByResourceGroupAsync(resourceGroup); + + var result = new List(); + foreach (var app in apps) + { + result.Add( + InstanceName.FromFunctionAppName(app.Name).PlainName); + } + return result; + } + + internal async Task Add(InstanceName instance, string location, string requiredVersion) { string rgName = instance.ResourceGroupName; + logger.WriteVerbose($"Checking if Resource Group {rgName} already exists"); if (!await azure.ResourceGroups.ContainAsync(rgName)) { + if (instance.IsCustom) + { + logger.WriteError($"Resource group {rgName} is custom and cannot be created."); + return false; + } + logger.WriteVerbose($"Creating resource group {rgName}"); await azure.ResourceGroups .Define(rgName) @@ -71,7 +92,7 @@ await azure.ResourceGroups logger.WriteInfo($"Resource group {rgName} created."); } - // TODO the template should create a Storage account and/or a Key Vault + // IDEA the template should create a Storage account and/or a Key Vault var resourceName = "aggregator.cli.Instances.instance-template.json"; string armTemplateString; var assembly = Assembly.GetExecutingAssembly(); @@ -122,23 +143,30 @@ await azure.ResourceGroups bool ok = await package.UpdateVersion(requiredVersion, instance, azure); if (ok) { - var vstsLogonData = VstsLogon.Load().connection; - if (vstsLogonData.Mode == VstsTokenType.PAT) + var devopsLogonData = DevOpsLogon.Load().connection; + if (devopsLogonData.Mode == DevOpsTokenType.PAT) { - logger.WriteVerbose($"Saving VSTS token"); - ok = await ChangeAppSettings(instance, vstsLogonData); - logger.WriteInfo($"VSTS token saved"); + logger.WriteVerbose($"Saving Azure DevOps token"); + ok = await ChangeAppSettings(instance, devopsLogonData, SaveMode.Default); + if (ok) + { + logger.WriteInfo($"Azure DevOps token saved"); + } + else + { + logger.WriteError($"Failed to save Azure DevOps token"); + } } else { - logger.WriteWarning($"VSTS token type {vstsLogonData.Mode} is unsupported"); + logger.WriteWarning($"Azure DevOps token type {devopsLogonData.Mode} is unsupported"); ok = false; } } return ok; } - internal async Task ChangeAppSettings(InstanceName instance, VstsLogon vstsLogonData) + internal async Task ChangeAppSettings(InstanceName instance, DevOpsLogon devopsLogonData, SaveMode saveMode) { var webFunctionApp = await azure .AppServices @@ -148,8 +176,9 @@ internal async Task ChangeAppSettings(InstanceName instance, VstsLogon vst instance.FunctionAppName); var configuration = new AggregatorConfiguration { - VstsTokenType = vstsLogonData.Mode, - VstsToken = vstsLogonData.Token + DevOpsTokenType = devopsLogonData.Mode, + DevOpsToken = devopsLogonData.Token, + SaveMode = saveMode }; configuration.Write(webFunctionApp); return true; @@ -159,32 +188,54 @@ internal async Task Remove(InstanceName instance, string location) { string rgName = instance.ResourceGroupName; logger.WriteVerbose($"Searching instance {instance.PlainName}..."); - if (await azure.ResourceGroups.ContainAsync(rgName)) + bool rgFound = await azure.ResourceGroups.ContainAsync(rgName); + if (!rgFound) { - logger.WriteVerbose($"Deleting resource group {rgName}"); - await azure.ResourceGroups.DeleteByNameAsync(rgName); - logger.WriteInfo($"Resource group {rgName} deleted."); + logger.WriteWarning($"Resource Group {rgName} not found in {location}."); + return false; } - else + var functionApp = await azure.AppServices.FunctionApps.GetByResourceGroupAsync(rgName, instance.FunctionAppName); + if (functionApp == null) + { + logger.WriteWarning($"Instance {functionApp.Name} not found in resource group {rgName}."); + return false; + } + + logger.WriteVerbose($"Deleting instance {functionApp.Name} in resource group {rgName}."); + await azure.AppServices.FunctionApps.DeleteByIdAsync(functionApp.Id); + logger.WriteInfo($"Instance {functionApp.Name} deleted."); + + // we delete the RG only if was made by us + logger.WriteVerbose($"Checking if last instance in resource group {rgName}"); + var apps = await azure.AppServices.FunctionApps.ListByResourceGroupAsync(rgName); + if (apps == null || apps.Count() == 0) { - logger.WriteWarning($"Instance {instance.PlainName} not found in {location}."); + if (instance.IsCustom) + { + logger.WriteWarning($"Resource group {rgName} is custom and won't be deleted."); + return true; + } + + logger.WriteVerbose($"Deleting empty resource group {rgName}"); + await azure.ResourceGroups.DeleteByNameAsync(rgName); + logger.WriteInfo($"Resource group {rgName} deleted."); } return true; } - internal async Task SetAuthentication(InstanceName instance, string location) + internal async Task ChangeAppSettings(InstanceName instance, string location, SaveMode saveMode) { bool ok; - var vstsLogonData = VstsLogon.Load().connection; - if (vstsLogonData.Mode == VstsTokenType.PAT) + var devopsLogonData = DevOpsLogon.Load().connection; + if (devopsLogonData.Mode == DevOpsTokenType.PAT) { - logger.WriteVerbose($"Saving VSTS token"); - ok = await ChangeAppSettings(instance, vstsLogonData); - logger.WriteInfo($"VSTS token saved"); + logger.WriteVerbose($"Saving Azure DevOps token"); + ok = await ChangeAppSettings(instance, devopsLogonData, saveMode); + logger.WriteInfo($"Azure DevOps token saved"); } else { - logger.WriteWarning($"VSTS token type {vstsLogonData.Mode} is unsupported"); + logger.WriteWarning($"Azure DevOps token type {devopsLogonData.Mode} is unsupported"); ok = false; } return ok; diff --git a/src/aggregator-cli/Instances/ConfigureInstanceCommand.cs b/src/aggregator-cli/Instances/ConfigureInstanceCommand.cs index c6ee339a..71229107 100644 --- a/src/aggregator-cli/Instances/ConfigureInstanceCommand.cs +++ b/src/aggregator-cli/Instances/ConfigureInstanceCommand.cs @@ -15,8 +15,15 @@ internal class ConfigureInstanceCommand : CommandBase [Option('l', "location", Required = true, HelpText = "Aggregator instance location (Azure region).")] public string Location { get; set; } + [Option('g', "resourceGroup", Required = false, Default = "", HelpText = "Azure Resource Group hosting the Aggregator instance.")] + public string ResourceGroup { get; set; } + [Option('a', "authentication", SetName = "auth", Required = true, HelpText = "Refresh authentication data.")] public bool Authentication { get; set; } + + [Option('m', "saveMode", SetName = "save", Required = false, HelpText = "Save behaviour.")] + public SaveMode SaveMode { get; set; } + // TODO add --swap.slot to support App Service Deployment Slots @@ -24,14 +31,14 @@ internal override async Task RunAsync() { var context = await Context .WithAzureLogon() - .WithVstsLogon() // need the token, so we can save it in the app settings + .WithDevOpsLogon() // need the token, so we can save it in the app settings .Build(); var instances = new AggregatorInstances(context.Azure, context.Logger); - var instance = new InstanceName(Name); + var instance = new InstanceName(Name, ResourceGroup); bool ok = false; if (Authentication) { - ok = await instances.SetAuthentication(instance, Location); + ok = await instances.ChangeAppSettings(instance, Location, SaveMode); } else { context.Logger.WriteError($"Unsupported command option(s)"); diff --git a/src/aggregator-cli/Instances/FunctionRuntimePackage.cs b/src/aggregator-cli/Instances/FunctionRuntimePackage.cs index 06d29684..4eb74916 100644 --- a/src/aggregator-cli/Instances/FunctionRuntimePackage.cs +++ b/src/aggregator-cli/Instances/FunctionRuntimePackage.cs @@ -31,37 +31,84 @@ internal FunctionRuntimePackage(ILogger logger) internal async Task UpdateVersion(string requiredVersion, InstanceName instance, IAzure azure) { - if (string.IsNullOrWhiteSpace(requiredVersion)) + string tag = string.IsNullOrWhiteSpace(requiredVersion) + ? "latest" + : (requiredVersion != "latest" + ? (requiredVersion[0] != 'v' + ? "v" + requiredVersion + : requiredVersion) + : requiredVersion); + + logger.WriteVerbose($"Checking runtime package versions in GitHub"); + (string rel_name, DateTimeOffset? rel_when, string rel_url) = await FindVersionInGitHub(tag); + if (string.IsNullOrEmpty(rel_name)) { - requiredVersion = "latest"; + logger.WriteError($"Requested runtime {requiredVersion} version does not exists."); + return false; } - logger.WriteVerbose($"Checking runtime package versions in GitHub"); - (string rel_name, DateTimeOffset? rel_when, string rel_url) = await FindVersionInGitHub(requiredVersion); if (rel_name[0] == 'v') rel_name = rel_name.Substring(1); var requiredRuntimeVer = SemVersion.Parse(rel_name); - logger.WriteInfo($"Latest Runtime package version is {requiredRuntimeVer} (released on {rel_when})."); + logger.WriteVerbose($"Latest Runtime package version is {requiredRuntimeVer} (released on {rel_when})."); string localPackageVersion = GetLocalPackageVersion(RuntimePackageFile); var localRuntimeVer = SemVersion.Parse(localPackageVersion); - logger.WriteInfo($"Cached Runtime package version is {localRuntimeVer}."); - if (ShouldUpdate(requiredRuntimeVer, localRuntimeVer)) + logger.WriteVerbose($"Cached Runtime package version is {localRuntimeVer}."); + + // TODO check the uploaded version before overwriting? + string manifestVersion = "0.0.0"; + logger.WriteVerbose($"Retrieving functions runtime"); + var kudu = new KuduApi(instance, azure, logger); + using (var client = new HttpClient()) + using (var request = await kudu.GetRequestAsync(HttpMethod.Get, $"api/vfs/site/wwwroot/aggregator-manifest.ini")) + using (var response = await client.SendAsync(request)) { - logger.WriteVerbose($"Downloading runtime package {rel_name}"); - await Download(rel_url); - logger.WriteInfo($"Runtime package downloaded."); + string manifest = await response.Content.ReadAsStringAsync(); + + if (response.IsSuccessStatusCode) + { + // HACK refactor so that there is a single class parsing the manifest + var parts = manifest.Split('='); + if (parts[0] == "version") + manifestVersion = parts[1]; + } + else + { + logger.WriteWarning($"Cannot read aggregator-manifest.ini: {response.ReasonPhrase}"); + } } + var uploadedRuntimeVer = SemVersion.Parse(manifestVersion); + logger.WriteVerbose($"Function Runtime version is {uploadedRuntimeVer}."); - logger.WriteVerbose($"Uploading runtime package to {instance.DnsHostName}"); - bool ok = await UploadRuntimeZip(instance, azure); - if (ok) + if (requiredRuntimeVer > uploadedRuntimeVer || localRuntimeVer > uploadedRuntimeVer) { - logger.WriteInfo($"Runtime package uploaded to {instance.PlainName}."); + if (requiredRuntimeVer > localRuntimeVer) + { + logger.WriteVerbose($"Downloading runtime package {rel_name}"); + await Download(rel_url); + logger.WriteInfo($"Runtime package downloaded."); + } + else + { + logger.WriteInfo($"Using cached runtime package {localRuntimeVer}"); + } + + logger.WriteVerbose($"Uploading runtime package to {instance.DnsHostName}"); + bool ok = await UploadRuntimeZip(instance, azure); + if (ok) + { + logger.WriteInfo($"Runtime package uploaded to {instance.PlainName}."); + } + else + { + logger.WriteError($"Failed uploading Runtime to {instance.DnsHostName}."); + } + return ok; } else { - logger.WriteError($"Failed uploading Runtime to {instance.DnsHostName}."); + logger.WriteInfo($"Runtime package is up to date."); + return true; } - return ok; } private string GetLocalPackageVersion(string runtimePackageFile) @@ -86,11 +133,6 @@ private string GetLocalPackageVersion(string runtimePackageFile) return manifestVersion; } - private bool ShouldUpdate(SemVersion lastRuntimeVer, SemVersion currentCliVer) - { - return lastRuntimeVer > currentCliVer; - } - private async Task<(string name, DateTimeOffset? when, string url)> FindVersionInGitHub(string tag = "latest") { var githubClient = new GitHubClient(new ProductHeaderValue("aggregator-cli", infoVersion)); diff --git a/src/aggregator-cli/Instances/InstallInstanceCommand.cs b/src/aggregator-cli/Instances/InstallInstanceCommand.cs index 2bb684f6..4c502698 100644 --- a/src/aggregator-cli/Instances/InstallInstanceCommand.cs +++ b/src/aggregator-cli/Instances/InstallInstanceCommand.cs @@ -15,6 +15,9 @@ class InstallInstanceCommand : CommandBase [Option('l', "location", Required = true, HelpText = "Aggregator instance location (Azure region).")] public string Location { get; set; } + [Option('g', "resourceGroup", Required = false, Default = "", HelpText = "Azure Resource Group hosting the Aggregator instances.")] + public string ResourceGroup { get; set; } + [Option("requiredVersion", Required = false, HelpText = "Version of Aggregator Runtime required.")] public string RequiredVersion { get; set; } @@ -22,10 +25,10 @@ internal override async Task RunAsync() { var context = await Context .WithAzureLogon() - .WithVstsLogon() // need the token, so we can save it in the app settings + .WithDevOpsLogon() // need the token, so we can save it in the app settings .Build(); var instances = new AggregatorInstances(context.Azure, context.Logger); - var instance = new InstanceName(Name); + var instance = new InstanceName(Name, ResourceGroup); bool ok = await instances.Add(instance, Location, RequiredVersion); return ok ? 0 : 1; } diff --git a/src/aggregator-cli/Instances/InstanceName.cs b/src/aggregator-cli/Instances/InstanceName.cs index 8e4267c1..de8cf6ab 100644 --- a/src/aggregator-cli/Instances/InstanceName.cs +++ b/src/aggregator-cli/Instances/InstanceName.cs @@ -9,29 +9,44 @@ internal class InstanceName const string resourceGroupPrefix = "aggregator-"; const string functionAppSuffix = "aggregator"; private readonly string name; + private readonly string resourceGroup; + // used only in ListInstances public static string ResourceGroupInstancePrefix => resourceGroupPrefix; - internal InstanceName(string name) + public InstanceName(string name, string resourceGroup) { this.name = name; + this.resourceGroup = string.IsNullOrEmpty(resourceGroup) + ? resourceGroupPrefix + name + : resourceGroup; } + // used only in ListInstances public static InstanceName FromResourceGroupName(string rgName) { - return new InstanceName(rgName.Remove(0, ResourceGroupInstancePrefix.Length)); + return new InstanceName(rgName.Remove(0, ResourceGroupInstancePrefix.Length), null); } + // used only in ListInstances + public static InstanceName FromFunctionAppName(string appName) + { + return new InstanceName(appName.Remove(appName.Length - functionAppSuffix.Length), null); + } + + // used only in mappings.ListAsync public static InstanceName FromFunctionAppUrl(string url) { string host = new Uri(url).Host; host = host.Substring(0, host.IndexOf('.')); - return new InstanceName(host.Remove(host.Length - functionAppSuffix.Length)); + return new InstanceName(host.Remove(host.Length - functionAppSuffix.Length), null); } public string PlainName => name; - internal string ResourceGroupName => resourceGroupPrefix + name; + internal string ResourceGroupName => resourceGroup; + + internal bool IsCustom => resourceGroup != resourceGroupPrefix + name; internal string FunctionAppName=> name + functionAppSuffix; diff --git a/src/aggregator-cli/Instances/ListInstancesCommand.cs b/src/aggregator-cli/Instances/ListInstancesCommand.cs index 3bb6569b..2e4c4f0a 100644 --- a/src/aggregator-cli/Instances/ListInstancesCommand.cs +++ b/src/aggregator-cli/Instances/ListInstancesCommand.cs @@ -11,21 +11,30 @@ class ListInstancesCommand : CommandBase [Option('l', "location", Required = false, HelpText = "Aggregator instance location (Azure region).")] public string Location { get; set; } + [Option('g', "resourceGroup", Required = false, Default = "", HelpText = "Azure Resource Group hosting the Aggregator instances.")] + public string ResourceGroup { get; set; } + internal override async Task RunAsync() { var context = await Context .WithAzureLogon() .Build(); var instances = new AggregatorInstances(context.Azure, context.Logger); - if (string.IsNullOrEmpty(Location)) + if (!string.IsNullOrEmpty(Location)) { - context.Logger.WriteVerbose($"Searching aggregator instances in subscription..."); - return await ListAllAsync(context, instances); + context.Logger.WriteVerbose($"Searching aggregator instances in {Location}..."); + return await ListByLocationAsync(context, instances); + + } + else if (!string.IsNullOrEmpty(ResourceGroup)) + { + context.Logger.WriteVerbose($"Searching aggregator instances in {ResourceGroup}..."); + return await ListInResourceGroupAsync(context, instances); } else { - context.Logger.WriteVerbose($"Searching aggregator instances in {Location}..."); - return await ListByLocationAsync(context, instances); + context.Logger.WriteVerbose($"Searching aggregator instances in subscription..."); + return await ListAllAsync(context, instances); } } @@ -47,6 +56,24 @@ private async Task ListByLocationAsync(CommandContext context, AggregatorIn return 0; } + private async Task ListInResourceGroupAsync(CommandContext context, AggregatorInstances instances) + { + var found = await instances.ListInResourceGroupAsync(ResourceGroup); + bool any = false; + foreach (var name in found) + { + context.Logger.WriteOutput( + name, + (data) => $"Instance {name}"); + any = true; + } + if (!any) + { + context.Logger.WriteInfo("No aggregator instances found."); + } + return 0; + } + private static async Task ListAllAsync(CommandContext context, AggregatorInstances instances) { var found = await instances.ListAllAsync(); diff --git a/src/aggregator-cli/Instances/UninstallInstanceCommand.cs b/src/aggregator-cli/Instances/UninstallInstanceCommand.cs index e70dc157..6177bf08 100644 --- a/src/aggregator-cli/Instances/UninstallInstanceCommand.cs +++ b/src/aggregator-cli/Instances/UninstallInstanceCommand.cs @@ -15,22 +15,25 @@ class UninstallInstanceCommand : CommandBase [Option('l', "location", Required = true, HelpText = "Aggregator instance location (Azure region).")] public string Location { get; set; } - [Option('m', "dont-remove-mappings", Required = false, HelpText = "Do not remove mappings from VSTS (default is to remove them).")] + [Option('g', "resourceGroup", Required = false, Default = "", HelpText = "Azure Resource Group hosting the Aggregator instance.")] + public string ResourceGroup { get; set; } + + [Option('m', "dont-remove-mappings", Required = false, HelpText = "Do not remove mappings from Azure DevOps (default is to remove them).")] public bool Mappings { get; set; } internal override async Task RunAsync() { var context = await Context .WithAzureLogon() - .WithVstsLogon() + .WithDevOpsLogon() .Build(); - var instance = new InstanceName(Name); + var instance = new InstanceName(Name, ResourceGroup); bool ok; if (!Mappings) { - var mappings = new AggregatorMappings(context.Vsts, context.Azure, context.Logger); + var mappings = new AggregatorMappings(context.Devops, context.Azure, context.Logger); ok = await mappings.RemoveInstanceAsync(instance); } diff --git a/src/aggregator-cli/Logon/VstsLogon.cs b/src/aggregator-cli/Logon/DevOpsLogon.cs similarity index 78% rename from src/aggregator-cli/Logon/VstsLogon.cs rename to src/aggregator-cli/Logon/DevOpsLogon.cs index c2327acd..035d7706 100644 --- a/src/aggregator-cli/Logon/VstsLogon.cs +++ b/src/aggregator-cli/Logon/DevOpsLogon.cs @@ -8,12 +8,12 @@ namespace aggregator.cli { - class VstsLogon : LogonDataBase + class DevOpsLogon : LogonDataBase { private static readonly string LogonDataTag = "davs"; public string Url { get; set; } - public VstsTokenType Mode { get; set; } + public DevOpsTokenType Mode { get; set; } public string Token { get; set; } public string Save() @@ -21,9 +21,9 @@ public string Save() return new LogonDataStore(LogonDataTag).Save(this); } - public static (VstsLogon connection, LogonResult reason) Load() + public static (DevOpsLogon connection, LogonResult reason) Load() { - var result = new LogonDataStore(LogonDataTag).Load(); + var result = new LogonDataStore(LogonDataTag).Load(); return (result.connection, result.reason); } @@ -32,10 +32,10 @@ public async Task LogonAsync() var clientCredentials = default(VssCredentials); switch (Mode) { - case VstsTokenType.Integrated: + case DevOpsTokenType.Integrated: clientCredentials = new VssCredentials(); break; - case VstsTokenType.PAT: + case DevOpsTokenType.PAT: clientCredentials=new VssBasicCredential("pat", Token); break; default: diff --git a/src/aggregator-cli/Logon/LogonVstsCommand.cs b/src/aggregator-cli/Logon/LogonDevOpsCommand.cs similarity index 60% rename from src/aggregator-cli/Logon/LogonVstsCommand.cs rename to src/aggregator-cli/Logon/LogonDevOpsCommand.cs index 8929dbb0..4705ea9c 100644 --- a/src/aggregator-cli/Logon/LogonVstsCommand.cs +++ b/src/aggregator-cli/Logon/LogonDevOpsCommand.cs @@ -7,16 +7,16 @@ namespace aggregator.cli { - [Verb("logon.vsts", HelpText = "Logon into Visual Studio Team Services.")] - class LogonVstsCommand : CommandBase + [Verb("logon.ado", HelpText = "Logon into Azure DevOps.")] + class LogonDevOpsCommand : CommandBase { [Option('u', "url", Required = true, HelpText = "Account/server URL, e.g. myaccount.visualstudio.com .")] public string Url { get; set; } [Option('m', "mode", Required = true, HelpText = "Logon mode (valid modes: PAT).")] - public VstsTokenType Mode { get; set; } + public DevOpsTokenType Mode { get; set; } - [Option('t', "token", SetName = "PAT", HelpText = "VSTS Personal Authentication Token.")] + [Option('t', "token", SetName = "PAT", HelpText = "Azure DevOps Personal Authentication Token.")] public string Token { get; set; } internal override async Task RunAsync() @@ -24,7 +24,7 @@ internal override async Task RunAsync() var context = await Context.Build(); - var data = new VstsLogon() + var data = new DevOpsLogon() { Url = this.Url, Mode = this.Mode, @@ -32,11 +32,11 @@ internal override async Task RunAsync() }; string path = data.Save(); // now check for validity - context.Logger.WriteInfo($"Connecting to VSTS using {Mode} credential..."); - var vsts = await data.LogonAsync(); - if (vsts == null) + context.Logger.WriteInfo($"Connecting to Azure DevOps using {Mode} credential..."); + var devops = await data.LogonAsync(); + if (devops == null) { - context.Logger.WriteError("Invalid VSTS credentials"); + context.Logger.WriteError("Invalid Azure DevOps credentials"); return 2; } return 0; diff --git a/src/aggregator-cli/Mappings/AggregatorMappings.cs b/src/aggregator-cli/Mappings/AggregatorMappings.cs index ab683c01..9cab0e43 100644 --- a/src/aggregator-cli/Mappings/AggregatorMappings.cs +++ b/src/aggregator-cli/Mappings/AggregatorMappings.cs @@ -13,28 +13,32 @@ namespace aggregator.cli { internal class AggregatorMappings { - private VssConnection vsts; + private VssConnection devops; private readonly IAzure azure; private readonly ILogger logger; - public AggregatorMappings(VssConnection vsts, IAzure azure, ILogger logger) + public AggregatorMappings(VssConnection devops, IAzure azure, ILogger logger) { - this.vsts = vsts; + this.devops = devops; this.azure = azure; this.logger = logger; } - internal async Task> ListAsync(InstanceName instance) + internal async Task> ListAsync(InstanceName instance, string projectName) { - logger.WriteVerbose($"Searching aggregator mappings in VSTS..."); - var serviceHooksClient = vsts.GetClient(); + logger.WriteVerbose($"Searching aggregator mappings in Azure DevOps..."); + var serviceHooksClient = devops.GetClient(); var subscriptions = await serviceHooksClient.QuerySubscriptionsAsync(); - var filteredSubs = subscriptions.Where(s - => s.PublisherId == VstsEvents.PublisherId - && s.ConsumerInputs["url"].ToString().StartsWith( - instance.FunctionAppUrl) - ); - var projectClient = vsts.GetClient(); + var filteredSubs = instance != null + ? subscriptions.Where(s + => s.PublisherId == DevOpsEvents.PublisherId + && s.ConsumerInputs["url"].ToString().StartsWith( + instance.FunctionAppUrl)) + : subscriptions.Where(s + => s.PublisherId == DevOpsEvents.PublisherId + // HACK + && s.ConsumerInputs["url"].ToString().IndexOf("aggregator.azurewebsites.net") > 8); + var projectClient = devops.GetClient(); var projects = await projectClient.GetProjects(); var projectsDict = projects.ToDictionary(p => p.Id); @@ -44,10 +48,14 @@ public AggregatorMappings(VssConnection vsts, IAzure azure, ILogger logger) var foundProject = projectsDict[ new Guid(subscription.PublisherInputs["projectId"]) ]; + if (!string.IsNullOrEmpty(projectName) && foundProject.Name != projectName) + { + continue; + } // HACK need to factor the URL<->rule_name string ruleUrl = subscription.ConsumerInputs["url"].ToString(); string ruleName = ruleUrl.Substring(ruleUrl.LastIndexOf('/')); - string ruleFullName = instance.PlainName + ruleName; + string ruleFullName = InstanceName.FromFunctionAppUrl(ruleUrl).PlainName + ruleName; result.Add( (ruleFullName, foundProject.Name, subscription.EventType, subscription.Status.ToString()) ); @@ -57,8 +65,8 @@ public AggregatorMappings(VssConnection vsts, IAzure azure, ILogger logger) internal async Task Add(string projectName, string @event, InstanceName instance, string ruleName) { - logger.WriteVerbose($"Reading VSTS project data..."); - var projectClient = vsts.GetClient(); + logger.WriteVerbose($"Reading Azure DevOps project data..."); + var projectClient = devops.GetClient(); var project = await projectClient.GetProject(projectName); logger.WriteInfo($"Project {projectName} data read."); @@ -67,11 +75,24 @@ internal async Task Add(string projectName, string @event, InstanceName in (string ruleUrl, string ruleKey) = await rules.GetInvocationUrlAndKey(instance, ruleName); logger.WriteInfo($"{ruleName} Function Key retrieved."); - var serviceHooksClient = vsts.GetClient(); + var serviceHooksClient = devops.GetClient(); // check if the subscription already exists and bail out var query = new SubscriptionsQuery { - PublisherId = VstsEvents.PublisherId, + PublisherId = DevOpsEvents.PublisherId, + PublisherInputFilters= new InputFilter[] { + new InputFilter { + Conditions = new List { + new InputFilterCondition + { + InputId = "projectId", + InputValue = project.Id.ToString(), + Operator = InputFilterOperator.Equals, + CaseSensitive = false + } + } + } + }, ConsumerInputFilters = new InputFilter[] { new InputFilter { Conditions = new List { @@ -93,7 +114,7 @@ internal async Task Add(string projectName, string @event, InstanceName in return Guid.Empty; } - // see https://docs.microsoft.com/en-us/vsts/service-hooks/consumers?toc=%2Fvsts%2Fintegrate%2Ftoc.json&bc=%2Fvsts%2Fintegrate%2Fbreadcrumb%2Ftoc.json&view=vsts#web-hooks + // see https://docs.microsoft.com/en-us/azure/devops/service-hooks/consumers?toc=%2Fvsts%2Fintegrate%2Ftoc.json&bc=%2Fvsts%2Fintegrate%2Fbreadcrumb%2Ftoc.json&view=vsts#web-hooks var subscriptionParameters = new Subscription() { ConsumerId = "webHooks", @@ -108,12 +129,12 @@ internal async Task Add(string projectName, string @event, InstanceName in { "detailedMessagesToSend", "none" }, }, EventType = @event, - PublisherId = VstsEvents.PublisherId, + PublisherId = DevOpsEvents.PublisherId, PublisherInputs = new Dictionary { { "projectId", project.Id.ToString() }, /* TODO consider offering additional filters using the following - { "tfsSubscriptionId", vsts.ServerId }, + { "tfsSubscriptionId", devops.ServerId }, { "teamId", null }, // Filter events to include only work items under the specified area path. { "areaPath", null }, @@ -135,19 +156,19 @@ internal async Task Add(string projectName, string @event, InstanceName in internal async Task RemoveInstanceAsync(InstanceName instance) { - return await RemoveRuleEventAsync("*", instance, "*"); + return await RemoveRuleEventAsync("*", instance, "*", "*"); } internal async Task RemoveRuleAsync(InstanceName instance, string rule) { - return await RemoveRuleEventAsync("*", instance, rule); + return await RemoveRuleEventAsync("*", instance, "*", rule); } - internal async Task RemoveRuleEventAsync(string @event, InstanceName instance, string rule) + internal async Task RemoveRuleEventAsync(string @event, InstanceName instance, string projectName, string rule) { - logger.WriteInfo($"Querying the VSTS subscriptions for rule(s) {instance.PlainName}/{rule}"); - var serviceHooksClient = vsts.GetClient(); - var subscriptions = await serviceHooksClient.QuerySubscriptionsAsync(VstsEvents.PublisherId); + logger.WriteInfo($"Querying the Azure DevOps subscriptions for rule(s) {instance.PlainName}/{rule}"); + var serviceHooksClient = devops.GetClient(); + var subscriptions = await serviceHooksClient.QuerySubscriptionsAsync(DevOpsEvents.PublisherId); var ruleSubs = subscriptions // TODO can we trust this equality? // && s.ActionDescription == $"To host {instance.DnsHostName}" @@ -157,6 +178,15 @@ internal async Task RemoveRuleEventAsync(string @event, InstanceName insta { ruleSubs = ruleSubs.Where(s => s.EventType == @event); } + if (projectName != "*") + { + logger.WriteVerbose($"Reading Azure DevOps project data..."); + var projectClient = devops.GetClient(); + var project = await projectClient.GetProject(projectName); + logger.WriteInfo($"Project {projectName} data read."); + + ruleSubs = ruleSubs.Where(s => s.PublisherInputs["projectId"] == project.Id.ToString()); + } if (rule != "*") { ruleSubs = ruleSubs @@ -165,9 +195,9 @@ internal async Task RemoveRuleEventAsync(string @event, InstanceName insta } foreach (var ruleSub in ruleSubs) { - logger.WriteVerbose($"Deleting subscription {ruleSub.EventDescription}..."); + logger.WriteVerbose($"Deleting subscription {ruleSub.EventDescription} {ruleSub.EventType}..."); await serviceHooksClient.DeleteSubscriptionAsync(ruleSub.Id); - logger.WriteInfo($"Subscription {ruleSub.EventDescription} deleted."); + logger.WriteInfo($"Subscription {ruleSub.EventDescription} {ruleSub.EventType} deleted."); } return true; diff --git a/src/aggregator-cli/Mappings/ListMappingsCommand.cs b/src/aggregator-cli/Mappings/ListMappingsCommand.cs index 0ef81f43..124414ee 100644 --- a/src/aggregator-cli/Mappings/ListMappingsCommand.cs +++ b/src/aggregator-cli/Mappings/ListMappingsCommand.cs @@ -7,21 +7,28 @@ namespace aggregator.cli { - [Verb("list.mappings", HelpText = "Lists mappings from existing VSTS Projects to Aggregator Rules.")] + [Verb("list.mappings", HelpText = "Lists mappings from existing Azure DevOps Projects to Aggregator Rules.")] class ListMappingsCommand : CommandBase { - [Option('i', "instance", Required = true, HelpText = "Aggregator instance name.")] + [Option('i', "instance", Required = false, HelpText = "Aggregator instance name.")] public string Instance { get; set; } + [Option('g', "resourceGroup", Required = false, Default = "", HelpText = "Azure Resource Group hosting the Aggregator instance.")] + public string ResourceGroup { get; set; } + + [Option('p', "project", Required = false, Default = "", HelpText = "Azure DevOps project name.")] + public string Project { get; set; } + internal override async Task RunAsync() { var context = await Context - .WithVstsLogon() + .WithDevOpsLogon() .Build(); - var instance = new InstanceName(Instance); - var mappings = new AggregatorMappings(context.Vsts, /*HACK*/null, context.Logger); + var instance = string.IsNullOrEmpty(Instance) ? null : new InstanceName(Instance, ResourceGroup); + // HACK we pass null as the next calls do not use the Azure connection + var mappings = new AggregatorMappings(context.Devops, null, context.Logger); bool any = false; - foreach (var item in await mappings.ListAsync(instance)) + foreach (var item in await mappings.ListAsync(instance, Project)) { context.Logger.WriteOutput( item, diff --git a/src/aggregator-cli/Mappings/MapRuleCommand.cs b/src/aggregator-cli/Mappings/MapRuleCommand.cs index c83fec1f..148b5321 100644 --- a/src/aggregator-cli/Mappings/MapRuleCommand.cs +++ b/src/aggregator-cli/Mappings/MapRuleCommand.cs @@ -6,18 +6,21 @@ namespace aggregator.cli { - [Verb("map.rule", HelpText = "Maps an Aggregator Rule to existing VSTS Projects.")] + [Verb("map.rule", HelpText = "Maps an Aggregator Rule to existing Azure DevOps Projects.")] class MapRuleCommand : CommandBase { - [Option('p', "project", Required = true, HelpText = "VSTS project name.")] + [Option('p', "project", Required = true, HelpText = "Azure DevOps project name.")] public string Project { get; set; } - [Option('e', "event", Required = true, HelpText = "VSTS event.")] + [Option('e', "event", Required = true, HelpText = "Azure DevOps event.")] public string Event { get; set; } [Option('i', "instance", Required = true, HelpText = "Aggregator instance name.")] public string Instance { get; set; } + [Option('g', "resourceGroup", Required = false, Default = "", HelpText = "Azure Resource Group hosting the Aggregator instance.")] + public string ResourceGroup { get; set; } + [Option('r', "rule", Required = true, HelpText = "Aggregator rule name.")] public string Rule { get; set; } @@ -25,16 +28,16 @@ internal override async Task RunAsync() { var context = await Context .WithAzureLogon() - .WithVstsLogon() + .WithDevOpsLogon() .Build(); - var mappings = new AggregatorMappings(context.Vsts, context.Azure, context.Logger); - bool ok = VstsEvents.IsValidEvent(Event); + var mappings = new AggregatorMappings(context.Devops, context.Azure, context.Logger); + bool ok = DevOpsEvents.IsValidEvent(Event); if (!ok) { context.Logger.WriteError($"Invalid event type."); return 2; } - var instance = new InstanceName(Instance); + var instance = new InstanceName(Instance, ResourceGroup); var id = await mappings.Add(Project, Event, instance, Rule); return id.Equals(Guid.Empty) ? 1 : 0; } diff --git a/src/aggregator-cli/Mappings/UnmapRuleCommand.cs b/src/aggregator-cli/Mappings/UnmapRuleCommand.cs index 5c5a0f8a..019fe187 100644 --- a/src/aggregator-cli/Mappings/UnmapRuleCommand.cs +++ b/src/aggregator-cli/Mappings/UnmapRuleCommand.cs @@ -6,15 +6,21 @@ namespace aggregator.cli { - [Verb("unmap.rule", HelpText = "Unmaps an Aggregator Rule from a VSTS Project.")] + [Verb("unmap.rule", HelpText = "Unmaps an Aggregator Rule from a Azure DevOps Project.")] class UnmapRuleCommand : CommandBase { - [Option('e', "event", Required = true, HelpText = "VSTS event.")] + [Option('e', "event", Required = true, HelpText = "Azure DevOps event.")] public string Event { get; set; } [Option('i', "instance", Required = true, HelpText = "Aggregator instance name.")] public string Instance { get; set; } + [Option('g', "resourceGroup", Required = false, Default = "", HelpText = "Azure Resource Group hosting the Aggregator instance.")] + public string ResourceGroup { get; set; } + + [Option('p', "project", Required = false, Default = "*", HelpText = "Azure DevOps project name.")] + public string Project { get; set; } + [Option('r', "rule", Required = true, HelpText = "Aggregator rule name.")] public string Rule { get; set; } @@ -22,11 +28,11 @@ internal override async Task RunAsync() { var context = await Context .WithAzureLogon() - .WithVstsLogon() + .WithDevOpsLogon() .Build(); - var instance = new InstanceName(Instance); - var mappings = new AggregatorMappings(context.Vsts, context.Azure, context.Logger); - bool ok = await mappings.RemoveRuleEventAsync(Event, instance, Rule); + var instance = new InstanceName(Instance, ResourceGroup); + var mappings = new AggregatorMappings(context.Devops, context.Azure, context.Logger); + bool ok = await mappings.RemoveRuleEventAsync(Event, instance, Project, Rule); return ok ? 0 : 1; } } diff --git a/src/aggregator-cli/Program.cs b/src/aggregator-cli/Program.cs index b251414a..31f21f96 100644 --- a/src/aggregator-cli/Program.cs +++ b/src/aggregator-cli/Program.cs @@ -7,23 +7,33 @@ namespace aggregator.cli /* Ideas for verbs and options: - logon.vsts --url URL --mode MODE --token TOKEN --slot SLOT + logon.azure --subscription SUBSCRIPTION_ID --client CLIENT_ID --password CLIENT_PASSWORD --tenant TENANT_ID --instance INSTANCE --resourceGroup RESOURCEGROUP + default values for instance and resource group + logon.ado --url URL --mode MODE --token TOKEN --project PROJECT + default value for project + + logon.ado --url URL --mode MODE --token TOKEN --slot SLOT to use different credentials + configure.instance --slot SLOT --swap --avzone ZONE add a deployment slot with the option to specify an availability zone, the swap option will set the new slot as primary configure.instance --listOutboundIPs use `azure.AppServices.WebApps.GetByResourceGroup(instance.ResourceGroupName,instance.FunctionAppName).OutboundIPAddresses` + configure.instance --MSI + support for Managed service identity when AzDO is backed by AAD + configure.rule --verbose --instance INSTANCE --name RULE --file FILE --slot SLOT change rule code on a single deployment slot - invoke.rule --verbose --local --ruleSource FILE --event EVENT --workItemId WORK_ITEM_ID + + invoke.rule --verbose --dryrun --local --ruleSource FILE --event EVENT --workItemId WORK_ITEM_ID runs rule code locally - invoke.rule --verbose --instance INSTANCE --rule RULE --event EVENT --workItemId WORK_ITEM_ID --slot SLOT + invoke.rule --verbose --dryrun --instance INSTANCE --rule RULE --event EVENT --workItemId WORK_ITEM_ID --slot SLOT emulates the event on the rule */ - class Program + public class Program { - static int Main(string[] args) + public static int Main(string[] args) { var parser = new Parser(settings => { @@ -31,16 +41,19 @@ static int Main(string[] args) // fails see https://github.com/commandlineparser/commandline/issues/198 settings.CaseInsensitiveEnumValues = true; }); - var parserResult = parser.ParseArguments(args, - typeof(LogonAzureCommand), typeof(LogonVstsCommand), + var types = new Type[] { + typeof(LogonAzureCommand), typeof(LogonDevOpsCommand), typeof(ListInstancesCommand), typeof(InstallInstanceCommand), typeof(UninstallInstanceCommand), - typeof(ListRulesCommand), typeof(AddRuleCommand), typeof(RemoveRuleCommand), typeof(ConfigureRuleCommand), + typeof(ConfigureInstanceCommand), + typeof(ListRulesCommand), typeof(AddRuleCommand), typeof(RemoveRuleCommand), + typeof(ConfigureRuleCommand), typeof(UpdateRuleCommand), typeof(InvokeRuleCommand), typeof(ListMappingsCommand), typeof(MapRuleCommand), typeof(UnmapRuleCommand) - ); + }; + var parserResult = parser.ParseArguments(args, types); int rc = -1; parserResult .WithParsed(cmd => rc = cmd.Run()) - .WithParsed(cmd => rc = cmd.Run()) + .WithParsed(cmd => rc = cmd.Run()) .WithParsed(cmd => rc = cmd.Run()) .WithParsed(cmd => rc = cmd.Run()) .WithParsed(cmd => rc = cmd.Run()) @@ -49,6 +62,8 @@ static int Main(string[] args) .WithParsed(cmd => rc = cmd.Run()) .WithParsed(cmd => rc = cmd.Run()) .WithParsed(cmd => rc = cmd.Run()) + .WithParsed(cmd => rc = cmd.Run()) + .WithParsed(cmd => rc = cmd.Run()) .WithParsed(cmd => rc = cmd.Run()) .WithParsed(cmd => rc = cmd.Run()) .WithParsed(cmd => rc = cmd.Run()) diff --git a/src/aggregator-cli/Rules/AddRuleCommand.cs b/src/aggregator-cli/Rules/AddRuleCommand.cs index fb4a317f..ab948258 100644 --- a/src/aggregator-cli/Rules/AddRuleCommand.cs +++ b/src/aggregator-cli/Rules/AddRuleCommand.cs @@ -9,6 +9,9 @@ namespace aggregator.cli [Verb("add.rule", HelpText = "Add a rule to existing Aggregator instance in Azure.")] class AddRuleCommand : CommandBase { + [Option('g', "resourceGroup", Required = false, Default = "", HelpText = "Azure Resource Group hosting the Aggregator instances.")] + public string ResourceGroup { get; set; } + [Option('i', "instance", Required = true, HelpText = "Aggregator instance name.")] public string Instance { get; set; } @@ -23,7 +26,7 @@ internal override async Task RunAsync() var context = await Context .WithAzureLogon() .Build(); - var instance = new InstanceName(Instance); + var instance = new InstanceName(Instance, ResourceGroup); var rules = new AggregatorRules(context.Azure, context.Logger); bool ok = await rules.AddAsync(instance, Name, File); return ok ? 0 : 1; diff --git a/src/aggregator-cli/Rules/AggregatorRules.cs b/src/aggregator-cli/Rules/AggregatorRules.cs index ec1dff32..10129a62 100644 --- a/src/aggregator-cli/Rules/AggregatorRules.cs +++ b/src/aggregator-cli/Rules/AggregatorRules.cs @@ -13,6 +13,10 @@ using Microsoft.Azure.Management.ResourceManager.Fluent; using Microsoft.Azure.Management.ResourceManager.Fluent.Authentication; using Microsoft.IdentityModel.Clients.ActiveDirectory; +using Microsoft.TeamFoundation.Core.WebApi; +using Microsoft.TeamFoundation.WorkItemTracking.WebApi; +using Microsoft.VisualStudio.Services.Common; +using Microsoft.VisualStudio.Services.WebApi; using Newtonsoft.Json; namespace aggregator.cli @@ -109,7 +113,7 @@ internal async Task AddAsync(InstanceName instance, string name, string fi bool ok = await UploadRuleFiles(instance, name, baseDirPath); if (ok) { - logger.WriteInfo($"{name} files uploaded to {instance.PlainName}."); + logger.WriteInfo($"All {name} files uploaded to {instance.PlainName}."); } CleanupRuleFiles(baseDirPath); logger.WriteInfo($"Cleaned local working directory."); @@ -162,27 +166,45 @@ Puts a file at path. PUT /api/vfs/{path}/ Creates a directory at path. The path can be nested, e.g. `folder1/folder2`. + + Note: when updating or deleting a file, ETag behavior will apply. You can pass a If-Match: "*" header to disable the ETag check. */ var kudu = new KuduApi(instance, azure, logger); string relativeUrl = $"api/vfs/site/wwwroot/{name}/"; var instances = new AggregatorInstances(azure, logger); - logger.WriteVerbose($"Creating function {name} in {instance.PlainName}..."); using (var client = new HttpClient()) { - using (var request = await kudu.GetRequestAsync(HttpMethod.Put, relativeUrl)) + bool exists = false; + + // check if function already exists + using (var request = await kudu.GetRequestAsync(HttpMethod.Head, relativeUrl)) { + logger.WriteVerbose($"Checking if function {name} already exists in {instance.PlainName}..."); using (var response = await client.SendAsync(request)) { - bool ok = response.IsSuccessStatusCode; - if (!ok) + exists = response.IsSuccessStatusCode; + } + } + + if (!exists) + { + logger.WriteVerbose($"Creating function {name} in {instance.PlainName}..."); + using (var request = await kudu.GetRequestAsync(HttpMethod.Put, relativeUrl)) + { + using (var response = await client.SendAsync(request)) { - logger.WriteError($"Upload failed with {response.ReasonPhrase}"); - return ok; + bool ok = response.IsSuccessStatusCode; + if (!ok) + { + logger.WriteError($"Upload failed with {response.ReasonPhrase}"); + return ok; + } } } + logger.WriteInfo($"Function {name} created."); } - logger.WriteInfo($"Function {name} created."); + var files = Directory.EnumerateFiles(baseDirPath, "*", SearchOption.AllDirectories); foreach (var file in files) { @@ -190,6 +212,8 @@ Puts a file at path. string fileUrl = $"{relativeUrl}{Path.GetFileName(file)}"; using (var request = await kudu.GetRequestAsync(HttpMethod.Put, fileUrl)) { + //HACK -> request.Headers.IfMatch.Add(new EntityTagHeaderValue("*", false)); <- won't work + request.Headers.Add("If-Match", "*"); request.Content = new StringContent(File.ReadAllText(file)); using (var response = await client.SendAsync(request)) { @@ -253,5 +277,61 @@ internal async Task UpdateAsync(InstanceName instance, string name, string } return ok; } + + internal async Task InvokeLocalAsync(string projectName, string @event, int workItemId, string ruleFilePath, bool dryRun, SaveMode saveMode) + { + if (!File.Exists(ruleFilePath)) + { + logger.WriteError($"Rule code not found at {ruleFilePath}"); + return false; + } + + var devopsLogonData = DevOpsLogon.Load().connection; + + logger.WriteVerbose($"Connecting to Azure DevOps using {devopsLogonData.Mode}..."); + var clientCredentials = default(VssCredentials); + if (devopsLogonData.Mode == DevOpsTokenType.PAT) + { + clientCredentials = new VssBasicCredential(devopsLogonData.Mode.ToString(), devopsLogonData.Token); + } + else + { + logger.WriteError($"Azure DevOps Token type {devopsLogonData.Mode} not supported!"); + throw new ArgumentOutOfRangeException(nameof(devopsLogonData.Mode)); + } + + string collectionUrl = devopsLogonData.Url; + using (var devops = new VssConnection(new Uri(collectionUrl), clientCredentials)) + { + await devops.ConnectAsync(); + logger.WriteInfo($"Connected to Azure DevOps"); + + Guid teamProjectId; + string teamProjectName; + using (var projectClient = devops.GetClient()) + { + logger.WriteVerbose($"Reading Azure DevOps project data..."); + var project = await projectClient.GetProject(projectName); + logger.WriteInfo($"Project {projectName} data read."); + teamProjectId = project.Id; + teamProjectName = project.Name; + } + + using (var witClient = devops.GetClient()) + { + logger.WriteVerbose($"Rule code found at {ruleFilePath}"); + string[] ruleCode = File.ReadAllLines(ruleFilePath); + + var engineLogger = new EngineWrapperLogger(logger); + var engine = new Engine.RuleEngine(engineLogger, ruleCode, saveMode); + engine.DryRun = dryRun; + + string result = await engine.ExecuteAsync(collectionUrl, teamProjectId, teamProjectName, devopsLogonData.Token, workItemId, witClient); + logger.WriteInfo($"Rule returned '{result}'"); + + return true; + } + } + } } } diff --git a/src/aggregator-cli/Rules/ConfigureRuleCommand.cs b/src/aggregator-cli/Rules/ConfigureRuleCommand.cs index a2626850..40fb620a 100644 --- a/src/aggregator-cli/Rules/ConfigureRuleCommand.cs +++ b/src/aggregator-cli/Rules/ConfigureRuleCommand.cs @@ -9,39 +9,32 @@ namespace aggregator.cli [Verb("configure.rule", HelpText = "Change a rule configuration.")] class ConfigureRuleCommand : CommandBase { + [Option('g', "resourceGroup", Required = false, Default = "", HelpText = "Azure Resource Group hosting the Aggregator instance.")] + public string ResourceGroup { get; set; } + [Option('i', "instance", Required = true, HelpText = "Aggregator instance name.")] public string Instance { get; set; } [Option('n', "name", Required = true, HelpText = "Aggregator rule name.")] public string Name { get; set; } - [Option('u', "update", SetName = "update", HelpText = "Update the runtime and rule code.")] - public string Update { get; set; } - [Option('d', "disable", SetName = "disable", HelpText = "Disable the rule.")] public bool Disable { get; set; } [Option('e', "enable", SetName = "enable", HelpText = "Enable the rule.")] public bool Enable { get; set; } - [Option("requiredVersion", Required = false, HelpText = "Version of Aggregator Runtime required.")] - public string RequiredVersion { get; set; } - internal override async Task RunAsync() { var context = await Context .WithAzureLogon() .Build(); - var instance = new InstanceName(Instance); + var instance = new InstanceName(Instance, ResourceGroup); var rules = new AggregatorRules(context.Azure, context.Logger); bool ok = false; if (Disable || Enable) { ok = await rules.EnableAsync(instance, Name, Disable); } - if (!string.IsNullOrEmpty(Update)) - { - ok = await rules.UpdateAsync(instance, Name, Update, RequiredVersion); - } return ok ? 0 : 1; } } diff --git a/src/aggregator-cli/Rules/EngineWrapperLogger.cs b/src/aggregator-cli/Rules/EngineWrapperLogger.cs new file mode 100644 index 00000000..e304baa5 --- /dev/null +++ b/src/aggregator-cli/Rules/EngineWrapperLogger.cs @@ -0,0 +1,32 @@ +namespace aggregator.cli +{ + internal class EngineWrapperLogger : IAggregatorLogger + { + private ILogger logger; + + public EngineWrapperLogger(ILogger logger) + { + this.logger = logger; + } + + public void WriteError(string message) + { + logger.WriteError(message); + } + + public void WriteInfo(string message) + { + logger.WriteInfo(message); + } + + public void WriteVerbose(string message) + { + logger.WriteVerbose(message); + } + + public void WriteWarning(string message) + { + logger.WriteWarning(message); + } + } +} \ No newline at end of file diff --git a/src/aggregator-cli/Rules/InvokeRuleCommand.cs b/src/aggregator-cli/Rules/InvokeRuleCommand.cs new file mode 100644 index 00000000..b26e26cb --- /dev/null +++ b/src/aggregator-cli/Rules/InvokeRuleCommand.cs @@ -0,0 +1,59 @@ +using CommandLine; +using System; +using System.Collections.Generic; +using System.Text; +using System.Threading.Tasks; + +namespace aggregator.cli +{ + [Verb("invoke.rule", HelpText = "Executes a rule locally or in an existing Aggregator instance.")] + class InvokeRuleCommand : CommandBase + { + [Option('d', "dryrun", Required = false, Default = false, HelpText = "Real or non-committing run.")] + public bool DryRun { get; set; } + + [Option('p', "project", Required = true, HelpText = "Azure DevOps project name.")] + public string Project { get; set; } + + [Option('e', "event", Required = true, HelpText = "Event to emulate.")] + public string Event { get; set; } + + [Option('w', "workItemId", Required = true, HelpText = "Id of workitem for the emulated event.")] + public int WorkItemId { get; set; } + + [Option('n', "local", SetName = "Local", Required = true, HelpText = "Rule run locally.")] + public bool Local { get; set; } + + [Option('s', "source", SetName = "Local", Required = true, HelpText = "Aggregator rule code.")] + public string Source { get; set; } + + [Option('m', "saveMode", SetName = "Local", Required = false, HelpText = "Save behaviour.")] + public SaveMode SaveMode { get; set; } + + [Option('i', "instance", SetName = "Remote", Required = true, HelpText = "Aggregator instance name.")] + public string Instance { get; set; } + + [Option('g', "resourceGroup", SetName = "Remote", Required = false, Default = "", HelpText = "Azure Resource Group hosting the Aggregator instances.")] + public string ResourceGroup { get; set; } + + internal override async Task RunAsync() + { + var context = await Context + .WithAzureLogon() + .WithDevOpsLogon() + .Build(); + var rules = new AggregatorRules(context.Azure, context.Logger); + if (Local) + { + bool ok = await rules.InvokeLocalAsync(Project, Event, WorkItemId, Source, DryRun, SaveMode); + return ok ? 0 : 1; + } + else + { + var instance = new InstanceName(Instance, ResourceGroup); + context.Logger.WriteWarning("Not implemented yet."); + return 2; + } + } + } +} diff --git a/src/aggregator-cli/Rules/ListRulesCommand.cs b/src/aggregator-cli/Rules/ListRulesCommand.cs index 28ddd586..a56b2b34 100644 --- a/src/aggregator-cli/Rules/ListRulesCommand.cs +++ b/src/aggregator-cli/Rules/ListRulesCommand.cs @@ -9,6 +9,9 @@ namespace aggregator.cli [Verb("list.rules", HelpText = "List the rule in existing Aggregator instance in Azure.")] class ListRulesCommand : CommandBase { + [Option('g', "resourceGroup", Required = false, Default = "", HelpText = "Azure Resource Group hosting the Aggregator instances.")] + public string ResourceGroup { get; set; } + [Option('i', "instance", Required = true, HelpText = "Aggregator instance name.")] public string Instance { get; set; } @@ -17,7 +20,7 @@ internal override async Task RunAsync() var context = await Context .WithAzureLogon() .Build(); - var instance = new InstanceName(Instance); + var instance = new InstanceName(Instance, ResourceGroup); var rules = new AggregatorRules(context.Azure, context.Logger); bool any = false; foreach (var item in await rules.ListAsync(instance)) diff --git a/src/aggregator-cli/Rules/RemoveRuleCommand.cs b/src/aggregator-cli/Rules/RemoveRuleCommand.cs index 59735b46..4c307731 100644 --- a/src/aggregator-cli/Rules/RemoveRuleCommand.cs +++ b/src/aggregator-cli/Rules/RemoveRuleCommand.cs @@ -9,6 +9,9 @@ namespace aggregator.cli [Verb("remove.rule", HelpText = "Remove a rule from existing Aggregator instance in Azure.")] class RemoveRuleCommand : CommandBase { + [Option('g', "resourceGroup", Required = false, Default = "", HelpText = "Azure Resource Group hosting the Aggregator instance.")] + public string ResourceGroup { get; set; } + [Option('i', "instance", Required = true, HelpText = "Aggregator instance name.")] public string Instance { get; set; } @@ -19,10 +22,10 @@ internal override async Task RunAsync() { var context = await Context .WithAzureLogon() - .WithVstsLogon() + .WithDevOpsLogon() .Build(); - var instance = new InstanceName(Instance); - var mappings = new AggregatorMappings(context.Vsts, context.Azure, context.Logger); + var instance = new InstanceName(Instance, ResourceGroup); + var mappings = new AggregatorMappings(context.Devops, context.Azure, context.Logger); bool ok = await mappings.RemoveRuleAsync(instance, Name); var rules = new AggregatorRules(context.Azure, context.Logger); diff --git a/src/aggregator-cli/Rules/UpdateRuleCommand.cs b/src/aggregator-cli/Rules/UpdateRuleCommand.cs new file mode 100644 index 00000000..4618f947 --- /dev/null +++ b/src/aggregator-cli/Rules/UpdateRuleCommand.cs @@ -0,0 +1,38 @@ +using CommandLine; +using System; +using System.Collections.Generic; +using System.Text; +using System.Threading.Tasks; + +namespace aggregator.cli +{ + [Verb("update.rule", HelpText = "Update a rule code and/or runtime.")] + class UpdateRuleCommand : CommandBase + { + [Option('g', "resourceGroup", Required = false, Default = "", HelpText = "Azure Resource Group hosting the Aggregator instance.")] + public string ResourceGroup { get; set; } + + [Option('i', "instance", Required = true, HelpText = "Aggregator instance name.")] + public string Instance { get; set; } + + [Option('n', "name", Required = true, HelpText = "Aggregator rule name.")] + public string Name { get; set; } + + [Option('f', "file", Required = true, HelpText = "Aggregator rule code.")] + public string File { get; set; } + + [Option("requiredVersion", Required = false, HelpText = "Version of Aggregator Runtime required.")] + public string RequiredVersion { get; set; } + + internal override async Task RunAsync() + { + var context = await Context + .WithAzureLogon() + .Build(); + var instance = new InstanceName(Instance, ResourceGroup); + var rules = new AggregatorRules(context.Azure, context.Logger); + bool ok = await rules.UpdateAsync(instance, Name, File, RequiredVersion); + return ok ? 0 : 1; + } + } +} diff --git a/src/aggregator-cli/Rules/run.csx b/src/aggregator-cli/Rules/run.csx index 18cf7735..cbd770d7 100644 --- a/src/aggregator-cli/Rules/run.csx +++ b/src/aggregator-cli/Rules/run.csx @@ -1,5 +1,5 @@ #r "../bin/aggregator-function.dll" -#r "../bin/aggregator-core.dll" +#r "../bin/aggregator-shared.dll" using aggregator; diff --git a/src/aggregator-cli/aggregator-cli.csproj b/src/aggregator-cli/aggregator-cli.csproj index 7089f27a..366648ce 100644 --- a/src/aggregator-cli/aggregator-cli.csproj +++ b/src/aggregator-cli/aggregator-cli.csproj @@ -30,8 +30,8 @@ - - + + @@ -43,7 +43,8 @@ - + + @@ -53,6 +54,12 @@ PreserveNewest + + PreserveNewest + + + PreserveNewest + PreserveNewest diff --git a/src/aggregator-cli/test/test2.rule b/src/aggregator-cli/test/test2.rule index 720a28ca..4f88eac4 100644 --- a/src/aggregator-cli/test/test2.rule +++ b/src/aggregator-cli/test/test2.rule @@ -1,8 +1,8 @@ string message = ""; if (self.Parent != null) { - var parent = store.GetWorkItem(self.Parent); - var children = store.GetWorkItems(parent.Children); + var parent = self.Parent; + var children = parent.Children; if (children.All(c => c.State == "Closed")) { parent.State = "Closed"; diff --git a/src/aggregator-cli/test/test3b.rule b/src/aggregator-cli/test/test3b.rule new file mode 100644 index 00000000..12859fc8 --- /dev/null +++ b/src/aggregator-cli/test/test3b.rule @@ -0,0 +1,2 @@ +.language=Csharp +return self.PreviousRevision.PreviousRevision.Description; diff --git a/src/aggregator-cli/test/test4.rule b/src/aggregator-cli/test/test4.rule new file mode 100644 index 00000000..1dd8618a --- /dev/null +++ b/src/aggregator-cli/test/test4.rule @@ -0,0 +1,8 @@ +.language=Csharp + +var parent = self; +var newChild = store.NewWorkItem("Task"); +newChild.Title = "Brand new child"; +parent.Relations.AddChild(newChild); + +return parent.Title; diff --git a/src/aggregator-core/AggregatorConfiguration.cs b/src/aggregator-core/AggregatorConfiguration.cs deleted file mode 100644 index 541c9c07..00000000 --- a/src/aggregator-core/AggregatorConfiguration.cs +++ /dev/null @@ -1,36 +0,0 @@ -using System; - -namespace aggregator -{ - public enum VstsTokenType - { - Integrated = 0, - PAT = 1, - } - - public class AggregatorConfiguration - { - public AggregatorConfiguration() {} - - static public AggregatorConfiguration Read(Microsoft.Extensions.Configuration.IConfiguration config) - { - var ac = new AggregatorConfiguration(); - Enum.TryParse(config["Aggregator_VstsTokenType"], out VstsTokenType vtt); - ac.VstsTokenType = vtt; - ac.VstsToken = config["Aggregator_VstsToken"]; - return ac; - } - - public void Write(Microsoft.Azure.Management.AppService.Fluent.IWebApp webApp) - { - webApp - .Update() - .WithAppSetting("Aggregator_VstsTokenType", VstsTokenType.ToString()) - .WithAppSetting("Aggregator_VstsToken", VstsToken) - .Apply(); - } - - public VstsTokenType VstsTokenType { get; set; } - public string VstsToken { get; set; } - } -} diff --git a/src/aggregator-function/AssemblyInfo.cs b/src/aggregator-function/AssemblyInfo.cs index cfcd6267..1fb40c12 100644 --- a/src/aggregator-function/AssemblyInfo.cs +++ b/src/aggregator-function/AssemblyInfo.cs @@ -8,8 +8,8 @@ [assembly: AssemblyConfiguration("Release")] #endif [assembly: AssemblyCopyright("TFS Aggregator Team")] -[assembly: AssemblyFileVersion("0.3.4.0")] -[assembly: AssemblyInformationalVersion("0.3.4")] +[assembly: AssemblyFileVersion("0.4.0.0")] +[assembly: AssemblyInformationalVersion("0.4.0")] [assembly: AssemblyProduct("Aggregator CLI")] [assembly: AssemblyTitle("Aggregator Runtime")] -[assembly: AssemblyVersion("0.3.4.0")] +[assembly: AssemblyVersion("0.4.0.0")] diff --git a/src/aggregator-function/AzureFunctionHandler.cs b/src/aggregator-function/AzureFunctionHandler.cs index 26d23ed9..de66630f 100644 --- a/src/aggregator-function/AzureFunctionHandler.cs +++ b/src/aggregator-function/AzureFunctionHandler.cs @@ -4,6 +4,7 @@ using Microsoft.Extensions.Logging; using Newtonsoft.Json; using System; +using System.Linq; using System.Net; using System.Net.Http; using System.Threading.Tasks; @@ -28,13 +29,12 @@ public async Task Run(HttpRequestMessage req) { log.LogDebug($"Context: {context.InvocationId} {context.FunctionName} {context.FunctionDirectory} {context.FunctionAppDirectory}"); - // TODO check expected version - string aggregatorVersion = null; + var aggregatorVersion = GetCustomAttribute()?.InformationalVersion; try { string rule = context.FunctionName; - log.LogInformation($"Welcome to rule '{rule}'"); + log.LogInformation($"Aggregator v{aggregatorVersion} executing rule '{rule}'"); } catch (Exception ex) { @@ -57,12 +57,12 @@ public async Task Run(HttpRequestMessage req) string eventType = data.eventType; // sanity check - if (!VstsEvents.IsValidEvent(eventType) - || data.publisherId != VstsEvents.PublisherId) + if (!DevOpsEvents.IsValidEvent(eventType) + || data.publisherId != DevOpsEvents.PublisherId) { return req.CreateResponse(HttpStatusCode.BadRequest, new { - error = "Not a good VSTS post..." + error = "Not a good Azure DevOps post..." }); } @@ -77,7 +77,7 @@ public async Task Run(HttpRequestMessage req) var wrapper = new RuleWrapper(configuration, logger, context.FunctionName, context.FunctionDirectory); try { - string execResult = await wrapper.Execute(aggregatorVersion, data); + string execResult = await wrapper.Execute(data); if (string.IsNullOrEmpty(execResult)) { @@ -106,5 +106,14 @@ public async Task Run(HttpRequestMessage req) return resp; } } + + private static T GetCustomAttribute() + where T : Attribute + { + return System.Reflection.Assembly + .GetExecutingAssembly() + .GetCustomAttributes(typeof(T), false) + .FirstOrDefault() as T; + } } } diff --git a/src/aggregator-function/Engine/EngineContext.cs b/src/aggregator-function/Engine/EngineContext.cs deleted file mode 100644 index aacb7798..00000000 --- a/src/aggregator-function/Engine/EngineContext.cs +++ /dev/null @@ -1,22 +0,0 @@ -using Microsoft.TeamFoundation.WorkItemTracking.WebApi; -using Microsoft.TeamFoundation.WorkItemTracking.WebApi.Models; -using System; -using System.Collections.Generic; -using System.Linq; - -namespace aggregator.Engine -{ - public class EngineContext - { - internal EngineContext(WorkItemTrackingHttpClient client, IAggregatorLogger logger) - { - Client = client; - Logger = logger; - Tracker = new Tracker(); - } - - internal WorkItemTrackingHttpClient Client { get; } - internal IAggregatorLogger Logger { get; } - internal Tracker Tracker { get; } - } -} diff --git a/src/aggregator-function/Engine/WorkItemStore.cs b/src/aggregator-function/Engine/WorkItemStore.cs deleted file mode 100644 index 3fbb8949..00000000 --- a/src/aggregator-function/Engine/WorkItemStore.cs +++ /dev/null @@ -1,91 +0,0 @@ -using Microsoft.TeamFoundation.WorkItemTracking.WebApi.Models; -using System; -using System.Collections.Generic; -using System.Linq; - -namespace aggregator.Engine -{ - public class WorkItemStore - { - private EngineContext _context; - - public WorkItemStore(EngineContext context) - { - _context = context; - } - - public WorkItemWrapper GetWorkItem(int id) - { - _context.Logger.WriteVerbose($"Getting workitem {id}"); - - return _context.Tracker.LoadWorkItem(id, (workItemId) => - { - _context.Logger.WriteInfo($"Loading workitem {workItemId}"); - var item = _context.Client.GetWorkItemAsync(workItemId, expand: WorkItemExpand.All).Result; - return new WorkItemWrapper(_context, item); - }); - } - - public WorkItemWrapper GetWorkItem(WorkItemRelationWrapper item) - { - int id = int.Parse( - item.Url.Substring( - item.Url.LastIndexOf('/') + 1)); - return GetWorkItem(id); - } - - public IList GetWorkItems(IEnumerable ids) - { - string idList = ids.Aggregate("", (s, i) => s += $",{i}"); - _context.Logger.WriteVerbose($"Getting workitems {idList.Substring(1)}"); - return _context.Tracker.LoadWorkItems(ids, (workItemIds) => - { - string idList2 = workItemIds.Aggregate("", (s, i) => s += $",{i}"); - _context.Logger.WriteVerbose($"Loading workitems {idList2.Substring(1)}"); - var items = _context.Client.GetWorkItemsAsync(workItemIds, expand: WorkItemExpand.All).Result; - return items.ConvertAll(i => new WorkItemWrapper(_context, i)); - }); - } - - public IList GetWorkItems(IEnumerable collection) - { - var ids = new List(); - foreach (var item in collection) - { - ids.Add( - int.Parse( - item.Url.Substring( - item.Url.LastIndexOf('/') + 1))); - } - - return GetWorkItems(ids); - } - - internal (int created, int updated) SaveChanges() - { - int created = 0; - int updated = 0; - foreach (var item in _context.Tracker.NewWorkItems) - { - _context.Logger.WriteInfo($"Creating a {item.WorkItemType} workitem in {item.TeamProject}"); - _context.Client.CreateWorkItemAsync( - item.Changes, - item.TeamProject, - item.WorkItemType - ); - created++; - } - - foreach (var item in _context.Tracker.ChangedWorkItems) - { - _context.Logger.WriteInfo($"Updating workitem {item.Id}"); - _context.Client.UpdateWorkItemAsync( - item.Changes, - item.Id.Value - ); - updated++; - } - return (created, updated); - } - } -} diff --git a/src/aggregator-function/ForwarderLogger.cs b/src/aggregator-function/ForwarderLogger.cs index 8f9cfcc9..aaf0b4ea 100644 --- a/src/aggregator-function/ForwarderLogger.cs +++ b/src/aggregator-function/ForwarderLogger.cs @@ -3,6 +3,9 @@ namespace aggregator { + /// + /// Forwards to Azure Function logging subsystem + /// internal class ForwarderLogger : IAggregatorLogger { private Microsoft.Extensions.Logging.ILogger log; diff --git a/src/aggregator-function/RuleWrapper.cs b/src/aggregator-function/RuleWrapper.cs index d1ef143d..c296c64f 100644 --- a/src/aggregator-function/RuleWrapper.cs +++ b/src/aggregator-function/RuleWrapper.cs @@ -32,94 +32,46 @@ public RuleWrapper(AggregatorConfiguration configuration, IAggregatorLogger logg this.functionDirectory = functionDirectory; } - internal async Task Execute(string aggregatorVersion, dynamic data) + internal async Task Execute(dynamic data) { - if (string.IsNullOrEmpty(aggregatorVersion)) - { - aggregatorVersion = "0.1"; - } - string collectionUrl = data.resourceContainers.collection.baseUrl; string eventType = data.eventType; int workItemId = (eventType != "workitem.updated") ? data.resource.id : data.resource.workItemId; + Guid teamProjectId = data.resourceContainers.project.id; + string teamProjectName = data.resourceContainers.project.name; - logger.WriteVerbose($"Connecting to VSTS using {configuration.VstsTokenType}..."); + logger.WriteVerbose($"Connecting to Azure DevOps using {configuration.DevOpsTokenType}..."); var clientCredentials = default(VssCredentials); - if (configuration.VstsTokenType == VstsTokenType.PAT) + if (configuration.DevOpsTokenType == DevOpsTokenType.PAT) { - clientCredentials = new VssBasicCredential(configuration.VstsTokenType.ToString(), configuration.VstsToken); + clientCredentials = new VssBasicCredential(configuration.DevOpsTokenType.ToString(), configuration.DevOpsToken); } else { - logger.WriteError($"VSTS Token type {configuration.VstsTokenType} not supported!"); - throw new ArgumentOutOfRangeException(nameof(configuration.VstsTokenType)); + logger.WriteError($"Azure DevOps Token type {configuration.DevOpsTokenType} not supported!"); + throw new ArgumentOutOfRangeException(nameof(configuration.DevOpsTokenType)); } - var vsts = new VssConnection(new Uri(collectionUrl), clientCredentials); - await vsts.ConnectAsync(); - logger.WriteInfo($"Connected to VSTS"); - var witClient = vsts.GetClient(); - var context = new Engine.EngineContext(witClient, logger); - var store = new Engine.WorkItemStore(context); - var self = store.GetWorkItem(workItemId); - logger.WriteInfo($"Initial WorkItem {workItemId} retrieved from {collectionUrl}"); - string ruleFilePath = Path.Combine(functionDirectory, $"{ruleName}.rule"); - if (!File.Exists(ruleFilePath)) + using (var devops = new VssConnection(new Uri(collectionUrl), clientCredentials)) { - logger.WriteError($"Rule code not found at {ruleFilePath}"); - return "Rule file not found!"; - } + await devops.ConnectAsync(); + logger.WriteInfo($"Connected to Azure DevOps"); + using (var witClient = devops.GetClient()) + { + string ruleFilePath = Path.Combine(functionDirectory, $"{ruleName}.rule"); + if (!File.Exists(ruleFilePath)) + { + logger.WriteError($"Rule code not found at {ruleFilePath}"); + return "Rule file not found!"; + } - logger.WriteVerbose($"Rule code found at {ruleFilePath}"); - string ruleCode = File.ReadAllText(ruleFilePath); + logger.WriteVerbose($"Rule code found at {ruleFilePath}"); + string[] ruleCode = File.ReadAllLines(ruleFilePath); - logger.WriteInfo($"Executing Rule..."); - var globals = new Engine.Globals { - self = self, - store = store - }; - - var types = new List() { - typeof(object), - typeof(System.Linq.Enumerable), - typeof(System.Collections.Generic.CollectionExtensions) - }; - var references = types.ConvertAll(t => t.Assembly).Distinct(); - - var scriptOptions = ScriptOptions.Default - .WithEmitDebugInformation(true) - .WithReferences(references) - // Add namespaces - .WithImports("System","System.Linq","System.Collections.Generic"); - var roslynScript = CSharpScript.Create( - code: ruleCode, - options: scriptOptions, - globalsType: typeof(Engine.Globals)); - var result = await roslynScript.RunAsync(globals); - if (result.Exception != null) - { - logger.WriteError($"Rule failed with {result.Exception}"); - } - else if(result.ReturnValue != null) - { - logger.WriteInfo($"Rule succeeded with {result.ReturnValue}"); - } - else - { - logger.WriteInfo($"Rule succeeded, no return value"); - } + var engine = new Engine.RuleEngine(logger, ruleCode, configuration.SaveMode); - logger.WriteVerbose($"Post-execution, save any change..."); - var saveRes = store.SaveChanges(); - if (saveRes.created + saveRes.updated > 0) - { - logger.WriteInfo($"Changes saved to VSTS: {saveRes.created} created, {saveRes.updated} updated."); - } - else - { - logger.WriteInfo($"No changes saved to VSTS."); + return await engine.ExecuteAsync(collectionUrl, teamProjectId, teamProjectName, configuration.DevOpsToken, workItemId, witClient); + } } - - return result.ReturnValue; } } } diff --git a/src/aggregator-function/aggregator-function.csproj b/src/aggregator-function/aggregator-function.csproj index 4ef5d5e0..1c588720 100644 --- a/src/aggregator-function/aggregator-function.csproj +++ b/src/aggregator-function/aggregator-function.csproj @@ -9,16 +9,12 @@ DEBUG;TRACE - - + - - - Always @@ -33,14 +29,9 @@ Never - - PreserveNewest - - - PreserveNewest - - - PreserveNewest - + + + + diff --git a/src/aggregator-function/aggregator-manifest.ini b/src/aggregator-function/aggregator-manifest.ini index d8a320c2..b52b6281 100644 --- a/src/aggregator-function/aggregator-manifest.ini +++ b/src/aggregator-function/aggregator-manifest.ini @@ -1 +1 @@ -version=0.3.4 \ No newline at end of file +version=0.4.0 \ No newline at end of file diff --git a/src/aggregator-function/test/function.json b/src/aggregator-function/test/function.json deleted file mode 100644 index 69b083a4..00000000 --- a/src/aggregator-function/test/function.json +++ /dev/null @@ -1,16 +0,0 @@ -{ - "bindings": [ - { - "type": "httpTrigger", - "direction": "in", - "webHookType": "genericJson", - "name": "req" - }, - { - "type": "http", - "direction": "out", - "name": "res" - } - ], - "disabled": false -} \ No newline at end of file diff --git a/src/aggregator-function/test/run.csx b/src/aggregator-function/test/run.csx deleted file mode 100644 index 18cf7735..00000000 --- a/src/aggregator-function/test/run.csx +++ /dev/null @@ -1,10 +0,0 @@ -#r "../bin/aggregator-function.dll" -#r "../bin/aggregator-core.dll" - -using aggregator; - -public static async Task Run(HttpRequestMessage req, ILogger logger, ExecutionContext context) -{ - var handler = new AzureFunctionHandler(logger, context); - return await handler.Run(req); -} diff --git a/src/aggregator-function/test/test.rule b/src/aggregator-function/test/test.rule deleted file mode 100644 index 0045cd73..00000000 --- a/src/aggregator-function/test/test.rule +++ /dev/null @@ -1,29 +0,0 @@ -/* -if (self.Parent != null) -{ - var parent = store.GetWorkItem(self.Parent); - var children = store.GetWorkItems(parent.Children); - if (children.All(c => c.State == "Closed")) - { - parent.State = "Closed"; - } - parent.Description = parent.Description + " aggregator was here."; -} -return $"Hello { self.WorkItemType } #{ self.Id } - { self.Title }!"; -return self.PreviousRevision.PreviousRevision.Description; -*/ -if (self.ParentLink != null) -{ - var parent = self.Parent; - var children = parent.Children; - if (children.All(c => c.State == "Closed")) - { - parent.State = "Closed"; - return "Had to close parent"; - } else { - return "Not all children are closed"; - } - //parent.Description = parent.Description + " aggregator was here."; -} else { - return "No parent!"; -} diff --git a/src/aggregator-ruleng/AssemblyInfo.cs b/src/aggregator-ruleng/AssemblyInfo.cs new file mode 100644 index 00000000..61c91eef --- /dev/null +++ b/src/aggregator-ruleng/AssemblyInfo.cs @@ -0,0 +1,15 @@ +using System; +using System.Reflection; + +[assembly: AssemblyCompany("TFS Aggregator Team")] +#if DEBUG +[assembly: AssemblyConfiguration("Debug")] +#else +[assembly: AssemblyConfiguration("Release")] +#endif +[assembly: AssemblyCopyright("TFS Aggregator Team")] +[assembly: AssemblyFileVersion("0.4.0.0")] +[assembly: AssemblyInformationalVersion("0.4.0")] +[assembly: AssemblyProduct("Aggregator CLI")] +[assembly: AssemblyTitle("Aggregator Rule Engine")] +[assembly: AssemblyVersion("0.4.0.0")] diff --git a/src/aggregator-ruleng/BatchRequest.cs b/src/aggregator-ruleng/BatchRequest.cs new file mode 100644 index 00000000..18c6e7fa --- /dev/null +++ b/src/aggregator-ruleng/BatchRequest.cs @@ -0,0 +1,14 @@ +using System; +using System.Collections.Generic; +using System.Text; + +namespace aggregator.Engine +{ + public class BatchRequest + { + public string method { get; set; } + public Dictionary headers { get; set; } + public object[] body { get; set; } + public string uri { get; set; } + } +} diff --git a/src/aggregator-function/Engine/CoreFieldRefNames.cs b/src/aggregator-ruleng/CoreFieldRefNames.cs similarity index 100% rename from src/aggregator-function/Engine/CoreFieldRefNames.cs rename to src/aggregator-ruleng/CoreFieldRefNames.cs diff --git a/src/aggregator-function/Engine/CoreRelationRefNames.cs b/src/aggregator-ruleng/CoreRelationRefNames.cs similarity index 100% rename from src/aggregator-function/Engine/CoreRelationRefNames.cs rename to src/aggregator-ruleng/CoreRelationRefNames.cs diff --git a/src/aggregator-ruleng/DirectivesParser.cs b/src/aggregator-ruleng/DirectivesParser.cs new file mode 100644 index 00000000..baf8d778 --- /dev/null +++ b/src/aggregator-ruleng/DirectivesParser.cs @@ -0,0 +1,85 @@ +using System; +using System.Collections.Generic; +using System.Text; + +namespace aggregator.Engine +{ + /// + /// Scan the top lines of a script looking for directives, whose lines start with a . + /// + internal class DirectivesParser + { + int firstCodeLine = 0; + private readonly IAggregatorLogger logger; + string[] ruleCode; + + internal DirectivesParser(IAggregatorLogger logger, string[] ruleCode) + { + this.ruleCode = ruleCode; + this.logger = logger; + + //defaults + Language = Languages.Csharp; + } + + /// + /// Grab directives + /// + internal bool Parse() + { + while (firstCodeLine < ruleCode.Length + && ruleCode[firstCodeLine].Length > 0 + && ruleCode[firstCodeLine][0] == '.') + { + string directive = ruleCode[firstCodeLine].Substring(1); + var parts = directive.Split('='); + + switch (parts[0].ToLowerInvariant()) + { + case "lang": + case "language": + if (parts.Length < 2) + { + logger.WriteWarning($"Unrecognized directive {directive}"); + return false; + } + else + { + switch (parts[1].ToUpperInvariant()) + { + case "C#": + case "CS": + case "CSHARP": + Language = Languages.Csharp; + break; + default: + logger.WriteWarning($"Unrecognized language {parts[1]}"); + return false; + } + } + break; + default: + logger.WriteWarning($"Unrecognized directive {directive}"); + return false; + }//switch + + firstCodeLine++; + }//while + return true; + } + + internal string GetRuleCode() + { + return string.Join(Environment.NewLine, ruleCode, firstCodeLine, ruleCode.Length - firstCodeLine); + } + + // directives + + internal enum Languages + { + Csharp + } + + internal Languages Language { get; private set; } + } +} diff --git a/src/aggregator-ruleng/EngineContext.cs b/src/aggregator-ruleng/EngineContext.cs new file mode 100644 index 00000000..9eba75cd --- /dev/null +++ b/src/aggregator-ruleng/EngineContext.cs @@ -0,0 +1,28 @@ +using Microsoft.TeamFoundation.WorkItemTracking.WebApi; +using Microsoft.TeamFoundation.WorkItemTracking.WebApi.Models; +using System; +using System.Collections.Generic; +using System.Linq; + +namespace aggregator.Engine +{ + public class EngineContext + { + public EngineContext(WorkItemTrackingHttpClientBase client, Guid projectId, string projectName, string personalAccessToken, IAggregatorLogger logger) + { + Client = client; + Logger = logger; + Tracker = new Tracker(); + ProjectId = projectId; + ProjectName = projectName; + PersonalAccessToken = personalAccessToken; + } + + public Guid ProjectId { get; internal set; } + public string ProjectName { get; internal set; } + public string PersonalAccessToken { get; internal set; } + internal WorkItemTrackingHttpClientBase Client { get; } + internal IAggregatorLogger Logger { get; } + internal Tracker Tracker { get; } + } +} diff --git a/src/aggregator-function/Engine/Globals.cs b/src/aggregator-ruleng/Globals.cs similarity index 100% rename from src/aggregator-function/Engine/Globals.cs rename to src/aggregator-ruleng/Globals.cs diff --git a/src/aggregator-function/ILogger.cs b/src/aggregator-ruleng/IAggregatorLogger.cs similarity index 87% rename from src/aggregator-function/ILogger.cs rename to src/aggregator-ruleng/IAggregatorLogger.cs index 522dc69f..c7214022 100644 --- a/src/aggregator-function/ILogger.cs +++ b/src/aggregator-ruleng/IAggregatorLogger.cs @@ -4,7 +4,7 @@ namespace aggregator { - interface IAggregatorLogger + public interface IAggregatorLogger { void WriteVerbose(string message); diff --git a/src/aggregator-ruleng/RelationPatch.cs b/src/aggregator-ruleng/RelationPatch.cs new file mode 100644 index 00000000..c5b0fb0f --- /dev/null +++ b/src/aggregator-ruleng/RelationPatch.cs @@ -0,0 +1,9 @@ +namespace aggregator.Engine +{ + internal class RelationPatch + { + public string rel { get; set; } + public string url { get; set; } + public object attributes { get; set; } + } +} \ No newline at end of file diff --git a/src/aggregator-ruleng/RuleEngine.cs b/src/aggregator-ruleng/RuleEngine.cs new file mode 100644 index 00000000..d99ce5fa --- /dev/null +++ b/src/aggregator-ruleng/RuleEngine.cs @@ -0,0 +1,129 @@ +using System; +using System.Collections.Generic; +using System.Linq; +using System.Threading.Tasks; +using Microsoft.CodeAnalysis.CSharp.Scripting; +using Microsoft.CodeAnalysis.Scripting; +using Microsoft.TeamFoundation.WorkItemTracking.WebApi; + +namespace aggregator.Engine +{ + public enum EngineState + { + Unknown, + Success, + Error + } + + /// + /// Entry point to execute rules, indipendent of environment + /// + public class RuleEngine + { + private readonly IAggregatorLogger logger; + private readonly Script roslynScript; + private readonly SaveMode saveMode; + + public RuleEngine(IAggregatorLogger logger, string[] ruleCode, SaveMode mode) + { + State = EngineState.Unknown; + + this.logger = logger; + this.saveMode = mode; + + var directives = new DirectivesParser(logger, ruleCode); + if (!directives.Parse()) + { + State = EngineState.Error; + return; + } + + if (directives.Language == DirectivesParser.Languages.Csharp) + { + var types = new List() { + typeof(object), + typeof(System.Linq.Enumerable), + typeof(System.Collections.Generic.CollectionExtensions), + typeof(Microsoft.VisualStudio.Services.WebApi.IdentityRef) + }; + var references = types.ConvertAll(t => t.Assembly).Distinct(); + + var scriptOptions = ScriptOptions.Default + .WithEmitDebugInformation(true) + .WithReferences(references) + // Add namespaces + .WithImports( + "System", + "System.Linq", + "System.Collections.Generic", + "Microsoft.VisualStudio.Services.WebApi" + ); + + this.roslynScript = CSharpScript.Create( + code: directives.GetRuleCode(), + options: scriptOptions, + globalsType: typeof(Globals)); + } + else + { + logger.WriteError($"Cannot execute rule: language is not supported."); + State = EngineState.Error; + } + } + + /// + /// State is used by unit tests + /// + public EngineState State { get; private set; } + public bool DryRun { get; set; } + + public async Task ExecuteAsync(string collectionUrl, Guid projectId, string projectName, string personalAccessToken, int workItemId, WorkItemTrackingHttpClientBase witClient) + { + if (State == EngineState.Error) + { + return string.Empty; + } + + var context = new EngineContext(witClient, projectId, projectName, personalAccessToken, logger); + var store = new WorkItemStore(context); + var self = store.GetWorkItem(workItemId); + logger.WriteInfo($"Initial WorkItem {workItemId} retrieved from {collectionUrl}"); + + var globals = new Globals + { + self = self, + store = store + }; + + logger.WriteInfo($"Executing Rule..."); + var result = await roslynScript.RunAsync(globals); + if (result.Exception != null) + { + logger.WriteError($"Rule failed with {result.Exception}"); + State = EngineState.Error; + } + else if (result.ReturnValue != null) + { + logger.WriteInfo($"Rule succeeded with {result.ReturnValue}"); + } + else + { + logger.WriteInfo($"Rule succeeded, no return value"); + } + State = EngineState.Success; + + logger.WriteVerbose($"Post-execution, save any change (mode {saveMode})..."); + var saveRes = await store.SaveChanges(saveMode, !this.DryRun); + if (saveRes.created + saveRes.updated > 0) + { + logger.WriteInfo($"Changes saved to Azure DevOps (mode {saveMode}): {saveRes.created} created, {saveRes.updated} updated."); + } + else + { + logger.WriteInfo($"No changes saved to Azure DevOps."); + } + + return result.ReturnValue; + } + } +} diff --git a/src/aggregator-function/Engine/Tracker.cs b/src/aggregator-ruleng/Tracker.cs similarity index 89% rename from src/aggregator-function/Engine/Tracker.cs rename to src/aggregator-ruleng/Tracker.cs index aafba88c..1b82d202 100644 --- a/src/aggregator-function/Engine/Tracker.cs +++ b/src/aggregator-ruleng/Tracker.cs @@ -22,12 +22,18 @@ internal TrackedWrapper(int id, WorkItemWrapper wrapper) } } + private int watermark = -1; IDictionary, TrackedWrapper> tracked = new Dictionary, TrackedWrapper>(); internal Tracker() { } + internal int GetNextWatermark() + { + return watermark--; + } + internal void TrackExisting(WorkItemWrapper workItemWrapper) { if (IsTracked(workItemWrapper)) @@ -71,7 +77,11 @@ internal IList LoadWorkItems( .GroupBy(k => tracked.ContainsKey(k)) .ToDictionary(g => g.Key, g => g.ToList()); - var inMemory = tracked.Where(w => groups[true].Contains(w.Key)).Select(w => w.Value.Current); + var inMemory = tracked + .Where(w => groups.ContainsKey(true) + ? groups[true].Contains(w.Key) + : false) + .Select(w => w.Value.Current); var loaded = loader(groups[false].Select(k => k.Value)); return inMemory.Union(loaded).ToList(); @@ -84,7 +94,7 @@ internal bool IsTracked(WorkItemWrapper workItemWrapper) internal IEnumerable NewWorkItems => tracked - .Where(w => !w.Value.Current.IsReadOnly && w.Value.Current.IsDirty && w.Value.Current.IsNew) + .Where(w => !w.Value.Current.IsReadOnly && w.Value.Current.IsNew) .Select(w=>w.Value.Current); internal IEnumerable ChangedWorkItems diff --git a/src/aggregator-ruleng/WorkItemBatchPostResponse.cs b/src/aggregator-ruleng/WorkItemBatchPostResponse.cs new file mode 100644 index 00000000..bb024f8f --- /dev/null +++ b/src/aggregator-ruleng/WorkItemBatchPostResponse.cs @@ -0,0 +1,21 @@ +using System; +using System.Collections.Generic; +using System.Text; +using Newtonsoft.Json; + +namespace aggregator.Engine +{ + public class WorkItemBatchPostResponse + { + public int count { get; set; } + [JsonProperty("value")] + public List values { get; set; } + + public class Value + { + public int code { get; set; } + public Dictionary headers { get; set; } + public string body { get; set; } + } + } +} diff --git a/src/aggregator-function/Engine/WorkItemId.cs b/src/aggregator-ruleng/WorkItemId.cs similarity index 93% rename from src/aggregator-function/Engine/WorkItemId.cs rename to src/aggregator-ruleng/WorkItemId.cs index 90fadc93..dfc88ae1 100644 --- a/src/aggregator-function/Engine/WorkItemId.cs +++ b/src/aggregator-ruleng/WorkItemId.cs @@ -6,10 +6,8 @@ namespace aggregator.Engine { public class TemporaryWorkItemId : WorkItemId { - private static int watermark = -1; - - public TemporaryWorkItemId() - : base(watermark--) + internal TemporaryWorkItemId(Tracker tracker) + : base(tracker.GetNextWatermark()) { } } diff --git a/src/aggregator-function/Engine/WorkItemRelationWrapper.cs b/src/aggregator-ruleng/WorkItemRelationWrapper.cs similarity index 93% rename from src/aggregator-function/Engine/WorkItemRelationWrapper.cs rename to src/aggregator-ruleng/WorkItemRelationWrapper.cs index fe7121e8..102ba2ce 100644 --- a/src/aggregator-function/Engine/WorkItemRelationWrapper.cs +++ b/src/aggregator-ruleng/WorkItemRelationWrapper.cs @@ -23,7 +23,7 @@ internal WorkItemRelationWrapper(WorkItemWrapper item, string type, string url, { Rel = type, Url = url, - Attributes = { { "comment", comment } } + Attributes = new Dictionary { { "comment", comment } } }; } diff --git a/src/aggregator-function/Engine/WorkItemRelationWrapperCollection.cs b/src/aggregator-ruleng/WorkItemRelationWrapperCollection.cs similarity index 86% rename from src/aggregator-function/Engine/WorkItemRelationWrapperCollection.cs rename to src/aggregator-ruleng/WorkItemRelationWrapperCollection.cs index d3087397..4a18c54c 100644 --- a/src/aggregator-function/Engine/WorkItemRelationWrapperCollection.cs +++ b/src/aggregator-ruleng/WorkItemRelationWrapperCollection.cs @@ -39,14 +39,13 @@ private void AddRelation(WorkItemRelationWrapper item) { Operation = Operation.Add, Path = "/relations/-", - Value = new + Value = new RelationPatch { rel = item.Rel, url = item.Url, - attributes = new - { - comment = item.Attributes["comment"] - } + attributes = item.Attributes != null && item.Attributes.ContainsKey("comment") + ? new { comment = item.Attributes["comment"] } + : null } }); @@ -90,6 +89,18 @@ public void Add(WorkItemRelationWrapper item) AddRelation(item); } + public void AddChild(WorkItemWrapper child) + { + var r = new WorkItemRelationWrapper(child, CoreRelationRefNames.Children, child.Url, string.Empty); + AddRelation(r); + } + + public void AddParent(WorkItemWrapper parent) + { + var r = new WorkItemRelationWrapper(parent, CoreRelationRefNames.Parent, parent.Url, string.Empty); + AddRelation(r); + } + public void AddLink(string type, string url, string comment) { AddRelation(new WorkItemRelationWrapper( diff --git a/src/aggregator-ruleng/WorkItemStore.cs b/src/aggregator-ruleng/WorkItemStore.cs new file mode 100644 index 00000000..1fafb293 --- /dev/null +++ b/src/aggregator-ruleng/WorkItemStore.cs @@ -0,0 +1,469 @@ +using Microsoft.TeamFoundation.WorkItemTracking.WebApi.Models; +using Microsoft.VisualStudio.Services.WebApi.Patch.Json; +using Newtonsoft.Json; +using Newtonsoft.Json.Linq; +using System; +using System.Collections.Generic; +using System.Linq; +using System.Net.Http; +using System.Net.Http.Headers; +using System.Text; +using System.Threading.Tasks; + +namespace aggregator.Engine +{ + public class WorkItemStore + { + private EngineContext _context; + + public WorkItemStore(EngineContext context) + { + _context = context; + } + + public WorkItemWrapper GetWorkItem(int id) + { + _context.Logger.WriteVerbose($"Getting workitem {id}"); + + return _context.Tracker.LoadWorkItem(id, (workItemId) => + { + _context.Logger.WriteInfo($"Loading workitem {workItemId}"); + var item = _context.Client.GetWorkItemAsync(workItemId, expand: WorkItemExpand.All).Result; + return new WorkItemWrapper(_context, item); + }); + } + + public WorkItemWrapper GetWorkItem(WorkItemRelationWrapper item) + { + int id = int.Parse( + item.Url.Substring( + item.Url.LastIndexOf('/') + 1)); + return GetWorkItem(id); + } + + public IList GetWorkItems(IEnumerable ids) + { + string idList = ids.Aggregate("", (s, i) => s += $",{i}"); + _context.Logger.WriteVerbose($"Getting workitems {idList.Substring(1)}"); + return _context.Tracker.LoadWorkItems(ids, (workItemIds) => + { + string idList2 = workItemIds.Aggregate("", (s, i) => s += $",{i}"); + _context.Logger.WriteInfo($"Loading workitems {idList2.Substring(1)}"); + var items = _context.Client.GetWorkItemsAsync(workItemIds, expand: WorkItemExpand.All).Result; + return items.ConvertAll(i => new WorkItemWrapper(_context, i)); + }); + } + + public IList GetWorkItems(IEnumerable collection) + { + var ids = new List(); + foreach (var item in collection) + { + ids.Add( + int.Parse( + item.Url.Substring( + item.Url.LastIndexOf('/') + 1))); + } + + return GetWorkItems(ids); + } + + public WorkItemWrapper NewWorkItem(string workItemType) + { + var item = new WorkItem() + { + Fields = new Dictionary() { + { CoreFieldRefNames.WorkItemType, workItemType }, + { CoreFieldRefNames.TeamProject, _context.ProjectId.ToString() } + }, + Relations = new List(), + Links = new Microsoft.VisualStudio.Services.WebApi.ReferenceLinks() + }; + var wrapper = new WorkItemWrapper(_context, item); + _context.Logger.WriteVerbose($"Made new workitem with temporary id {wrapper.Id.Value}"); + //HACK + string baseUriString = _context.Client.BaseAddress.AbsoluteUri; + item.Url = $"{baseUriString}/_apis/wit/workitems/{wrapper.Id.Value}"; + return wrapper; + } + + public async Task<(int created, int updated)> SaveChanges(SaveMode mode, bool commit) + { + switch (mode) + { + case SaveMode.Default: + _context.Logger.WriteVerbose($"No save mode specified, assuming {SaveMode.TwoPhases}."); + goto case SaveMode.TwoPhases; + case SaveMode.Item: + return await SaveChanges_ByItem(commit); + case SaveMode.Batch: + return await SaveChanges_Batch(commit); + case SaveMode.TwoPhases: + return await SaveChanges_TwoPhases(commit); + default: + throw new ApplicationException($"Unsupported save mode: {mode}."); + } + } + + private async Task<(int created, int updated)> SaveChanges_ByItem(bool commit) + { + int created = 0; + int updated = 0; + foreach (var item in _context.Tracker.NewWorkItems) + { + if (commit) + { + _context.Logger.WriteInfo($"Creating a {item.WorkItemType} workitem in {_context.ProjectId}"); + var wi = await _context.Client.CreateWorkItemAsync( + item.Changes, + _context.ProjectId, + item.WorkItemType + ); + } + else + { + _context.Logger.WriteInfo($"Dry-run mode: should create a {item.WorkItemType} workitem in {_context.ProjectId}"); + } + created++; + } + + foreach (var item in _context.Tracker.ChangedWorkItems) + { + if (commit) + { + _context.Logger.WriteInfo($"Updating workitem {item.Id}"); + var wi = await _context.Client.UpdateWorkItemAsync( + item.Changes, + item.Id.Value + ); + } + else + { + _context.Logger.WriteInfo($"Dry-run mode: should update workitem {item.Id} in {_context.ProjectId}"); + } + updated++; + } + return (created, updated); + } + + private async Task<(int created, int updated)> SaveChanges_Batch(bool commit) + { + // see https://github.com/redarrowlabs/vsts-restapi-samplecode/blob/master/VSTSRestApiSamples/WorkItemTracking/Batch.cs + // and https://docs.microsoft.com/en-us/rest/api/vsts/wit/workitembatchupdate?view=vsts-rest-4.1 + // BUG this code won't work if there is a relation between a new (id<0) work item and an existing one (id>0): it is an API limit + + const string ApiVersion = "api-version=4.1"; + + int created = _context.Tracker.NewWorkItems.Count(); + int updated = _context.Tracker.ChangedWorkItems.Count(); + + string baseUriString = _context.Client.BaseAddress.AbsoluteUri; + + BatchRequest[] batchRequests = new BatchRequest[created + updated]; + Dictionary headers = new Dictionary() { + { "Content-Type", "application/json-patch+json" } + }; + string credentials = Convert.ToBase64String(ASCIIEncoding.ASCII.GetBytes($":{_context.PersonalAccessToken}")); + + int index = 0; + + foreach (var item in _context.Tracker.NewWorkItems) + { + _context.Logger.WriteInfo($"Found a request for a new {item.WorkItemType} workitem in {_context.ProjectName}"); + + batchRequests[index++] = new BatchRequest + { + method = "PATCH", + uri = $"/{_context.ProjectName}/_apis/wit/workitems/${item.WorkItemType}?{ApiVersion}", + headers = headers, + body = item.Changes + .Where(c => c.Operation != Microsoft.VisualStudio.Services.WebApi.Patch.Operation.Test) + .ToArray() + }; + } + foreach (var item in _context.Tracker.ChangedWorkItems) + { + _context.Logger.WriteInfo($"Found a request to update workitem {item.Id.Value} in {_context.ProjectName}"); + + batchRequests[index++] = new BatchRequest + { + method = "PATCH", + uri = $"/_apis/wit/workitems/{item.Id.Value}?{ApiVersion}", + headers = headers, + body = item.Changes + .Where(c => c.Operation != Microsoft.VisualStudio.Services.WebApi.Patch.Operation.Test) + .ToArray() + }; + } + var converters = new JsonConverter[] { new JsonPatchOperationConverter() }; + string requestBody = JsonConvert.SerializeObject(batchRequests, Formatting.Indented, converters); + _context.Logger.WriteVerbose(requestBody); + + if (commit) + { + using (var client = new HttpClient()) + { + client.DefaultRequestHeaders.Accept.Clear(); + client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json")); + client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", credentials); + + var batchRequest = new StringContent(requestBody, Encoding.UTF8, "application/json"); + var method = new HttpMethod("POST"); + + // send the request + var request = new HttpRequestMessage(method, $"{baseUriString}/_apis/wit/$batch?{ApiVersion}") { Content = batchRequest }; + var response = client.SendAsync(request).Result; + + if (response.IsSuccessStatusCode) + { + WorkItemBatchPostResponse batchResponse = response.Content.ReadAsAsync().Result; + string stringResponse = JsonConvert.SerializeObject(batchResponse, Formatting.Indented); + _context.Logger.WriteVerbose(stringResponse); + bool succeeded = true; + foreach (var batchElement in batchResponse.values) + { + if (batchElement.code != 200) + { + _context.Logger.WriteError($"Save failed: {batchElement.body}"); + succeeded = false; + } + } + if (!succeeded) + throw new ApplicationException($"Save failed."); + } + else + { + string stringResponse = await response.Content.ReadAsStringAsync(); + _context.Logger.WriteError($"Save failed: {stringResponse}"); + throw new ApplicationException($"Save failed: {response.ReasonPhrase}."); + } + }//using + } + else + { + _context.Logger.WriteWarning($"Dry-run mode: no updates sent to Azure DevOps."); + }//if + + return (created, updated); + } + + private async Task<(int created, int updated)> SaveChanges_TwoPhases(bool commit) + { + // see https://github.com/redarrowlabs/vsts-restapi-samplecode/blob/master/VSTSRestApiSamples/WorkItemTracking/Batch.cs + // and https://docs.microsoft.com/en-us/rest/api/vsts/wit/workitembatchupdate?view=vsts-rest-4.1 + // The workitembatchupdate API has a huge limit: + // it fails adding a relation between a new (id<0) work item and an existing one (id>0) + + const string ApiVersion = "api-version=4.1"; + string baseUriString = _context.Client.BaseAddress.AbsoluteUri; + Dictionary headers = new Dictionary() { + { "Content-Type", "application/json-patch+json" } + }; + string credentials = Convert.ToBase64String(ASCIIEncoding.ASCII.GetBytes($":{_context.PersonalAccessToken}")); + var converters = new JsonConverter[] { new JsonPatchOperationConverter() }; + + int created = _context.Tracker.NewWorkItems.Count(); + int updated = _context.Tracker.ChangedWorkItems.Count(); + + BatchRequest[] newWorkItemsBatchRequests = new BatchRequest[created]; + int index = 0; + foreach (var item in _context.Tracker.NewWorkItems) + { + _context.Logger.WriteInfo($"Found a request for a new {item.WorkItemType} workitem in {_context.ProjectName}"); + + newWorkItemsBatchRequests[index++] = new BatchRequest + { + method = "PATCH", + uri = $"/{_context.ProjectName}/_apis/wit/workitems/${item.WorkItemType}?{ApiVersion}", + headers = headers, + body = item.Changes + .Where(c => c.Operation != Microsoft.VisualStudio.Services.WebApi.Patch.Operation.Test) + // remove relations as we might incour in API failure + .Where(c => c.Path != "/relations/-") + .ToArray() + }; + } + string requestBody = JsonConvert.SerializeObject(newWorkItemsBatchRequests, Formatting.Indented, converters); + _context.Logger.WriteVerbose($"New workitem(s) batch request:"); + _context.Logger.WriteVerbose(requestBody); + + if (commit) + { + using (var client = new HttpClient()) + { + client.DefaultRequestHeaders.Accept.Clear(); + client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json")); + client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", credentials); + + var batchRequest = new StringContent(requestBody, Encoding.UTF8, "application/json"); + var method = new HttpMethod("POST"); + + // send the request + var request = new HttpRequestMessage(method, $"{baseUriString}/_apis/wit/$batch?{ApiVersion}") { Content = batchRequest }; + var response = client.SendAsync(request).Result; + + if (response.IsSuccessStatusCode) + { + WorkItemBatchPostResponse batchResponse = response.Content.ReadAsAsync().Result; + string stringResponse = JsonConvert.SerializeObject(batchResponse, Formatting.Indented); + _context.Logger.WriteVerbose($"New workitem(s) batch response:"); + _context.Logger.WriteVerbose(stringResponse); + bool succeeded = true; + foreach (var batchElement in batchResponse.values) + { + if (batchElement.code != 200) + { + _context.Logger.WriteError($"Save failed: {batchElement.body}"); + succeeded = false; + } + } + if (!succeeded) + { + throw new ApplicationException($"Save failed."); + } + else + { + _context.Logger.WriteVerbose($"Updating work item ids..."); + // Fix back + var realIds = new Dictionary(); + index = 0; + foreach (var item in _context.Tracker.NewWorkItems) + { + int oldId = item.Id.Value; + // the response order matches the request order + string createdWorkitemJson = batchResponse.values[index++].body; + dynamic createdWorkitemResult = JsonConvert.DeserializeObject(createdWorkitemJson); + int newId = createdWorkitemResult.id; + item.ReplaceIdAndResetChanges(item.Id.Value, newId); + realIds.Add(oldId, newId); + } + foreach (var item in _context.Tracker.ChangedWorkItems) + { + item.RemapIdReferences(realIds); + } + } + } + else + { + string stringResponse = await response.Content.ReadAsStringAsync(); + _context.Logger.WriteError($"Save failed: {stringResponse}"); + throw new ApplicationException($"Save failed: {response.ReasonPhrase}."); + } + }//using + } + else + { + _context.Logger.WriteWarning($"Dry-run mode: no updates sent to Azure DevOps."); + }//if + + var batchRequests = new List(); + var allWorkItems = _context.Tracker.NewWorkItems.Concat(_context.Tracker.ChangedWorkItems); + foreach (var item in allWorkItems) + { + var changes = item.Changes + .Where(c => c.Operation != Microsoft.VisualStudio.Services.WebApi.Patch.Operation.Test); + if (changes.Any()) + { + _context.Logger.WriteInfo($"Found a request to update workitem {item.Id.Value} in {_context.ProjectName}"); + + batchRequests.Add(new BatchRequest + { + method = "PATCH", + uri = $"/_apis/wit/workitems/{item.Id.Value}?{ApiVersion}", + headers = headers, + body = changes.ToArray() + }); + } + } + + requestBody = JsonConvert.SerializeObject(batchRequests.ToArray(), Formatting.Indented, converters); + _context.Logger.WriteVerbose($"Update workitem(s) batch request:"); + _context.Logger.WriteVerbose(requestBody); + + if (commit) + { + using (var client = new HttpClient()) + { + client.DefaultRequestHeaders.Accept.Clear(); + client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json")); + client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", credentials); + + var batchRequest = new StringContent(requestBody, Encoding.UTF8, "application/json"); + var method = new HttpMethod("POST"); + + // send the request + var request = new HttpRequestMessage(method, $"{baseUriString}/_apis/wit/$batch?{ApiVersion}") { Content = batchRequest }; + var response = client.SendAsync(request).Result; + + if (response.IsSuccessStatusCode) + { + WorkItemBatchPostResponse batchResponse = response.Content.ReadAsAsync().Result; + string stringResponse = JsonConvert.SerializeObject(batchResponse, Formatting.Indented); + _context.Logger.WriteVerbose(stringResponse); + bool succeeded = true; + foreach (var batchElement in batchResponse.values) + { + if (batchElement.code != 200) + { + _context.Logger.WriteError($"Save failed: {batchElement.body}"); + succeeded = false; + } + } + if (!succeeded) + throw new ApplicationException($"Save failed."); + } + else + { + string stringResponse = await response.Content.ReadAsStringAsync(); + _context.Logger.WriteError($"Save failed: {stringResponse}"); + throw new ApplicationException($"Save failed: {response.ReasonPhrase}."); + } + }//using + } + else + { + _context.Logger.WriteWarning($"Dry-run mode: no updates sent to Azure DevOps."); + }//if + + return (created, updated); + } + } + + + class JsonPatchOperationConverter : JsonConverter + { + public override bool CanRead => false; + + public override JsonPatchOperation ReadJson(JsonReader reader, Type objectType, JsonPatchOperation existingValue, bool hasExistingValue, JsonSerializer serializer) + { + throw new NotImplementedException("Unnecessary because CanRead is false. The type will skip the converter."); + } + + public override void WriteJson(JsonWriter writer, JsonPatchOperation value, JsonSerializer serializer) + { + JToken t = JToken.FromObject(value); + + if (t.Type != JTokenType.Object) + { + t.WriteTo(writer); + } + else + { + writer.WriteStartObject(); + writer.WritePropertyName("op"); + writer.WriteValue(value.Operation.ToString().ToLower()); + writer.WritePropertyName("path"); + writer.WriteValue(value.Path); + if (!string.IsNullOrEmpty(value.From)) + { + writer.WritePropertyName("from"); + writer.WriteValue(value.From); + } + writer.WritePropertyName("value"); + t = JToken.FromObject(value.Value); + t.WriteTo(writer); + writer.WriteEndObject(); + } + } + } +} \ No newline at end of file diff --git a/src/aggregator-function/Engine/WorkItemWrapper.cs b/src/aggregator-ruleng/WorkItemWrapper.cs similarity index 74% rename from src/aggregator-function/Engine/WorkItemWrapper.cs rename to src/aggregator-ruleng/WorkItemWrapper.cs index da93a0b4..1f4100a6 100644 --- a/src/aggregator-function/Engine/WorkItemWrapper.cs +++ b/src/aggregator-ruleng/WorkItemWrapper.cs @@ -1,5 +1,6 @@ using Microsoft.TeamFoundation.WorkItemTracking.WebApi; using Microsoft.TeamFoundation.WorkItemTracking.WebApi.Models; +using Microsoft.VisualStudio.Services.WebApi; using Microsoft.VisualStudio.Services.WebApi.Patch; using Microsoft.VisualStudio.Services.WebApi.Patch.Json; using System; @@ -12,6 +13,8 @@ public class WorkItemWrapper { private EngineContext _context; private WorkItem _item; + private WorkItemRelationWrapperCollection _relationCollection; + private readonly JsonPatchDocument _changes = new JsonPatchDocument(); private readonly bool _isReadOnly = false; @@ -19,6 +22,7 @@ internal WorkItemWrapper(EngineContext context, WorkItem item) { _context = context; _item = item; + _relationCollection = new WorkItemRelationWrapperCollection(this, _item.Relations); if (item.Id.HasValue) { @@ -33,12 +37,12 @@ internal WorkItemWrapper(EngineContext context, WorkItem item) } else { - Id = new TemporaryWorkItemId(); + Id = new TemporaryWorkItemId(_context.Tracker); Changes.Add(new JsonPatchOperation() { - Operation = Operation.Test, + Operation = Operation.Add, Path = "/id", - Value = Id + Value = Id.Value }); _context.Tracker.TrackNew(this); } @@ -48,18 +52,19 @@ public WorkItemWrapper(EngineContext context, string project, string type) { _context = context; - Id = new TemporaryWorkItemId(); + Id = new TemporaryWorkItemId(_context.Tracker); _item = new WorkItem(); _item.Fields[CoreFieldRefNames.TeamProject] = project; _item.Fields[CoreFieldRefNames.WorkItemType] = type; _item.Fields[CoreFieldRefNames.Id] = Id.Value; + _relationCollection = new WorkItemRelationWrapperCollection(this, _item.Relations); Changes.Add(new JsonPatchOperation() { - Operation = Operation.Test, + Operation = Operation.Add, Path = "/id", - Value = Id + Value = Id.Value }); _context.Tracker.TrackNew(this); } @@ -68,18 +73,19 @@ public WorkItemWrapper(EngineContext context, WorkItemWrapper template, string t { _context = context; - Id = new TemporaryWorkItemId(); + Id = new TemporaryWorkItemId(_context.Tracker); _item = new WorkItem(); _item.Fields[CoreFieldRefNames.TeamProject] = template.TeamProject; _item.Fields[CoreFieldRefNames.WorkItemType] = type; _item.Fields[CoreFieldRefNames.Id] = Id.Value; + _relationCollection = new WorkItemRelationWrapperCollection(this, _item.Relations); Changes.Add(new JsonPatchOperation() { - Operation = Operation.Test, + Operation = Operation.Add, Path = "/id", - Value = Id + Value = Id.Value }); _context.Tracker.TrackNew(this); } @@ -99,6 +105,7 @@ internal WorkItemWrapper(EngineContext context, WorkItem item, bool isReadOnly) }); _isReadOnly = isReadOnly; _item = item; + _relationCollection = new WorkItemRelationWrapperCollection(this, _item.Relations); _context.Tracker.TrackRevision(this); } @@ -132,11 +139,20 @@ public IEnumerable Revisions } } - public IEnumerable Relations + public IEnumerable RelationLinks + { + get + { + return _relationCollection; + } + } + + + public WorkItemRelationWrapperCollection Relations { get { - return new WorkItemRelationWrapperCollection(this, _item.Relations); + return _relationCollection; } } @@ -144,7 +160,7 @@ public IEnumerable ChildrenLinks { get { - return new WorkItemRelationWrapperCollection(this, _item.Relations) + return _relationCollection .Where(rel => rel.Rel == CoreRelationRefNames.Children); } } @@ -162,7 +178,7 @@ public IEnumerable RelatedLinks { get { - return new WorkItemRelationWrapperCollection(this, _item.Relations) + return _relationCollection .Where(rel => rel.Rel == CoreRelationRefNames.Related); } } @@ -171,7 +187,7 @@ public IEnumerable Hyperlinks { get { - return new WorkItemRelationWrapperCollection(this, _item.Relations) + return _relationCollection .Where(rel => rel.Rel == CoreRelationRefNames.Hyperlink); } } @@ -180,7 +196,7 @@ public WorkItemRelationWrapper ParentLink { get { - return new WorkItemRelationWrapperCollection(this, _item.Relations) + return _relationCollection .Where(rel => rel.Rel == CoreRelationRefNames.Parent) .SingleOrDefault(); } @@ -198,6 +214,7 @@ public WorkItemWrapper Parent public WorkItemId Id { get; + private set; } public int Rev @@ -234,9 +251,9 @@ public string AreaPath set { SetFieldValue(CoreFieldRefNames.AreaPath, value); } } - public string AssignedTo + public IdentityRef AssignedTo { - get { return GetFieldValue(CoreFieldRefNames.AssignedTo); } + get { return GetFieldValue(CoreFieldRefNames.AssignedTo); } set { SetFieldValue(CoreFieldRefNames.AssignedTo, value); } } @@ -246,15 +263,15 @@ public int AttachedFileCount set { SetFieldValue(CoreFieldRefNames.AttachedFileCount, value); } } - public string AuthorizedAs + public IdentityRef AuthorizedAs { - get { return GetFieldValue(CoreFieldRefNames.AuthorizedAs); } + get { return GetFieldValue(CoreFieldRefNames.AuthorizedAs); } set { SetFieldValue(CoreFieldRefNames.AuthorizedAs, value); } } - public string ChangedBy + public IdentityRef ChangedBy { - get { return GetFieldValue(CoreFieldRefNames.ChangedBy); } + get { return GetFieldValue(CoreFieldRefNames.ChangedBy); } set { SetFieldValue(CoreFieldRefNames.ChangedBy, value); } } @@ -264,9 +281,9 @@ public DateTime? ChangedDate set { SetFieldValue(CoreFieldRefNames.ChangedDate, value); } } - public string CreatedBy + public IdentityRef CreatedBy { - get { return GetFieldValue(CoreFieldRefNames.CreatedBy); } + get { return GetFieldValue(CoreFieldRefNames.CreatedBy); } set { SetFieldValue(CoreFieldRefNames.CreatedBy, value); } } @@ -393,13 +410,26 @@ private void SetFieldValue(string field, object value) throw new InvalidOperationException("Work item is read-only."); } - _item.Fields[field] = value; - Changes.Add(new JsonPatchOperation() + if (_item.Fields.ContainsKey(field)) { - Operation = Operation.Add, - Path = "/fields/" + field, - Value = value - }); + _item.Fields[field] = value; + Changes.Add(new JsonPatchOperation() + { + Operation = Operation.Replace, + Path = "/fields/" + field, + Value = value + }); + } + else + { + _item.Fields.Add(field, value); + Changes.Add(new JsonPatchOperation() + { + Operation = Operation.Add, + Path = "/fields/" + field, + Value = value + }); + } IsDirty = true; } @@ -410,5 +440,46 @@ private T GetFieldValue(string field) ? (T)_item.Fields[field] : default(T); } + + internal void ReplaceIdAndResetChanges(int oldId, int newId) + { + if (oldId >= 0) throw new ArgumentOutOfRangeException(nameof(oldId)); + + Id = new PermanentWorkItemId(newId); + + var candidates = Changes.Where(op => op.Path == "/relations/-"); + foreach (var op in candidates) + { + var patch = op.Value as RelationPatch; + string url = patch.url; + int pos = url.LastIndexOf('/') + 1; + int relId = int.Parse(url.Substring(pos)); + if (relId == oldId) + { + patch.url = url.Substring(0, pos) + newId.ToString(); + break; + } + } + + Changes.RemoveAll(op => op.Path.StartsWith("/fields/") || op.Path == "/id"); + } + + internal void RemapIdReferences(IDictionary realIds) + { + var candidates = Changes.Where(op => op.Path == "/relations/-"); + foreach (var op in candidates) + { + var patch = op.Value as RelationPatch; + string url = patch.url; + int pos = url.LastIndexOf('/') + 1; + int relId = int.Parse(url.Substring(pos)); + if (realIds.ContainsKey(relId)) + { + int newId = realIds[relId]; + string newUrl = url.Substring(0, pos) + newId.ToString(); + patch.url = newUrl; + } + } + } } } diff --git a/src/aggregator-ruleng/aggregator-ruleng.csproj b/src/aggregator-ruleng/aggregator-ruleng.csproj new file mode 100644 index 00000000..4ce61346 --- /dev/null +++ b/src/aggregator-ruleng/aggregator-ruleng.csproj @@ -0,0 +1,27 @@ + + + + netstandard2.0 + aggregator.Engine + false + + + + DEBUG;TRACE + + + + + + + + + + + + + + + + + diff --git a/src/aggregator-shared/AggregatorConfiguration.cs b/src/aggregator-shared/AggregatorConfiguration.cs new file mode 100644 index 00000000..abef957e --- /dev/null +++ b/src/aggregator-shared/AggregatorConfiguration.cs @@ -0,0 +1,52 @@ +using System; + +namespace aggregator +{ + public enum DevOpsTokenType + { + Integrated = 0, + PAT = 1, + } + + public enum SaveMode + { + Default = 0, + Item = 1, + Batch = 2, + TwoPhases = 3 + } + + /// + /// This class tracks the configuration data that CLI writes and Function runtime reads + /// + public class AggregatorConfiguration + { + public AggregatorConfiguration() {} + + static public AggregatorConfiguration Read(Microsoft.Extensions.Configuration.IConfiguration config) + { + var ac = new AggregatorConfiguration(); + Enum.TryParse(config["Aggregator_VstsTokenType"], out DevOpsTokenType vtt); + ac.DevOpsTokenType = vtt; + ac.DevOpsToken = config["Aggregator_VstsToken"]; + ac.SaveMode = Enum.TryParse(config["Aggregator_SaveMode"], out SaveMode sm) + ? sm + : SaveMode.Item; + return ac; + } + + public void Write(Microsoft.Azure.Management.AppService.Fluent.IWebApp webApp) + { + webApp + .Update() + .WithAppSetting("Aggregator_VstsTokenType", DevOpsTokenType.ToString()) + .WithAppSetting("Aggregator_VstsToken", DevOpsToken) + .WithAppSetting("Aggregator_SaveMode", SaveMode.ToString()) + .Apply(); + } + + public DevOpsTokenType DevOpsTokenType { get; set; } + public string DevOpsToken { get; set; } + public SaveMode SaveMode { get; set; } + } +} diff --git a/src/aggregator-core/AssemblyInfo.cs b/src/aggregator-shared/AssemblyInfo.cs similarity index 62% rename from src/aggregator-core/AssemblyInfo.cs rename to src/aggregator-shared/AssemblyInfo.cs index f01e13bd..ec63e80c 100644 --- a/src/aggregator-core/AssemblyInfo.cs +++ b/src/aggregator-shared/AssemblyInfo.cs @@ -8,8 +8,8 @@ [assembly: AssemblyConfiguration("Release")] #endif [assembly: AssemblyCopyright("TFS Aggregator Team")] -[assembly: AssemblyFileVersion("0.3.4.0")] -[assembly: AssemblyInformationalVersion("0.3.4")] +[assembly: AssemblyFileVersion("0.4.0.0")] +[assembly: AssemblyInformationalVersion("0.4.0")] [assembly: AssemblyProduct("Aggregator CLI")] -[assembly: AssemblyTitle("Aggregator Core")] -[assembly: AssemblyVersion("0.3.4.0")] +[assembly: AssemblyTitle("Aggregator Shared")] +[assembly: AssemblyVersion("0.4.0.0")] diff --git a/src/aggregator-core/VstsEvents.cs b/src/aggregator-shared/VstsEvents.cs similarity index 79% rename from src/aggregator-core/VstsEvents.cs rename to src/aggregator-shared/VstsEvents.cs index ac0c1bcd..761e7595 100644 --- a/src/aggregator-core/VstsEvents.cs +++ b/src/aggregator-shared/VstsEvents.cs @@ -5,7 +5,10 @@ namespace aggregator { - public class VstsEvents + /// + /// This class tracks the VSTS/AzureDevOps Events exposed both in CLI and Rules + /// + public class DevOpsEvents { // TODO this table should be visible in the help static string[] validValues = new string[] { diff --git a/src/aggregator-core/aggregator-core.csproj b/src/aggregator-shared/aggregator-shared.csproj similarity index 81% rename from src/aggregator-core/aggregator-core.csproj rename to src/aggregator-shared/aggregator-shared.csproj index 2a779012..4eff7b4a 100644 --- a/src/aggregator-core/aggregator-core.csproj +++ b/src/aggregator-shared/aggregator-shared.csproj @@ -2,8 +2,9 @@ netstandard2.0 - aggregator.core + aggregator false + aggregator-shared @@ -11,7 +12,7 @@ - + diff --git a/src/integrationtests-cli/End2EndScenarioBase.cs b/src/integrationtests-cli/End2EndScenarioBase.cs new file mode 100644 index 00000000..dda58387 --- /dev/null +++ b/src/integrationtests-cli/End2EndScenarioBase.cs @@ -0,0 +1,37 @@ +using System; +using System.IO; +using Xunit.Abstractions; + +namespace integrationtests.cli +{ + public abstract class End2EndScenarioBase + { + private readonly ITestOutputHelper xunitOutput; + + public End2EndScenarioBase(ITestOutputHelper output) + { + this.xunitOutput = output; + } + + protected (int rc, string output) RunAggregatorCommand(string commandLine) + { + var args = commandLine.Split(' '); + + var save_out = Console.Out; + var save_err = Console.Error; + var buffered = new StringWriter(); + Console.SetOut(buffered); + Console.SetError(buffered); + + int rc = aggregator.cli.Program.Main(args); + + Console.SetOut(save_out); + Console.SetError(save_err); + + string output = buffered.ToString(); + xunitOutput.WriteLine(output); + + return (rc, output); + } + } +} \ No newline at end of file diff --git a/src/integrationtests-cli/Scenario3_MultiInstance.cs b/src/integrationtests-cli/Scenario3_MultiInstance.cs new file mode 100644 index 00000000..9e82fa7c --- /dev/null +++ b/src/integrationtests-cli/Scenario3_MultiInstance.cs @@ -0,0 +1,139 @@ +using System; +using System.Collections.Generic; +using System.IO; +using System.Text; +using Xunit; +using Xunit.Abstractions; +using XUnitPriorityOrderer; + +[assembly: CollectionBehavior(DisableTestParallelization = true)] +// addset the custom test's collection orderer +[assembly: TestCollectionOrderer(CollectionPriorityOrderer.TypeName, CollectionPriorityOrderer.AssembyName)] + +namespace integrationtests.cli +{ + [TestCaseOrderer(CasePriorityOrderer.TypeName, CasePriorityOrderer.AssembyName)] + public class Scenario3_MultiInstance : End2EndScenarioBase + { + public Scenario3_MultiInstance(ITestOutputHelper output) + : base(output) + { + } + + const string location = "westeurope"; + const string resourceGroup = "test-aggregator45"; + const string project = "WorkItemTracking"; + + [Fact, Order(1)] + void Logon() + { + dynamic data = Newtonsoft.Json.JsonConvert.DeserializeObject(File.ReadAllText("logon-data.json")); + + (int rc, string output) = RunAggregatorCommand( + $"logon.azure --subscription {data.subscription} --client {data.client} --password {data.password} --tenant {data.tenant}"); + Assert.Equal(0, rc); + Assert.DoesNotContain("] Failed!", output); + (int rc2, string output2) = RunAggregatorCommand( + $"logon.ado --url {data.devopsUrl} --mode PAT --token {data.pat}"); + Assert.Equal(0, rc2); + Assert.DoesNotContain("] Failed!", output2); + } + + [Theory, Order(2)] + [InlineData("my4")] + [InlineData("my5")] + void InstallInstances(string instance) + { + (int rc, string output) = RunAggregatorCommand($"install.instance --name {instance} --resourceGroup {resourceGroup} --location {location}"); + + Assert.Equal(0, rc); + Assert.DoesNotContain("] Failed!", output); + } + + [Fact, Order(3)] + void ListInstances() + { + (int rc, string output) = RunAggregatorCommand($"list.instances --resourceGroup {resourceGroup}"); + + Assert.Equal(0, rc); + Assert.Contains("Instance my4", output); + Assert.Contains("Instance my5", output); + } + + [Theory, Order(4)] + [InlineData("my4", "test4")] + [InlineData("my5", "test5")] + void AddRules(string instance, string rule) + { + (int rc, string output) = RunAggregatorCommand($"add.rule --instance {instance} --resourceGroup {resourceGroup} --name {rule} --file {rule}.rule"); + + Assert.Equal(0, rc); + Assert.DoesNotContain("] Failed!", output); + } + + [Theory, Order(5)] + [InlineData("my4", "test4")] + [InlineData("my5", "test5")] + void ListRules(string instance, string rule) + { + (int rc, string output) = RunAggregatorCommand($"list.rules --instance {instance} --resourceGroup {resourceGroup}"); + + Assert.Equal(0, rc); + Assert.Contains($"Rule {instance}/{rule}", output); + Assert.DoesNotContain("] Failed!", output); + } + + [Theory, Order(6)] + [InlineData("my4", "test4")] + [InlineData("my5", "test5")] + void MapRules(string instance, string rule) + { + (int rc, string output) = RunAggregatorCommand($"map.rule --project {project} --event workitem.created --instance {instance} --resourceGroup {resourceGroup} --rule {rule}"); + + Assert.Equal(0, rc); + Assert.DoesNotContain("] Failed!", output); + } + + [Theory, Order(7)] + [InlineData("my4", "test4")] + [InlineData("my5", "test5")] + void ListMappings(string instance, string rule) + { + (int rc, string output) = RunAggregatorCommand($"list.mappings --instance {instance} --resourceGroup {resourceGroup}"); + + Assert.Equal(0, rc); + Assert.Contains($"invokes rule {instance}/{rule}", output); + Assert.DoesNotContain("] Failed!", output); + } + + [Theory, Order(8)] + [InlineData("my4")] + void UninstallInstances(string instance) + { + (int rc, string output) = RunAggregatorCommand($"uninstall.instance --name {instance} --resourceGroup {resourceGroup} --location {location}"); + + Assert.Equal(0, rc); + Assert.DoesNotContain("] Failed!", output); + } + + [Fact, Order(9)] + void ListInstancesAfterUninstall() + { + (int rc, string output) = RunAggregatorCommand($"list.instances --resourceGroup {resourceGroup}"); + + Assert.Equal(0, rc); + Assert.DoesNotContain("Instance my4", output); + Assert.Contains("Instance my5", output); + } + + [Theory, Order(10)] + [InlineData("my5", "test5")] + void UnmapRules(string instance, string rule) + { + (int rc, string output) = RunAggregatorCommand($"unmap.rule --project {project} --event workitem.created --instance {instance} --resourceGroup {resourceGroup} --rule {rule}"); + + Assert.Equal(0, rc); + Assert.DoesNotContain("] Failed!", output); + } + } +} diff --git a/src/integrationtests-cli/integrationtests-cli.csproj b/src/integrationtests-cli/integrationtests-cli.csproj new file mode 100644 index 00000000..15c01bc9 --- /dev/null +++ b/src/integrationtests-cli/integrationtests-cli.csproj @@ -0,0 +1,37 @@ + + + + netcoreapp2.1 + integrationtests.cli + + false + + + + + + + all + runtime; build; native; contentfiles; analyzers + + + + + + + + + + + + PreserveNewest + + + PreserveNewest + + + PreserveNewest + + + + diff --git a/src/integrationtests-cli/logon-data.json b/src/integrationtests-cli/logon-data.json new file mode 100644 index 00000000..4df3f829 --- /dev/null +++ b/src/integrationtests-cli/logon-data.json @@ -0,0 +1,8 @@ +{ + "subscription": "guid", + "client": "guid", + "password": "password", + "tenant": "guid", + "devopsUrl": "https://dev.azure.com/organization", + "pat": "PAT" +} diff --git a/src/integrationtests-cli/test4.rule b/src/integrationtests-cli/test4.rule new file mode 100644 index 00000000..2a1c2c37 --- /dev/null +++ b/src/integrationtests-cli/test4.rule @@ -0,0 +1 @@ +$"Hello { self.WorkItemType } #{ self.Id } from Rule 4!" \ No newline at end of file diff --git a/src/integrationtests-cli/test5.rule b/src/integrationtests-cli/test5.rule new file mode 100644 index 00000000..8c91087b --- /dev/null +++ b/src/integrationtests-cli/test5.rule @@ -0,0 +1 @@ +$"Hello { self.WorkItemType } #{ self.Id } from Rule 5!" \ No newline at end of file diff --git a/src/unittests-core/unittests-core.csproj b/src/unittests-core/unittests-core.csproj index 67dce70b..5f8bd182 100644 --- a/src/unittests-core/unittests-core.csproj +++ b/src/unittests-core/unittests-core.csproj @@ -15,7 +15,7 @@ - + diff --git a/src/unittests-ruleng/FakeWorkItemTrackingHttpClientTests.cs b/src/unittests-ruleng/FakeWorkItemTrackingHttpClientTests.cs new file mode 100644 index 00000000..9f55eeb3 --- /dev/null +++ b/src/unittests-ruleng/FakeWorkItemTrackingHttpClientTests.cs @@ -0,0 +1,23 @@ +using System; +using aggregator.unittests; +using Microsoft.TeamFoundation.WorkItemTracking.WebApi.Models; +using Xunit; + +namespace unittests_ruleng +{ + public class FakeWorkItemTrackingHttpClientTests + { + [Fact] + public void GetWorkItem_ById_Succeeds() + { + var baseUrl = new Uri("https://dev.azure.com/fake-account/fake-project"); + using (var sut = new FakeWorkItemTrackingHttpClient(baseUrl, null)) + { + var wi = sut.GetWorkItemAsync(42, expand: WorkItemExpand.All).Result; + + Assert.NotNull(wi); + Assert.Equal(42, wi.Id); + } + } + } +} diff --git a/src/unittests-ruleng/FakesNStubs/FakeWorkItemTrackingHttpClient.cs b/src/unittests-ruleng/FakesNStubs/FakeWorkItemTrackingHttpClient.cs new file mode 100644 index 00000000..136706e3 --- /dev/null +++ b/src/unittests-ruleng/FakesNStubs/FakeWorkItemTrackingHttpClient.cs @@ -0,0 +1,238 @@ +using Microsoft.TeamFoundation.WorkItemTracking.WebApi; +using Microsoft.TeamFoundation.WorkItemTracking.WebApi.Models; +using Microsoft.VisualStudio.Services.Common; +using Microsoft.VisualStudio.Services.WebApi.Patch.Json; +using System; +using System.Collections.Generic; +using System.Diagnostics; +using System.Linq; +using System.Net.Http; +using System.Text; +using System.Threading; +using System.Threading.Tasks; + +namespace aggregator.unittests +{ + internal class FakeWorkItemTrackingHttpClient : WorkItemTrackingHttpClientBase + { + Dictionary> workItemFactory = new Dictionary>(); + + public FakeWorkItemTrackingHttpClient(Uri baseUrl, VssCredentials credentials) + : base(baseUrl, credentials) + { + string workItemsBaseUrl = $"{baseUrl.AbsoluteUri}/example-project/_apis/wit/workItems"; + workItemFactory.Add(1, () => new WorkItem() + { + Id = 1, + Fields = new Dictionary() + { + { "System.WorkItemType", "User Story" }, + { "System.State", "Open" }, + { "System.TeamProject", "example-project" }, + { "System.Title", "Hello" }, + }, + Rev = 12, + Relations = new List() + { + new WorkItemRelation + { + Rel = "System.LinkTypes.Hierarchy-Forward", + Url = $"{workItemsBaseUrl}/42" + }, + new WorkItemRelation + { + Rel = "System.LinkTypes.Hierarchy-Forward", + Url = $"{workItemsBaseUrl}/99" + } + }, + Url = $"{workItemsBaseUrl}/1" + }); + + workItemFactory.Add(42, () => new WorkItem() + { + Id = 42, + Fields = new Dictionary() + { + { "System.WorkItemType", "Bug" }, + { "System.State", "Open" }, + { "System.TeamProject", "example-project" }, + { "System.Title", "Hello" }, + }, + Rev = 3, + Relations = new List() + { + new WorkItemRelation + { + Rel = "System.LinkTypes.Hierarchy-Reverse", + Url = $"{workItemsBaseUrl}/1" + } + }, + Url = $"{workItemsBaseUrl}/42" + }); + + workItemFactory.Add(99, () => new WorkItem() + { + Id = 99, + Fields = new Dictionary() + { + { "System.WorkItemType", "Bug" }, + { "System.State", "Open" }, + { "System.TeamProject", "example-project" }, + { "System.Title", "Hello" }, + }, + Rev = 3, + Relations = new List() + { + new WorkItemRelation + { + Rel = "System.LinkTypes.Hierarchy-Reverse", + Url = $"{workItemsBaseUrl}/1" + } + }, + Url = $"{workItemsBaseUrl}/99" + }); + } + + public override Task GetWorkItemAsync(int id, IEnumerable fields = null, DateTime? asOf = null, WorkItemExpand? expand = null, object userState = null, CancellationToken cancellationToken = default(CancellationToken)) + { + Debug.WriteLine($"FakeWorkItemTrackingHttpClient.GetWorkItemAsync({id})"); + if (expand == null) + { + throw new ArgumentNullException(nameof(expand)); + } + if (expand != WorkItemExpand.All) + { + throw new ArgumentException("Must be WorkItemExpand.All", nameof(expand)); + } + + if (workItemFactory.ContainsKey(id)) + { + var t = new Task(workItemFactory[id]); + t.RunSynchronously(); + return t; + } else + { + return null; + } + } + + public override Task> GetWorkItemsAsync(IEnumerable ids, IEnumerable fields = null, DateTime? asOf = null, WorkItemExpand? expand = null, WorkItemErrorPolicy? errorPolicy = null, object userState = null, CancellationToken cancellationToken = default(CancellationToken)) + { + string sid = ids.Aggregate(string.Empty, (s, i) => s + "," + i.ToString()); + Debug.WriteLine($"FakeWorkItemTrackingHttpClient.GetWorkItemsAsync({sid})"); + if (expand == null) + { + throw new ArgumentNullException(nameof(expand)); + } + if (expand != WorkItemExpand.All) + { + throw new ArgumentException("Must be WorkItemExpand.All", nameof(expand)); + } + + var t = new Task>(() => { + var result = new List(); + foreach (var id in ids) + { + if (workItemFactory.ContainsKey(id)) + { + var wi = workItemFactory[id](); + result.Add(wi); + } + } + return result; + }); + t.RunSynchronously(); + return t; + } + + public override Task GetWorkItemTypeAsync(Guid project, string type, object userState = null, CancellationToken cancellationToken = default(CancellationToken)) + { + var t = new Task(() => new WorkItemType() + { + Name = "Bug", + IsDisabled = false, + FieldInstances = new List() + { + new WorkItemTypeFieldInstance() + { + ReferenceName = "System.State", + IsIdentity = false, + AlwaysRequired = true + // MISSING: Data type! + } + }, + Transitions = new Dictionary() + { + { + "New", (new WorkItemStateTransition[] { + new WorkItemStateTransition() + { + To = "Active", Actions = new string[] { "" } + } + }) + }, + { + "Active", (new WorkItemStateTransition[] { + new WorkItemStateTransition() + { + To = "Closed", Actions = new string[] { "" } + } + }) + } + } + }); + t.RunSynchronously(); + return t; + } + + public override Task QueryByWiqlAsync(Wiql wiql, Guid project, bool? timePrecision = null, int? top = null, object userState = null, CancellationToken cancellationToken = default(CancellationToken)) + { + var t = new Task(() => new WorkItemQueryResult() { + QueryType = QueryType.Flat, + WorkItems = new List() + { + new WorkItemReference() + { + Id = 33, + Url = $"{BaseAddress.AbsoluteUri}/{project}/_apis/wit/workItems/33" + } + } + }); + + t.RunSynchronously(); + return t; + } + + static int newId = 100; + public override Task CreateWorkItemAsync(JsonPatchDocument document, Guid project, string type, bool? validateOnly = null, bool? bypassRules = null, bool? suppressNotifications = null, object userState = null, CancellationToken cancellationToken = default(CancellationToken)) + { + string workItemsBaseUrl = $"{BaseAddress.AbsoluteUri}/example-project/_apis/wit/workItems"; + + workItemFactory.Add(newId, () => new WorkItem() + { + Id = newId, + Fields = new Dictionary() + { + { "System.WorkItemType", type }, + { "System.State", "New" }, + { "System.TeamProject", "example-project" }, + { "System.Title", "Hello" }, + }, + Rev = 1, + Relations = new List(), + Url = $"{workItemsBaseUrl}/{newId}" + }); + + var t = new Task(workItemFactory[newId]); + t.RunSynchronously(); + return t; + } + + public override Task UpdateWorkItemAsync(JsonPatchDocument document, int id, bool? validateOnly = null, bool? bypassRules = null, bool? suppressNotifications = null, object userState = null, CancellationToken cancellationToken = default(CancellationToken)) + { + var t = new Task(() => new WorkItem()); + t.RunSynchronously(); + return t; + } + } +} diff --git a/src/unittests-ruleng/FakesNStubs/MockAggregatorLogger.cs b/src/unittests-ruleng/FakesNStubs/MockAggregatorLogger.cs new file mode 100644 index 00000000..cca705c0 --- /dev/null +++ b/src/unittests-ruleng/FakesNStubs/MockAggregatorLogger.cs @@ -0,0 +1,40 @@ +using System; +using System.Collections.Generic; +using System.Text; +using aggregator; + +namespace aggregator.unittests +{ + class MockLogEntry + { + internal string Level { get; set; } + internal string Message { get; set; } + } + + class MockAggregatorLogger : IAggregatorLogger + { + List log = new List(10); + + public void WriteError(string message) + { + log.Add(new MockLogEntry { Level = "Error", Message = message }); + } + + public void WriteInfo(string message) + { + log.Add(new MockLogEntry { Level = "Info", Message = message }); + } + + public void WriteVerbose(string message) + { + log.Add(new MockLogEntry { Level = "Verbose", Message = message }); + } + + public void WriteWarning(string message) + { + log.Add(new MockLogEntry { Level = "Warning", Message = message }); + } + + public MockLogEntry[] GetMessages() => log.ToArray(); + } +} diff --git a/src/unittests-ruleng/RuleTests.cs b/src/unittests-ruleng/RuleTests.cs new file mode 100644 index 00000000..2d19b936 --- /dev/null +++ b/src/unittests-ruleng/RuleTests.cs @@ -0,0 +1,131 @@ +using System; +using System.Collections.Generic; +using System.Text; +using aggregator; +using aggregator.Engine; +using aggregator.unittests; +using Xunit; + +namespace unittests_ruleng +{ + static class StringExtensions + { + internal static string[] Mince(this string ruleCode) + { + return ruleCode.Split(Environment.NewLine); + } + } + + public class RuleTests + { + const string collectionUrl = "https://dev.azure.com/fake-organization"; + Guid projectId = Guid.NewGuid(); + const string projectName = "test-project"; + const string personalAccessToken = "***personalAccessToken***"; + FakeWorkItemTrackingHttpClient client = new FakeWorkItemTrackingHttpClient(new Uri($"{collectionUrl}"), null); + MockAggregatorLogger logger = new MockAggregatorLogger(); + + [Fact] + public async void HelloWorldRule_Succeeds() + { + int workItemId = 42; + string ruleCode = @" +return $""Hello { self.WorkItemType } #{ self.Id } - { self.Title }!""; +"; + + var engine = new RuleEngine(logger, ruleCode.Mince(), SaveMode.Item); + string result = await engine.ExecuteAsync(collectionUrl, projectId, projectName, personalAccessToken, workItemId, client); + + Assert.Equal("Hello Bug #42 - Hello!", result); + } + + [Fact] + public async void LanguageDirective_Succeeds() + { + int workItemId = 42; + string ruleCode = @".lang=CS +return string.Empty; +"; + + var engine = new RuleEngine(logger, ruleCode.Mince(), SaveMode.Item); + string result = await engine.ExecuteAsync(collectionUrl, projectId, projectName, personalAccessToken, workItemId, client); + + Assert.Equal(EngineState.Success, engine.State); + Assert.Equal(string.Empty, result); + } + + [Fact] + public async void LanguageDirective_Fails() + { + int workItemId = 42; + string ruleCode = @".lang=WHAT +return string.Empty; +"; + + var engine = new RuleEngine(logger, ruleCode.Mince(), SaveMode.Item); + string result = await engine.ExecuteAsync(collectionUrl, projectId, projectName, personalAccessToken, workItemId, client); + + Assert.Equal(EngineState.Error, engine.State); + } + + [Fact] + public async void Parent_Succeeds() + { + int workItemId = 42; + string ruleCode = @" +string message = """"; +var parent = self.Parent; +if (parent != null) +{ + message = $""Parent is {parent.Id}""; +} +return message; +"; + + var engine = new RuleEngine(logger, ruleCode.Mince(), SaveMode.Item); + string result = await engine.ExecuteAsync(collectionUrl, projectId, projectName, personalAccessToken, workItemId, client); + + Assert.Equal("Parent is 1", result); + } + + [Fact] + public async void New_Succeeds() + { + int workItemId = 1; + string ruleCode = @" +var wi = store.NewWorkItem(""Task""); +wi.Title = ""Brand new""; +"; + + var engine = new RuleEngine(logger, ruleCode.Mince(), SaveMode.Item); + string result = await engine.ExecuteAsync(collectionUrl, projectId, projectName, personalAccessToken, workItemId, client); + + Assert.Null(result); + Assert.Contains( + logger.GetMessages(), + m => m.Message == "Changes saved to Azure DevOps (mode Item): 1 created, 0 updated." + && m.Level == "Info"); + } + + [Fact] + public async void AddChild_Succeeds() + { + int workItemId = 1; + string ruleCode = @" +var parent = self; +var newChild = store.NewWorkItem(""Task""); +newChild.Title = ""Brand new""; +parent.Relations.AddChild(newChild); +"; + + var engine = new RuleEngine(logger, ruleCode.Mince(), SaveMode.Item); + string result = await engine.ExecuteAsync(collectionUrl, projectId, projectName, personalAccessToken, workItemId, client); + + Assert.Null(result); + Assert.Contains( + logger.GetMessages(), + m => m.Message == "Changes saved to Azure DevOps (mode Item): 1 created, 1 updated." + && m.Level == "Info"); + } + } +} diff --git a/src/unittests-ruleng/WorkItemStoreTests.cs b/src/unittests-ruleng/WorkItemStoreTests.cs new file mode 100644 index 00000000..cba26515 --- /dev/null +++ b/src/unittests-ruleng/WorkItemStoreTests.cs @@ -0,0 +1,81 @@ +using System; +using System.Linq; +using aggregator; +using aggregator.Engine; +using aggregator.unittests; +using Xunit; + +namespace unittests_ruleng +{ + public class WorkItemStoreTests + { + const string collectionUrl = "https://dev.azure.com/fake-organization"; + Guid projectId = Guid.NewGuid(); + const string projectName = "test-project"; + const string personalAccessToken = "***personalAccessToken***"; + FakeWorkItemTrackingHttpClient client = new FakeWorkItemTrackingHttpClient(new Uri($"{collectionUrl}"), null); + MockAggregatorLogger logger = new MockAggregatorLogger(); + + [Fact] + public void GetWorkItem_ById_Succeeds() + { + var context = new EngineContext(client, projectId, projectName, personalAccessToken, logger); + var sut = new WorkItemStore(context); + + var wi = sut.GetWorkItem(42); + + Assert.NotNull(wi); + Assert.Equal(42, wi.Id.Value); + } + + [Fact] + public void GetWorkItems_ByIds_Succeeds() + { + var context = new EngineContext(client, projectId, projectName, personalAccessToken, logger); + var sut = new WorkItemStore(context); + + var wis = sut.GetWorkItems(new int[] { 42, 99 }); + + Assert.NotEmpty(wis); + Assert.Equal(2, wis.Count); + Assert.Contains(wis, (x) => x.Id.Value == 42); + Assert.Contains(wis, (x) => x.Id.Value == 99); + } + + [Fact] + public void NewWorkItem_Succeeds() + { + var context = new EngineContext(client, projectId, projectName, personalAccessToken, logger); + var sut = new WorkItemStore(context); + + var wi = sut.NewWorkItem("Task"); + wi.Title = "Brand new"; + var save = sut.SaveChanges(SaveMode.Item, true).Result; + + Assert.NotNull(wi); + Assert.True(wi.IsNew); + Assert.Equal(1, save.created); + Assert.Equal(0, save.updated); + Assert.Equal(-1, wi.Id.Value); + } + + [Fact] + public void AddChild_Succeeds() + { + var context = new EngineContext(client, projectId, projectName, personalAccessToken, logger); + var sut = new WorkItemStore(context); + + var parent = sut.GetWorkItem(1); + Assert.Equal(2, parent.Relations.Count()); + + var newChild = sut.NewWorkItem("Task"); + newChild.Title = "Brand new"; + parent.Relations.AddChild(newChild); + + Assert.NotNull(newChild); + Assert.True(newChild.IsNew); + Assert.Equal(-1, newChild.Id.Value); + Assert.Equal(3, parent.Relations.Count()); + } + } +} diff --git a/src/unittests-cli/unittests-cli.csproj b/src/unittests-ruleng/unittests-ruleng.csproj similarity index 55% rename from src/unittests-cli/unittests-cli.csproj rename to src/unittests-ruleng/unittests-ruleng.csproj index 86afc14e..9b12ed8f 100644 --- a/src/unittests-cli/unittests-cli.csproj +++ b/src/unittests-ruleng/unittests-ruleng.csproj @@ -2,20 +2,23 @@ netcoreapp2.1 - unittests_cli + unittests_ruleng false - - + + + all + runtime; build; native; contentfiles; analyzers + - +