Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
4238599
BREAKING CHANGE: renamed aggregator-core project to aggregator-shared
giuliov Oct 27, 2018
8a742ca
bump version number to underline breaking change
giuliov Oct 27, 2018
f0372f4
Moved Rule code independent from Azure Function in its own project to…
giuliov Oct 27, 2018
4675a45
First draft of unit tests for the Rule Engine
giuliov Oct 27, 2018
f58f58e
Changed string from VSTS to Azure DevOps
giuliov Oct 27, 2018
d6c95c9
Renamed Vsts to DevOps in code
giuliov Oct 27, 2018
e2cf538
Refactored so most of the Execute code is now in the RuleEngine class
giuliov Oct 27, 2018
7decc2d
work for NewWorkItem
giuliov Oct 29, 2018
3fa068d
Fix issue about mapping multiple project to the same rule
giuliov Oct 30, 2018
314a417
Added support for --resourceGroup option
giuliov Oct 30, 2018
72fdea9
Added Project option to unmap.rule verb to fix #10
giuliov Oct 31, 2018
2a81d4d
list.mappings extended to filter by project
giuliov Oct 31, 2018
973dacb
moved rule update to its own command and refreshed the list of sample…
giuliov Oct 31, 2018
7e9250c
Check on uploaded runtime version and better messages
giuliov Oct 31, 2018
3a79fdc
update uninstall.instance to only remove the resource group if no oth…
Nov 1, 2018
e4e9902
Merge pull request #13 from pswetz/rel-0.4
giuliov Nov 1, 2018
5853489
Do not delete custom resource groups, also check on creation
giuliov Nov 1, 2018
763e795
fixes errors in examples
giuliov Nov 3, 2018
dfb7031
more commmand examples
giuliov Nov 3, 2018
95f58c6
fixed broken, incomplete test
giuliov Nov 3, 2018
b870bdc
relations starts to make sense
giuliov Nov 4, 2018
88182cc
Fixes If-Match issue on update.rule
giuliov Nov 6, 2018
60e5437
Fixes Verb 'configure.instance' is not recognized
giuliov Nov 6, 2018
1fd36ce
better logging
giuliov Nov 8, 2018
ecc11cf
remove spurious test function
giuliov Nov 8, 2018
7b99e36
checks if GutHub release exists
giuliov Nov 8, 2018
1ddff24
IdentityRef raw
giuliov Nov 8, 2018
6c657b1
doc updated
giuliov Nov 8, 2018
654b34e
invoke.rule command
giuliov Nov 10, 2018
e183885
e2e integration tests for CLI
giuliov Nov 10, 2018
1240027
fix bug on --requiredVersion latest
giuliov Nov 10, 2018
5635a20
doc edit
giuliov Nov 11, 2018
7ceb207
Rewrote SaveChanges to use batch API: to be tested
giuliov Nov 11, 2018
d0e3293
Support for Batch save mode
giuliov Nov 15, 2018
13164b4
Two pass supports the new item linked to an existing item
giuliov Nov 17, 2018
3db2c40
Updated NuGet packages
giuliov Nov 17, 2018
8b2b9da
Updated docs
giuliov Nov 17, 2018
721e649
Add test to cleanup
giuliov Nov 17, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -332,6 +332,8 @@ ASALocalRun/

# MFractors (Xamarin productivity tool) working folder
.mfractor/

## Specific to this repo
secrets/
src/aggregator-cli/FunctionRuntime.zip
*.exe
Expand Down
60 changes: 34 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,32 +1,35 @@
# aggregator-cli

![](https://tfsaggregator.visualstudio.com/_apis/public/build/definitions/1cca877b-3e26-4880-b5b8-79e4b10fbfb4/16/badge)
[![Build status: master](https://dev.azure.com/TfsAggregator/Aggregator3/_apis/build/status/Aggregator3-CI?branchName=master)](https://dev.azure.com/TfsAggregator/Aggregator3/_build/latest?definitionId=16)
[![Build status](https://dev.azure.com/TfsAggregator/Aggregator3/_apis/build/status/Aggregator3-CI)](https://dev.azure.com/TfsAggregator/Aggregator3/_build/latest?definitionId=16)


This is the successor to TFS Aggregator.
The current Server Plugin version (2.x) will be maintained to support TFS.
The Web Service flavor will be discontinued in favor of this (its deployment and configuration was too complex for most users).

The main scenario for Aggregator (3.x) is supporting VSTS and the cloud scenario. It will work for TFS as long as it is reachable from Internet.
The main scenario for Aggregator (3.x) is supporting Azure DevOps and the cloud scenario. It will work for TFS as long as it is reachable from Internet.

> **This is an early version (alpha)**: we might change verbs and rule language before the final release!
> **This is an early version (beta)**: we might change verbs and rule language before the final release!
*Note*: The documentation is limited to this page and the content of the `doc` folder.

## Major features

- use of new REST API
- use of new Azure DevOps REST API
- simple deployment via CLI tool
- similar model for Rules
- Rule object model similar to v2

## Planned features

- Support for Deployment Slots for blue/green-style deployments
- OAuth support to avoid maintain access tokens
- Additional VSTS events
- Additional VSTS objects
- Additional Azure DevOps events
- Additional Azure DevOps objects

## How it works

An Aggregator Instance is an Azure Function Application in its own Resource Group,
sharing the same VSTS credential. You can have only one Application per Resource Group.
sharing the same Azure DevOps credential. You can have only one Application per Resource Group.
If the Resource Group does not exists, Aggregator will try to create it.
*Note*: The Instance name must be **unique** amongst all Aggregator Instances in Azure!

Expand All @@ -36,28 +39,28 @@ To work, it uses an Aggregator Runtime.
Aggregator checks its latest GitHub Release to ensure that Aggregator Runtime is up-to-date before uploading the function.
*Note*: We use [Azure Functions Runtime](https://docs.microsoft.com/en-us/azure/azure-functions/functions-versions) 2.0 for C# which is still in Preview.

An Aggregator Mapping is a VSTS Service Hook for a specific work item event that invokes an Aggregator Rule i.e. the Azure Function hosting the Rule code. VSTS saves the Azure Function Key in the Service Hook configuration.
An Aggregator Mapping is a Azure DevOps Service Hook for a specific work item event that invokes an Aggregator Rule i.e. the Azure Function hosting the Rule code. Azure DevOps saves the Azure Function Key in the Service Hook configuration.

You can deploy the same Rule in many Instances or map the same VSTS event to many Rules: it is up how to organize.
You can deploy the same Rule in many Instances or map the same Azure DevOps event to many Rules: it is up to you choosing the best way to organize.

## Authentication

You must instruct Aggregator which credential to use.
To do this, run the `login.azure` and `login.vsts` commands.
To do this, run the `login.azure` and `login.ado` commands.

To create the credentials, you need an Azure Service Principal and a VSTS Personal Access Token.
To create the credentials, you need an Azure Service Principal and a Azure DevOps Personal Access Token.

These documents will guide you
These documents will guide you in creating the credentials
* [Use portal to create an Azure Active Directory application and service principal that can access resources](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal)
* [Create personal access tokens to authenticate access](https://docs.microsoft.com/en-us/vsts/organizations/accounts/use-personal-access-tokens-to-authenticate?view=vsts#create-personal-access-tokens-to-authenticate-access).
* [Create personal access tokens to authenticate access](https://docs.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate).

Logon credentials are stored locally and expire after 2 hours.

The PAT is stored in the Azure Function settings: **whoever has access to the Resource Group can read it!**
The PAT is also stored in the Azure Function settings: **whoever has access to the Resource Group can read it!**

The Service Principal must have Contributor permission to the Azure Subscription.
In alternative, pre-create the `aggregator-` Resource Group in Azure and give the service account Contributor permission to the Resource Group.
The `instance` parameter prefixes `aggregator-` to identify the Resource Group.
The Service Principal must have Contributor permission to the Azure Subscription or, in alternative, pre-create the Resource Group in Azure and give the service account Contributor permission to the Resource Group.
![Permission on existing Resource Group](doc/contributor-on-rg.png)
If you go this route, remember add the `--resourceGroup` to all commands requiring an instance, otherwise the `instance` parameter adds an `aggregator-` prefixe to find the Resource Group.

## Usage

Expand All @@ -71,24 +74,29 @@ To run Aggregator use
Verb | Use
--------------------|----------------------------------------
logon.azure | Logon into Azure.
logon.vsts | Logon into Visual Studio Team Services.
logon.ado | Logon into Azure DevOps.
list.instances | Lists Aggregator instances.
install.instance | Creates a new Aggregator instance in Azure.
uninstall.instance | Destroy an Aggregator instance in Azure.
configure.instance | Configures an existing Aggregator instance.
list.rules | List the rule in existing Aggregator instance in Azure.
add.rule | Add a rule to existing Aggregator instance in Azure.
remove.rule | Remove a rule from existing Aggregator instance in Azure.
configure.rule | Change a rule configuration and code.
list.mappings | Lists mappings from existing VSTS Projects to Aggregator Rules.
map.rule | Maps an Aggregator Rule to existing VSTS Projects.
unmap.rule | Unmaps an Aggregator Rule from a VSTS Project.
configure.rule | Change a rule configuration.
update.rule | Update a rule code and/or runtime.
invoke.rule | Executes a rule locally or in an existing Aggregator instance.
list.mappings | Lists mappings from existing Azure DevOps Projects to Aggregator Rules.
map.rule | Maps an Aggregator Rule to existing Azure DevOps Projects.
unmap.rule | Unmaps an Aggregator Rule from a Azure DevOps Project.
help | Display more information on a specific command.
version | Display version information.

## Examples
You can see a few Command examples in [Sample Aggregator CLI usage](doc/command-examples.md), see also [Manual Tests](doc/test-matrix.md).

## Rule language

You can see a few Command examples in [Sample Aggregator CLI usage](doc/command-examples.md).
You can see a few Rule examples in [Rule Examples](doc/rule-examples.md).
See [Rule Language](doc/rule-language.md) for a list of objects and properties to use.
For examples see [Rule Examples](doc/rule-examples.md).

## Troubleshooting

Expand Down
23 changes: 23 additions & 0 deletions doc/build-and-test.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Build
Building locally requires
- Visual Studio 2017 15.8.9
- Azure Functions and Web Jobs Tools

# Debug

## Custom/development Aggregator runtime
In Visual Studio, `src\aggregator-function\Directory.Build.targets` will automatically package and copy the runtime needed by CLI.
You might have to change the version number in `src\aggregator-function\aggregator-manifest.ini` to force your local version.

You can also use the *Pack* right-click command on the `aggregator-function` project and make sure to copy the created zip into your CLI directory so it uploads the correct one when creating an instance.

## CLI
Set `aggregator-cli` as Start-up project
Use the Visual Studio Project properties to set the Command line arguments

## Runtime
Set `aggregator-function` as Start-up project
Use **Postman** or similar tool to send the request at http://localhost:7071/api/name_of_rule

# Integration tests
`git update-index --assume-unchanged src/integrationtests-cli/logon-data.json` and edit the file content
48 changes: 34 additions & 14 deletions doc/command-examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,40 +5,60 @@ Remember that the Instance name must be unique in Azure.
```
# logon
logon.azure --subscription 9c********08 --client 5a********b6 --password P@assword1 --tenant 3c********1d
logon.vsts --url https://someaccount.visualstudio.com --mode PAT --token 2**************************************q
logon.ado --url https://someaccount.visualstudio.com --mode PAT --token 2**************************************q

# create an Azure Function Application
install.instance --verbose --name my1 --location westeurope
install.instance --name my3 --resourceGroup myRG1 --location westeurope --requiredVersion latest
# search instances in the Azure subscription
list.instances
# search instances in the Azure Resource Group
list.instances --resourceGroup myRG1

# create three Azure Functions
# create two Azure Functions
add.rule --verbose --instance my1 --name test1 --file test\test1.rule
add.rule --verbose --instance my1 --name test2 --file test\test2.rule
add.rule --verbose --instance my1 --name test3 --file test\test3.rule
list.rules --verbose --instance my1

# adds two Service Hook to VSTS, each invoking a different rule
# create Azure Function in specified App and Resource Group
add.rule --verbose --instance my3 --resourceGroup myRG1 --name test3 --file test\test3.rule

# adds two Service Hook to Azure DevOps, each invoking a different rule
map.rule --verbose --project SampleProject --event workitem.created --instance my1 --rule test1
map.rule --verbose --project SampleProject --event workitem.updated --instance my1 --rule test2
map.rule --verbose --project SampleProject --event workitem.created --instance my3 --resourceGroup myRG1 --rule test3


list.mappings --verbose --instance my1
list.mappings --verbose --project SampleProject
list.mappings --instance my1 --project SampleProject

# disable a rule
# disable an existing rule
configure.rule --verbose --instance my1 --name test1 --disable
# re-enable a rule
configure.rule --verbose --instance my1 --name test1 --enable
# update the code of a rule
configure.rule --verbose --instance my1 --name test --update test.rule
# update the code and runtime of a rule
update.rule --verbose --instance my1 --name test1 --file test1.rule --requiredVersion 0.4.0
update.rule --verbose --instance my3 --resourceGroup myRG1 --name test3 --file test\test3.rule

# updates the VSTS credential stored by the rules
configure.instance --authentication
# updates the Azure DevOps credential stored in Azure Function and used by rules to connect back
configure.instance --name my1 --location westeurope --authentication
configure.instance --name my3 --resourceGroup myRG1 --location westeurope --authentication

# remove a Service Hook from VSTS
# remove a Service Hook from Azure DevOps
unmap.rule --verbose --event workitem.created --instance my1 --rule test1
unmap.rule --verbose --event workitem.updated --project SampleProject --instance my1 --rule test2
unmap.rule --verbose --project SampleProject --event workitem.created --instance my3 --resourceGroup myRG1 --rule test3

# deletes two Azure Functions
# deletes an Azure Function and all Service Hooks referring to it
remove.rule --verbose --instance my1 --name test1
remove.rule --verbose --instance my1 --name test2
remove.rule --verbose --instance my3 --resourceGroup myRG1 --name test3

# delete the Azure Function Application leaving the Service Hooks in place
uninstall.instance --name my1 --location westeurope --dont-remove-mappings
# delete the Azure Function Application and any Service Hooks referring to it
uninstall.instance --verbose --name my3 --resourceGroup myRG1 --location westeurope

# delete the Azure Function Application
uninstall.instance --verbose --name my1 --location westeurope
# run rule locally, no change is sent to Azure DevOps
invoke.rule --dryrun --project SampleProject --event workitem.created --workItemId 14 --local --source test\test2.rule
```
Binary file added doc/contributor-on-rg.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
18 changes: 14 additions & 4 deletions doc/rule-examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,14 @@ $"Hello { self.WorkItemType } #{ self.Id } - { self.Title }!"

This is more similar to classic TFS Aggregator.
It move a parent work item to Closed state, if all children are closed.
The major difference is the navigation: `Parent` and `Children` properties do not returns work items but relation. You have to explicitly query VSTS to retrieve the referenced work items.
The major difference is the navigation: `Parent` and `Children` properties do not returns work items but relation. You have to explicitly query Azure DevOps to retrieve the referenced work items.

```
string message = "";
if (self.Parent != null)
var parent = self.Parent;
if (parent != null)
{
var parent = store.GetWorkItem(self.Parent);
var children = store.GetWorkItems(parent.Children);
var children = parent.Children;
if (children.All(c => c.State == "Closed"))
{
parent.State = "Closed";
Expand All @@ -41,3 +41,13 @@ return message;
```
return self.PreviousRevision.PreviousRevision.Description;
```

# Create new Work Item
```
var parent = self;
var newChild = store.NewWorkItem("Task");
newChild.Title = "Brand new child";
parent.Relations.AddChild(newChild);

return parent.Title;
```
122 changes: 122 additions & 0 deletions doc/rule-language.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
# Directives

`.lang=C#`
`.language=Csharp`



# WorkItem Object

## Revisions
Navigate to previous versions of the work item.
`WorkItem PreviousRevision`
`IEnumerable<WorkItem> Revisions`

## Relations
Navigate to related work items.
`IEnumerable<WorkItemRelation> RelationLinks`
`WorkItemRelationCollection Relations`
`IEnumerable<WorkItemRelation> ChildrenLinks`
`IEnumerable<WorkItem> Children`
`WorkItemRelation ParentLink`
`WorkItem Parent`

## Links
Navigate links to non-workitem objects.
`IEnumerable<WorkItemRelation> RelatedLinks`
`IEnumerable<WorkItemRelation> Hyperlinks`
`int ExternalLinkCount`
`int HyperLinkCount`
`int RelatedLinkCount`

## Fields
Data fields of the work item.
`WorkItemId<int> Id` Read-only.
`int Rev` Read-only.
`string Url` Read-only.
`string WorkItemType` Read-only.
`string State`
`int AreaId`
`string AreaPath`
`IdentityRef AssignedTo`
`IdentityRef AuthorizedAs`
`IdentityRef ChangedBy`
`DateTime? ChangedDate`
`IdentityRef CreatedBy`
`DateTime? CreatedDate`
`string Description`
`string History`
`int IterationId`
`string IterationPath`
`string Reason`
`DateTime? RevisedDate`
`DateTime? AuthorizedDate`
`string TeamProject`
`string Tags`
`string Title`
`double Watermark` Read-only.
`bool IsDeleted` Read-only.
`bool IsReadOnly` Read-only, returns `true` if work item cannot be modified.
`bool IsNew` Read-only.
`bool IsDirty` Read-only, returns `true` if work item changed after retrieval.
`object this[string field]` access to non-core fields.

## Attachments
`int AttachedFileCount`



# WorkItemStore Object
Retrival, creation and removal of work items.

`WorkItem GetWorkItem(int id)`
`WorkItem GetWorkItem(WorkItemRelation item)`

`IList<WorkItem> GetWorkItems(IEnumerable<int> ids)`
`IList<WorkItem> GetWorkItems(IEnumerable<WorkItemRelation> collection)`

`WorkItemWrapper NewWorkItem(string workItemType)`



# WorkItemRelationCollection
Navigate and modify related objects.

`IEnumerator<WorkItemRelation> GetEnumerator()`
`Add(WorkItemRelation item)`
`AddChild(WorkItemWrapper child)`
`AddParent(WorkItemWrapper parent)`
`AddLink(string type, string url, string comment)`
`AddHyperlink(string url, string comment = null)`
`AddRelatedLink(WorkItem item, string comment = null)`
`AddRelatedLink(string url, string comment = null)`
`Clear()`
`bool Contains(WorkItemRelation item)`
`bool Remove(WorkItemRelation item)`
`int Count`
`bool IsReadOnly`



# WorkItemRelation

`string Title`
`string Rel`
`string Url`
`IDictionary<string, object> Attributes`



# IdentityRef
Represents a User identity.

`string DirectoryAlias`
`string DisplayName`
`string Id`
`string ImageUrl`
`bool Inactive`
`bool IsAadIdentity`
`bool IsContainer`
`string ProfileUrl`
`string UniqueName`
`string Url`
Binary file added doc/service-principal.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading