Enable Azure pipeline build & release for UI

[planetf1]

Before we can have this component as a dependency of the main egeria code base we need an Azure build pipeline

The build process needn't use maven, or be version synchronized to egeria, but it must be possible to build PRs, merge builds, and establish a build pipline

[planetf1]

We use azure pipelines for the egeria builds.

Azure pipelines has support for an npm task to assist in building the javascript apps. More details at https://docs.microsoft.com/en-us/azure/devops/pipelines/ecosystems/javascript?view=azure-devops&tabs=code

The basic tasks to invoke a build should be pretty simple - and we can launch this in various ways. for example for base egeria we do this on a PR (to verify it) and also after a merge. So we need to think if they are both the same, or if we want any differences such as running different tests. Most likely not.

I don't see any compelling reason to add maven/gradle if we're only building a single component

That first part should be easy.

We also have to decide how to distribute the artifacts.

For base egeria we have maven artifacts. For a PR build we do not publish these artifacts anywhere (technically they are saved, so we could look at them, but it's pretty hidden in azure)

For a merge build we publish to artifactory, as a snapshot release to a egeria-snapshot repo

When we RELEASE we have a seperate pipeline which currently does a full rebuild, and then publishes to a staging repository. Once complete we promote those artifacts to egeria-release, and then sync these with bintray, and onwards to maven central.

The staging approach has merits, though there are still deficiencies in our implementation of that process.

For another example we could look at what we do for docker.

There the merge build publishes DIRECTLY to docker hub. There is no PR build (but there is an issue open to create one, to aid in testing). The container image version matches the maven version is 2.4-SNAPSHOT .. then 2.4 when near released. Imperfect but good enough

As part of the above, the ODPi support team manage credentials for dockerhub and artifactory - and in the latter case artifacts are signed automatically.

We also have other options including publishing to Azure or github repos but haven't done this before. Or we could just store as a local build artifact & pick it up in the egeria build (we just need to know it's identity) though this feels a little closely coupled?

So it seems to me we can

• Quickly prototype a build only for PR and merge
• Decide what the 'release' process for the npm artifacts is - do we go straight to upstream repo like for docker (with a SNAPSHOT label) or take the more refined approach of going via staging.
• We may need to look at what support artifactory offers, and plan the relevant repos and/or evaluate github/azure as staging repos
• We need to request the linux foundation to get account(s) for npm repo & configure the secrets for our pipeline environment
• We need to figure out how to publish (and likely sign) the relevant npm artifact

@sarbull It would be useful tomorrow to understand what the new artifact is, how it's consumed, and how you see versioning work. We can then discuss the release process. I can show you what we have for egeria base. If it makes sense we can probably set you up with access to building the pipeline.

Part 2 -- which we should open up an issue on core egeria
With that done we need to think how the new component is pulled into the base egeria build. I am assuming it is via a npm module though it could be a file artifact of some kind. If that is run within the existing server chassis then it's likely no further changes are needed.

If we need a static content server, then we need to ensure we explain this in the docs & release notes (as it's a big change), and consider if we want to add a new Docker container. We already have one for presentation server, so this will be similar, and the build process is in place. We'll also likely need to create a new maven assembly with the content required for the container.

In all cases the docs need updating for the developer & end user

After we discuss I'll try and update this issue with a more precise breakdown of tasks - the above is a bit of a quick braindump to hit on a few topics we should discuss

cc: @davidradl @lpalashevski

[planetf1]

Summary from webconf discussion

• Currently static content isn't that useful as an npm package
• Could use npm to get content from github directly
• Easy to add to docker image/charts
• Users deploying UI themselves (and developers) will need to launch a local webserver -- README to cover this
• release notes to be updated on change
  but mostly
• Build process for PR and release to verify code is good
• Initial release cycle is simply to use a git tag on the code level
• consumers will specify git tag (ie V0.5.1) they wish to use
• for full release cycle will need npm registry credentials . May wish to use staging via artifactory (supports npm)

[planetf1]

Thanks @sarbull for the first pipeline definition. I've set these up in azure pipelines, so from now on you should just be able to edit the yaml.

The builds are under a new folder 'Egeria UI' - I also added a PR in addition to merge, and moved to .azure-pipelines to avoid confusion the real source with build instructions (same as egeria)

You may want to think about:

• whether you want to save any artifacts after the build (we don't do much of this in Egeria, but it could be useful ie for PD there)
• what you want to do for the docker build - in Egeria we do this as part of the 'merge' build. A seperate build - chained or timed are other options.
• once you have docker build done we can remove from egeria (though leave the compose/k8s pieces -- they just consume a docker image)
• whether you need any release pipeline - though I presume this comes later.

Suggest we close this once we're happy with the basics...

Nigel.

[planetf1]

Egeria UI builds -> https://dev.azure.com/ODPi/Egeria/_build?definitionScope=%5CEgeria%20UI

[planetf1]

PR verification seems to run ok

[planetf1]

[planetf1]

[sarbull]

we also need to have a release button in azure, that runs the release cycle [0] and pushes to the master branch.

[0] - https://github.com/odpi/egeria-ui#release-cycle

[planetf1]

I created an empty job at https://dev.azure.com/ODPi/Egeria/_release?_a=releases&view=mine&definitionId=11 if you'd like to experiment?

Hopefully you have all the necessary pipelines to work on the release.

Annoyingly these pipelines are GUI based - and NOT checked in to git.

You can also use the build pipelines to do something similar (we can easily register a new one of those if you check in).

I suggest taking a look to figure out what works for the UI? Happy to answer any questions (I'll be away wed/thu)

[planetf1]

Azure builds are working for PR and merging.

In egeria base we have migrated PR builds to github actions. will raise issue to discuss